I'm regularly finding myself in 6 vs 11 or other similarly insane team size differences. In casual games it doesn't seem crazy to just swap people from one team to another, if you cant get new players loading in immediately.