Can join friend's party but not game.

Inviting a friend to your game through Steam invites them to your party but not your game. As well as joining a game as the leader of a part doesn't bring the members of your party with you.

I agree, I've had times where I got disconnected and couldn't join my friends game back

Looks like your connection to Focus Home Interactive - Official Forums was lost, please wait while we try to reconnect.