Same! I have Vega 64 and it started to happen to me too! When i first bought the game. It didnt do it at all! It started to glitch about two days ago and i havent found a fix for it.
First, want to say the game is absolutely fantastic! Second, I want to leave feedback on the ranked matchmaking system.
The ranked matchmaking system is less than adequate for ranked play. I've played multiple matches with a team mate that has disconnected and did not have the option to reconnect to the match. Adding a reconnect option for disconnecting players would be ideal. Also, I feel that the ranked system doesnt rank up /derank correctly. I played multiple matches some won and some lost. I was Gold 4 at the beginning of the session and deranked all the way to Gold 1. Granted I did lose a couple matches but also won several. I think its something worth looking into!