So, to be clear, you're not able to do this in co-op currently? I ask because my friends and I have been waiting for I:S to release on console ever since it was being developed for PC. Just not sure how these mechanics work exactly and NWI hasn't been forthcoming with many details on what we can expect on console this August.
For co-op, as with nearly every other co-op mode we play, we really have zero interest in playing with randoms. If I:S doesn't already include the option to only play with friends, it should be added IMO.
For some reason, I already thought it did.
I don't believe there is currently any way to play on public servers with only friends, unless you happen to have a group of friends that will fill the server.
Your only option for now would be to create a private server and only give your friends the password, I've no idea how that may work on consoles though.