Every Champion Ladder without changes is a wasted season

@Arne said in Every Champion Ladder without changes is a wasted season:

Have you published those sims yet? If so: where can I find them? If not: Could you provide them in some way?

No, I didn't - they were quite some time ago, and the sims simply generate datasets which at the time nobody but me was actually working with. I've been working on a new sim system so next time I'll keep the datasets for posting.

@Arne said in Every Champion Ladder without changes is a wasted season:

I would be interested to see what you did. I did some simple simulations too and went on with doing preparations/concepts for more complex ones, but I stopped at one point due to the lack of time

The sims I do are programs which generate a (large) number of virtual teams and then run through a (large) number of matchmaking iterations generating match results and rankings, writing the records in CSV format to disk, updating the virtual team records in memory, and repeating the process until it has generated the specified amount of match data.

The systems use a combination of data-derived distributions and, frankly, semi-arbitrary suppositions (eg, the amount of influence that skill, or TV, or whatever has on outcome) to generate outcomes... but that's not really a problem so long as all of those values are constant between models that are going to be compared.

Given that the numeric results of sims can't be directly plugged into the real world you probably won't see me reporting those... I only ever say "the sims suggested that..."

Maybe a bit of a necro to this thread, but as someone who has historically been against TV++, I think I know why after reading through a bunch of these posts and wanted to add my piece. The problem is the ranking system, not TV++ itself.

Lets take the two main systems I see. The 'sports team' system, used by for example football/soccer, and the 'online gaming' system, used by most competitive online ranked games.

Football/Soccer - teams are 'uneven' going into the game, like BB without TV++. Skill means one team will be more likely to win than another, even with similar 'players', like BB without TV++ (can't just buy good players without a good tactician/manager and win everything, but you can win more than you used to. Just like a powerful team in BB). Rankings are based on wins/losses and points ascribed to them, like BB without TV++.

Most competitive online games - Matchmaking tries to balance things to 50% winrates, through arbitrary rules we won't go into as they are different per game, but all aim for the same end result. Good, meaningful matchups. BBs TV++ would mean you can artificially -create- players who are harder to beat, meaning matches happen much more regularly, but would likely still default to matching with similar skill players
when there is no team difference. This covers the first two points above in the 'football' style approach. And now comes the big one. RANKINGS. These systems use some form of league/ranking ladder that shows -points- or -rank-, not win record. So Mr Fancypants McWinsalot may go 14/14/14, but he did it against people the system though should be beating him regularly, and so he is higher ranked than someone who goes 14/14/14 -against worse opponents-. Mr McWinsalot has the same result, but from 'harder' matches. Someone with 14/14/14 while being bad at the game would go something like 0/4/38 with the same TV++ adjustments that Mr McWinsalot had. That shows skill on his part. He is better, even with the same results. He is higher ranked.

This is where things fall apart for BB. As good as TV++ might be for seeing who is -truly- the best, because even with all the boosts to their TV, a new coach is going to lose a LOT of games -before- they become competitive through sheer TV bonuses, and a great coach will win a bunch before the TV bonus they give away outclasses their skill, it does not address rankings. And everyone is looking at the system as it is now, a football based system. And tbh that is rubbish. You don't want to see Mr Worldbest at 14/14/14 record. It makes no sense. Because you aren't seeing that those results were against people who were good enough. You are just seeing results. Instead, you need to adopt a completely new ranking system to make TV++ acceptable. /most/ of the coaches who hate the idea of 50% winrates would probably be on board with a ranking system where they see they are in the top 5% of coaches, instead of team win rates. And the great thing is, with a ranking system, you could be ranked based on your overall performance, not just with one random team you got great rolls on.

So yeah, long story short, in my mind TV++ isn't actually the hard sell, it is a new ranking system. Get that sorted and TV++ suddenly makes a ton more sense to me, and will allow actually seeing who the best is, and giving the top coaches a reason to play to try and climb the rankings against their peers, instead of just picking a race with no great record to try and push for a qualifying spot.

Mr Fancypants McWinsalot will not be 14-14-14. He'll have a pile of wins in order for him to have a high zSum (the TV+ bonus) meaning he'll have far more wins than losses. When he eventually reaches the point that most people are competitive against him he will then start to be at a 50% win rate. So, for example, he'll go 10-0-0 then start to lose games, eventually getting a 50% rate at maybe 20-3-2 (zSum +17). He'll still be in the top few teams with that record.

It's worth being clear that TV+ is a team metric, not a coach metric. Every time Mr Fancypants McWinsalot starts a team it starts with a TV+ rating of 0. The fact that he is Mr Fancypants McWinsalot has no effect.

And the great thing is, with a ranking system, you could be ranked based on your overall performance, not just with one random team you got great rolls on.

No, you couldn't. The base issue with this is we are ranking teams to qualify for the playoffs, not coaches. Any system which ranks teams but uses a coach rating to match people will enable people to tank their coach rating in order to get easier matchups with one team, sacrificing another.

Indeed, which is exactly why I say you would need to implement a new ranking system -first-, before TV++ would fully make sense to take over, based on the general objections.

The system itself works perfectly well for what it does. Yes, the random win/draw/loss numbers I put were '50%', when that wouldn't start that way, but the point is more that seeing your 50% win rate (once you have earned the TV++ to make it that way) is what most people hate. "If I'm better, I should get the wins!"

But to win over people and get them on board, and to have it make sense overall for the game, you would really need to make actual rankings. So a -team- can be 20-3-2 (zSum +17) as you mention, but a -coach- could be Master Rank #7 (or whatever naming/numbering system picked).

As to how that ranking system would work, that would be for people who have been involved in game development more than me, and understand how best to do this. But from my point of view, if there was an overall coach ranking (not just a single team/league ranking) it would greatly ease the transition to getting even games through something like TV++, as the win rate isn't the only thing that matters to show how good you are.

Currently, someone who goes 20-3-2 looks like a pretty good coach (depending on the team), with TV++ they wouldn't necessarily look as good. That is peoples problem (ignoring the fact that -everyone else- would also have the same problem, so you should still be getting better rankings than average joes) and having a coach ranking would mean people have something to make playing those games/teams more valuable. I'm sure it would need some kind of weighting on ranking points to prevent just making new teams every 5 games to never get too big a negative, but again, that is for game devs to worry about. I'm just talking about the perceived problems the community has against TV++, which often end in 'well it is just shit' arguments because neither side are talking about the same thing.

Well the whole point of the Champion Ladder is to qualify teams, not coaches. I personally wouldn't have an issue with a coach ranking (however you may want to do it) being calculated alongside a team ranking, but it won't have anything to do with ranking or with qualifying because that's a team thing.

The Champion Ladder format is about finding the best performing team to qualify, not the best coach. If there's another World Cup that might be a different matter, and I recently helped rank an open format tournament on OCC where we ranked by coach rather than by team (using a similar formula, just altering a few variables), so it can be done, but that's not this format.

Ideally, ofc, we'd be able to adjust the TV+ weighting (currently set at 50) so tournaments can set the weighting to 0 (making it more football-style), or we'd be able to choose between coach ranking and team ranking. We can't, though, and there is little appetite from the devs for making things optional.

Yep, that seems to match up with what I was thinking and as always 'optional' fixes everyones problems, but does seem to be something unwanted by the devs.

I possibly didn't emphasise it enough as this is the CCL forum, it is just where I found the TV++ discussion. The overall ranking wouldn't be something I foresee being used for a specific CCL team ranking for a single season, it would be an overarching thing (as I imagine it). You could do it many ways. Fully based on entire coaching history (from implementation), a Coach Championship Ranking which carries over season to season and ranks coaches overall for all teams played in the champ ladder, a 'official league/ladder' ranking, for all games played in any official manner, which maybe feeds into a new world cup invitational, making sure the truly best of the best go into it, no matter their preferred ladder etc.

Essentially though, give people a way to see how good a coach they are in general, and they likely will care a lot less about how good they are with one single team run as a point of prestige. Sure, try to win the individual thing, it is something to aim towards, but performing well even if you don't win the tournament then matters to them.

Ok, that makes more sense to me. While I can't implement anything in-game there are places such a coach ranking system might be of use. I'll have a think and a chat with a few people and something unofficial might be possible.

Makes me think Jamtoast is asking for an "ELO type" ranking for coaches independant of the team ladder.

There was a time OCC (when I was a member of this league) did that by BB1 Race, I believe this was an awesome features.

I believe lots of people would love it if it were implemented.

@JamToast said in Every Champion Ladder without changes is a wasted season:

Maybe a bit of a necro to this thread, but as someone who has historically been against TV++, I think I know why after reading through a bunch of these posts and wanted to add my piece. The problem is the ranking system, not TV++ itself.

If you mean full TVPlus, then somehow you missed the dozens of times it has been mentioned that full TVPlus does not use W/D/L records for its rankings. You can see the suggested ranking system (how to calculate a team's rank on the leaderboard) under TVPlus in this thread from before TVPlus matchmaking was introduced to BB2 in 2016 which details the formula.

@JamToast said in Every Champion Ladder without changes is a wasted season:

Indeed, which is exactly why I say you would need to implement a new ranking system -first-, before TV++ would fully make sense to take over, based on the general objections.

You'd have to implement both simultaneously, actually, since the necessary ranking system wouldn't rank teams properly if you tried using their records that were created under a non full TVPlus system..

@JRCO said in Every Champion Ladder without changes is a wasted season:

Makes me think Jamtoast is asking for an "ELO type" ranking for coaches independant of the team ladder.
There was a time OCC (when I was a member of this league) did that by BB1 Race, I believe this was an awesome features.

OCC did track ELO ratings which is how we were able to look at whether ELO ratings would be a good means of matchmaking, and how ELO rating fared in terms of predicting match outcomes relative to things like TV, FF, TVPlus, etc... and found ELO was worse than just FF, which was worse than zSum which was worse than TVPlus rating.

@VoodooMike

I wasn't suggesting to use Elo to pair players , but as a feature that can be an indication how a player master (or not) a certain race. I believe it was great move from OCC to track that Elo as within a race, at least for the race I did play (mostly vamp), I believe it gave a good indication of hirarchy within a race. Bantha and Kjelstad(not sure about the exact name) where at the top of Vamp Elo and I believe it was not by chance.

@JRCO said in Every Champion Ladder without changes is a wasted season:

I wasn't suggesting to use Elo to pair players , but as a feature that can be an indication how a player master (or not) a certain race.

Well, the point of a rating like ELO is that it is supposed to allow you to gauge who is more likely to win in a match between two people... the one with the higher rating is more likely to win. Thus, ELO rating is meant to be a predictor of match outcome.. or its of no value to anybody. When we looked at OCC's ELO ratings, they were worse predictors of outcome than those specified measures.

Even if you don't intent to match on it, using any rating that is itself a better predictor of match outcome is better for anything you'd want to use ELO for, including ranking people on a list of "who is best with what".

@VoodooMike

Wel I still Believe that Both Bantha or Kjesltad or AndyDavo who had better Elo than me with Vampire had more chance than me to prevail when playing with Vampires and as such I believe these Elo Ranking were not such a poor indicator. If I recall well Bantha Managed a 1700 ELO with Vampires and was regulary playing in OCC division 1 (He finished second one season).

Even in Chess ELO do not tell the whole story, ie I managed to beat Player with a higher ELO than me and sometimes did lose or Draw with player with Elo inferior than Mine.

I still believe Elo is good indicator if you use it by Race. I also believe it is a good tool to motivate player to play and it has the advantage over ranking that ELO can reflect performance over several seasons and not just the current season.

@JRCO said in Every Champion Ladder without changes is a wasted season:

Wel I still Believe that Both Bantha or Kjesltad or AndyDavo who had better Elo than me with Vampire had more chance than me to prevail when playing with Vampires and as such I believe these Elo Ranking were not such a poor indicator.

They do - I'm not saying that ELO isn't an indicator of who is better, I'm saying that we analyzed the OCC data and compared various predictive measures, and there were multiple measures that were better than ELO at predicting match outcomes... which means those other measures would be better metrics to use for anything you're thinking of... so why bother with ELO in the first place?

@JRCO said in Every Champion Ladder without changes is a wasted season:

I still believe Elo is good indicator if you use it by Race

Sure, but there's better... in fact, there's better that's easier to calculate.

@VoodooMike

You keep telling there is better Than Elo. But maybe I have missed a thing, but I do not see any alternative implemented either.

Just curious : What better alternative do you suggest should be used and could be a measure valid over a several season span and as easy to understand ?

last edited by JRCO

@JRCO said in Every Champion Ladder without changes is a wasted season:

You keep telling there is better Than Elo. But maybe I have missed a thing, but I do not see any alternative implemented either.

I told you a few of the things that are better than Elo: FF, zSum, TVPlus rating. All of these are "implemented", they're just not what the rankings are being based on. FF is already tracked for each team, and both zSum and TVPlus ratings can be calculated in place (meaning they don't need to be calculated every match to provide an accurate number later on).

@JRCO said in Every Champion Ladder without changes is a wasted season:

Just curious : What better alternative do you suggest should be used and could be a measure valid over a several season span and as easy to understand ?

Any of them works. If you're talking about ranking by coach rather than by individual teams then you just average whatever metric across all the teams, preferably weighted by the number of matches each team played. Even win% is easily good enough so long as you're talking about a team with enough matches under its belt.

Elo is not a rating that is easy to back-calculate, as the change to each team's rating depends on each team's rating at the time of the match. FF is already tracked because it's part of the game... zSum is just (Wins - Losses)... TVPlus rating is just TV + (50 x zSum).

Not sure if it's worth mentioning that there is a very strong correlation between zSum and ranking points in the top 50, top 100, top 500 and overall team data. You'd expect there to be a strong correlation overall, but it is even stronger at 0.97 in the top 50 and top 100 and at 0.94 in the top 500.

@dode74 said in Every Champion Ladder without changes is a wasted season:

Not sure if it's worth mentioning that there is a very strong correlation between zSum and ranking points in the top 50, top 100, top 500 and overall team data. You'd expect there to be a strong correlation overall, but it is even stronger at 0.97 in the top 50 and top 100 and at 0.94 in the top 500.

If the correlation coefficient is that strong (0.9+ is ridiculously strong) at those levels then it suggests would could outright do away with the ranking system and rank solely on zSum. I thought it would overly benefit teams with high numbers of games played, but 0.97 is practically identity, and that's at the range where we actually care about the rankings.

You lose some granularity with zSum. 11-2-2 and 12-2-3 are very slightly differentiated in the rankings but are not at all with zSum. There are (as of the data dump I took last week), within the top 20, 4 teams with zSums of 10 which are all on slightly different rankpoints.

let's say you are right and these rating are more accurate and appropriate than elo (I can't argue as I am not a mathematician I have only a vague idea and understanding how these alternative ratings are calculated), They may be implemented but they not apparent. When I meet a coach I don't Know my Szum, I don't his.

All of my saying is that I believe it would be awesome if a rating of any coach performance be it Elo or something else appears ingame.

I still prefer ELo because my understanding is szum or TV+ or TV++ are linked to a team and when you start a new team or start a new season you start again at zero. There is no persistence overtime like the Elo rating in Chess, so currently we have no apparent measurement of coach performances over several season.

last edited by JRCO

While I can see the merit in having a persistent coach ranking score which is entirely separate from matching methods, I would reiterate that the purpose of CCL is not to find the best coach, but to find the best performing team over the course of a season.

I'll also point out that there are downsides to having a displayed coach rating. I know there have been cases on FUMBBL of matches being dropped when one person found out their opponent was particularly highly rated, for example: the equivalent of dropping with a large TV difference, if you like.

Looks like your connection to Focus Home Interactive - Official Forums was lost, please wait while we try to reconnect.