What do you want in a Ladder?

[TinCow]

There has been some interested expressed in an Org-hosted ladder using our current Tournament/Ladder utility. From my perspective, ladders are a bit different from tournaments because they are persistent and not really 'hosted' as much as tournaments are. They are long-term things and thus rely very heavily on the ladder players themselves to push the thing forward, with the organizers mainly existing to provide assistance whenever a problem is encountered. So, it seems to me that it is important for ladders to represent what the players themselves want in terms of competition. So, I'd like to hear some opinions on what kind of rules and restrictions we should use to make an Org ladder high-quality and useful to the TW community.

What kind of rules for matches? A full set like for tournaments or something more partial that relies more heavily on agreement between players? In both cases, what should the rules be? What kind of scoring (points) should we allow for wins/losses? Do we want to permit draws for stalemate situations?

Please post anything and everything you can think of that you would like to see in a ladder.

[00owl]

Something that ISN'T based solely on number of games played like the current version.

[TinCow]

Something that ISN'T based solely on number of games played like the current version.

Our system can rate people on the ladder in multiple ways, but what you're talking about would be best achieved with an ELO rating system, which we can do.

[AggonyKing]

Leagues is a must

[AMP]

The current system built in stw2 is very much flawed and yet people cry when the servers go down because their games aren't counted? Wow... just wow. And having p2p to add to the mess and people can't see that either? Wow... just wow.

Ladders shouldn't be based on just grinding your way to the top by the number of games played, lol. It should be who you played against (their skill rating) and you should go up in rating with win streaks (which can be people of all skill ratings including lower) and beating people with equal skill rating and higher skill rating. You should go down in skill rating when losing against people of same skill rating and lower skill rating. It really can't be that hard to come up with something.

[TinCow]

Leagues is a must

We can do that too, the ladder system has a built-in League option with promotion/relegation abilities. How many Leagues though, and how do we determine which league each player starts in?

Ignore these questions, I've just made a test league and it works a bit differently than I thought. The groups option is more of an NFL-style division that keeps smaller groups playing each other, it's not a promotion/relegation system like I thought. A system like that would link in well with a final tournament that would then take the winners of each group and pit them against each other for the league title. However, we could also just run the league with a single group so that everyone plays everyone. The main difference with the league is that the system assigns the games itself, so people do not get to pick who they play. In contrast, the ladder lets people pick their own opponents by challenging each other.

The biggest issue I see with this is making sure the games get played. Since the match-ups are automatically assigned, you could not avoid fighting an inactive player. We would need some kind of screening process to ensure that only serious players were allowed into the league, and there would have to be some means of policing the players to ensure that games get played. Maybe have two separate leagues? 'Hardcore' and 'Casual' (Samurai and Ashigaru?) Perhaps in the Hardcore/Samurai league, all entering players commit to playing their scheduled matches within 1 week, while Casual/Ashigaru league players get a longer time period? Failure to play the game by the time limit results in a Draw for both players maybe?

Ladders shouldn't be based on just grinding your way to the top by the number of games played, lol. It should be who you played against (their skill rating) and you should go up in rating with win streaks (which can be people of all skill ratings including lower) and beating people with equal skill rating and higher skill rating. You should go down in skill rating when losing against people of same skill rating and lower skill rating. It really can't be that hard to come up with something.

I'm pretty sure that's what the ELO system does. I think the number of points you gain for a victory depends on who you beat. If you beat someone with a much higher score than you, your rank goes up a lot. If you beat someone with a much lower score than you, your rank doesn't move much at all. That said, I haven't actually tested the system yet. I'll try and find time to run a test ladder this coming week to see precisely how the ELO rating system works.

[AggonyKing]

ideally this is what i was thinking:

Have different leagues that players can go in and out off and you can only play players from your league or the bottom portion of players from the league above you. (perhaps also top portion from the league below you)

that way players will get a chance to rise through leagues, improving their skills and will get points based on opponents. Basically, the way its done in SC2. We can also have leagues for different countries (or else one clan will just dominate the leagues >_> ) and then pit the top players from each league in a team contest to see which country or region has the best players that time around ^_^

[AMP]

I'm pretty sure that's what the ELO system does. I think the number of points you gain for a victory depends on who you beat. If you beat someone with a much higher score than you, your rank goes up a lot. If you beat someone with a much lower score than you, your rank doesn't move much at all. That said, I haven't actually tested the system yet. I'll try and find time to run a test ladder this coming week to see precisely how the ELO rating system works.

That's not 100% what I said though. You should only be able to gain skill rating beating lower ranks through your win streaks and also you lose skill rating when losing against people of equal skill rating and lower skill rating. There is abit more to it then that, but that's just the basic idea.

Since we're not gonna have thousands of players it can be something different anyway. Maybe what king is suggesting.

[TinCow]

ideally this is what i was thinking:

Have different leagues that players can go in and out off and you can only play players from your league or the bottom portion of players from the league above you. (perhaps also top portion from the league below you)

that way players will get a chance to rise through leagues, improving their skills and will get points based on opponents. Basically, the way its done in SC2. We can also have leagues for different countries (or else one clan will just dominate the leagues >_> ) and then pit the top players from each league in a team contest to see which country or region has the best players that time around ^_^

As I understand it, leagues and ladders have a major difference in our current system. For ladders, players 'challenge' another person in the ladder. If the other person accepts, a match is setup and they play it and report a result. All matches are done by this method, so players get to pick who they fight. For leagues, all matches are created by the system itself, which tells everyone who to play, though scheduling and result reporting are up to the players themselves. The idea of a league (as I understand it) is to ensure that everyone in the group plays everyone else in the group, so you get an even distribution of competition.

So, what you want sounds like it would actually be best accomplished with ladders and some manual organization. Essentially, have multiple ladders with rules about who can sign up for the ladders. Ladder competition then continues for a finite period of time, with players competing as per a normal ladder system. When the time limit expires, the top players from the bottom ladder move to the top ladder, and the bottom players from the top ladder move to the bottom ladder. This would have to be determined manually, but would not be complex. New players would enter at the bottom ladder and would have to win to move up to the top. Add in more ladders (2/3/4/etc) depending on how many tiers you want.

If we can get a small group of people, say 4 to 8, it might be a good idea to simply run quick tests of both systems. Create a test ladder and a test league, put all the volunteers in them, and have them use both system as they're designed to work. Afterwards, review what worked and what didn't, pick one system, and make any necessary tweaks, then create a full system for large-scale use.

That's not 100% what I said though. You should only be able to gain skill rating beating lower ranks through your win streaks and also you lose skill rating when losing against people of equal skill rating and lower skill rating. There is abit more to it then that, but that's just the basic idea.

Since we're not gonna have thousands of players it can be something different anyway. Maybe what king is suggesting.

Automation is the key, as it simply won't work if we've got to have someone doing the math on every match. So, we're kind of constrained by the systems available to us in the current utility. I could certainly ask the developer of the utility to add in another system, but I'd need to be really, really specific about the kind of methodology needed before making the request.

[00owl]

Doesn't the ELO account for losing games by removing skill points already? Unless I'm mis-understanding AMP I think that the ELO (as implemented in say, chess for example) causes people to lose ranking when they lose. They lose more ranking when they lose against someone who had smaller rank and lose less ranking when losing against someone of higher skill level. The ELO includes that sort of handicapping element already, however, that isn't to say it can't be abused, as is evidenced in other online games which utilize this scoring system where people will create games only play against noobs in order to get a high ranking.

If anything what I would like to see in a competitive atmosphere is the guarantee of sportsman-like conduct. I have found that when people get competitive it ruins the 'fun' atmosphere of the game because they end up resorting to verbal abuse and other non-game related means to win.

[TinCow]

That sounds right, but I've never actually used an ELO system so I can't say for sure. I'll run a test with it on monday or tuesday to get some solid answers.

Sportsman-like conduct would be an integral part of any official Org ladder/league.

[AMP]

You might be right about losing skill rating, just I couldn't understand how someone with so many losses and not that many more wins than me could be on top on the ladder, something isn't adding up. Just looked broken to me...

[Swoosh So]

The best league format is a football league (no amp not some oval ball u throw around) with relegation and promotion.

[AMP]

Just need to make sure we have the numbers or it really won't be worthwhile. That's why things like this work best built into the game so your going up against everyone that is playing the game, some day CA will understand.

[TinCow]

Just need to make sure we have the numbers or it really won't be worthwhile. That's why things like this work best built into the game so your going up against everyone that is playing the game, some day CA will understand.

What kind of numbers do you think we need at a minimum? 20+? 50+?

If it's successful, I'm sure it will draw more later, but establishing a good starting level would be mandatory.

[TinCow]

Ok, I've just run a test of the ELO ranking ladder system, and here are the results:

It was a 5 person test ladder, with the players: Tester A, Tester B, Tester C, Tester D, Tester E

All players started with an ELO rating of 1000.

Game 1, Tester A (1000) defeats Tester B (1000).
Result, Tester A gains 25 ELO points, Tester B loses 25 ELO points.

Game 2, Tester C (1000) defeats Tester D (1000).
Result, Tester C gains 25 ELO points, Tester D loses 25 ELO points.

Game 3, Tester A (1025) defeats Tester C (1025).
Result, Tester A gains 25 ELO points, Tester C loses 25 ELO points.

Game 4, Tester A (1050) defeats Tester D (975).
Result, Tester A gains 18 ELO points, Tester B loses 18 ELO points.

Game 5, Tester D (957) defeats Tester A (1068).
Result, Tester A loses 35 ELO points, Tester D gains 35 ELO points.

Game 6, Tester E (1000) defeats Tester C (1000).
Result, Tester E gains 25 ELO points, Tester C loses 25 ELO points.

Game 7, Tester A (1033) defeats Tester E (1025).
Result, Tester A gains 24 ELO points, Tester E loses 23 ELO points.

So, every result adds points to the winner and subtracts them from the loser. The point change varies depending on the relative ranks of the people who are fighting. If both players have the same ELO rating, the point change is exactly 25. However, when the players do not have the same ELO rating, the points are different. If the winner is the player with the higher rating, they gain fewer than 25 points. If the winner is the player with the lower rating, they gain more than 25 points. The point changes are not always even, as represented by Game 7, and they appear to scale relative to the ranks of the players.

You can see the current state of this test game here.

Overall, I think this looks like a good ranking system, as it takes into account each person's rank when determining their scores. It is far more important about who you beat than how many wins you have. Does this look like a ranking system that would be fair?

[00owl]

You might be right about losing skill rating, just I couldn't understand how someone with so many losses and not that many more wins than me could be on top on the ladder, something isn't adding up. Just looked broken to me...

Yeah I think the current ladder version that is live in-game, is a mixture of the ELO and some messed up half-baked system.

Though it is technically possible in the ELO system that someone with more losses could be higher than someone with more wins if the wins of the first person, though few and far between, were all against the highest ranking people, and the wins of the second person were against the lowest ranking people.

200 wins at 1 point each is less than 50 wins at 10 points each.

EDIT: *Historical Note*
I believe that the ELO was designed specifically for, and is still used by, professional chess leagues.

[Swoosh So]

why not just do 3 points for a win?

[AMP]

What kind of numbers do you think we need at a minimum? 20+? 50+?

If it's successful, I'm sure it will draw more later, but establishing a good starting level would be mandatory.

That would be a good start as long as we can draw more.

Our own working ELO system would be nice, but it's also nice to have replays of matches, know ones wins/losses, know if they are on a win streak and know who they played and when and how many times. I don't wanna look at someone and just see a rating on them and that's it... because if there can be so much more then there should be. When you look someone up it should have their ELO rating and stats... some or all the ones I listed.

The thing that makes it a grind is where someone who plays a ton more has a better chance to be on top of course, that's why you need stats as well to keep it more real and interesting. You also can't have someone milking out someone else playing them a tons, so there should be a cap on games vs the same person and after a certain time has passed you can play them again. Maybe a cap of 5 games vs the same person in a couple months and only can start playing that same person again as the last game vs him/her has passed that date and then not play again until the next game has passed that date etc.

Maybe have a cap where each player has to like 10 different people best of 3 and no more after that. Then you can put people in different tiers and continue the cycle having top players move up and bottom players move down in tiers. Something like the clan meta just ALOT better. :p

Maybe later down the road once it is establisted and running smooth to start offering prizes to help draw more people in and make it more rewarding to play.

Yeah I think the current ladder version that is live in-game, is a mixture of the ELO and some messed up half-baked system.

Though it is technically possible in the ELO system that someone with more losses could be higher than someone with more wins if the wins of the first person, though few and far between, were all against the highest ranking people, and the wins of the second person were against the lowest ranking people.

200 wins at 1 point each is less than 50 wins at 10 points each.

EDIT: *Historical Note*
I believe that the ELO was designed specifically for, and is still used by, professional chess leagues.

Whatever they have now isn't that great and of course the exploiting isn't helping and they lack so much info. I wanna be able to look up people and see stats like who they played , when they played them, how many times, results of the matches, and replays.

p2p = fail

[TinCow]

That would be a good start as long as we can draw more.

Our own working ELO system would be nice, but it's also nice to have replays of matches, know ones wins/losses, know if they are on a win streak and know who they played and when and how many times. I don't wanna look at someone and just see a rating on them and that's it... because if there can be so much more then there should be. When you look someone up it should have their ELO rating and stats... some or all the ones I listed.

If you look at the current system, the displayed stats are: ELO rating, Total Matches, Wins, Losses, Streaks, and Last Activity. There are also stats for Draws, Points Scored, Points Conceded, and Experience, but I have turned them off since they don't seem very relevant to what we need. You can also see a list of the recent matches, showing who fought who and what the score was. There's even a comment ability for each match for discussions about specific results. That would be a good place to link replays and videos of the matches. I'll see if I can add an ability to see a list of ALL match results, not just recent ones.

What other stats do you want?

The thing that makes it a grind is where someone who plays a ton more has a better chance to be on top of course, that's why you need stats as well to keep it more real and interesting. You also can't have someone milking out someone else playing them a tons, so there should be a cap on games vs the same person and after a certain time has passed you can play them again. Maybe a cap of 5 games vs the same person in a couple months and only can start playing that same person again as the last game vs him/her has passed that date and then not play again until the next game has passed that date etc.

In the current setup, players cannot challenge each other more than once every 24 hours. However, we can change that to any time period. Maybe no more than once per week against the same player?

Maybe have a cap where each player has to like 10 different people best of 3 and no more after that. Then you can put people in different tiers and continue the cycle having top players move up and bottom players move down in tiers. Something like the clan meta just ALOT better. :p

I've looked at the system more closely, and there actually is a built-in promotion/demotion function for leagues, so we don't even have to do that by hand. If we turn it on, at the end of every 'season' of the league, the system will automatically promote and demote the top and bottom players in each 'group' within the league. The current options allow for promotion/demotion of the top/bottom 1, 2, 3, or 4 players in each group, with the new groups starting in the next 'season.' In order to make that meaningful, I think we'd have to make seasons about a month long. If people are sufficiently active, that would be enough time for them to get a large number of games in, but not so long that promotion/demotion never happens.

As I said before though, leagues and ladders behave differently on who plays who. In leagues, all matches are automatically assigned to ensure everyone plays everyone. In a ladder, players are free to pick who they play. If you really want a completely fair setup, a league is probably the way to go.

Maybe later down the road once it is establisted and running smooth to start offering prizes to help draw more people in and make it more rewarding to play.

That's quite easy to do, and the Org would happily supply prizes if there was serious competition going. In fact, we can actually link small tournaments directly into the League/ladder itself. We can do even do small 'championship' tournaments of about 4 to 8 people at the end of each season or something, and award bonus ELO points to the winner, along with an official prize.

[00owl]

This sounds like a really decent league/ladder system they got going on here.

[TinCow]

I just noticed another difference between ladders and leagues: in ladders, new players can join at any time, but in leagues new players can only join at the start of each 'season.' This is required because otherwise it would be impossible to make sure all players in a league group played each other.

Basically, the ladder is the easiest system to use, but the league is the better judge of skill and is more fair. A league would produce a very high quality result indicating which player really was the best performer, but the mandatory scheduling system would require a high level of activity from the players to ensure that the games really did get played. We would have to require that each player commit to being very active and playing games promptly. A league would truly be for the serious competitors, and all casual players would need to be kept out to keep it running smoothly. Ladders would be better for a more relaxed style of play with less emphasis on frequent gameplay.

Based on what people have been saying, I think the League is really what we want here as that's what is not being provided by the official ladder. If the rest of you think so too, we can start drawing up some rules for this thing. Once those are in place, we can go out recruiting to get a good group of serious players involved. I would recommend that the first 'season' not involve any groups, as we don't really have an objective way to evaluate player levels at the moment. After the first season is complete, we could then split it into groups and begin the promotion/demotion changes from then on out, as there would be a good baseline skill level established.

[lampekap]

I see some of the objections with the in game system but a new ladder is not going to change anything. The ladder right now actually works according to a system similar to ELO. It will always be a matter of how many games you play. Maybe some maths are a little different and maybe losses aren't punished strongly enough to your tastes but the system is the same.

Maybe the game should incorporate mini tournaments of 4/8/12 players with games that are fought in concession at the same time. And then make a ladder for wins like that.

[AMP]

Yes a ladder like ELO will always be a matter of games played, just a constant grind and not a real good indicator of skill.

League is the way to go for sure. If you put a cap on the amount of games someone can play each season and have it where that player can't be regulated until he's played the cap amount of games, then that's a more fair way. So really anyone can join at anytime, just will have to wait for maybe two seasons if he can't play his cap amount of games for regulation. In the end if we get enough players we'll probably only have maybe 3 tiers max, which is fine. The number of tiers should depend on the number of players.

The other stats which would be nice to are replays maybe linked to recent games played and player stat info of every game played such as name of player, the date, and end result screen shot. At the very least a link were you can add replays of your games for people to download and watch if they want.

I wouldn't award ELO points though for winning tournaments, prizes are nice, but some kind of icon for tournament victories would be better than ELO points. I'd leave that strictly to games played only.

Really it's looking good so far, just wish CA had something like this with *dedicated servers*. :p

[TinCow]

League is the way to go for sure. If you put a cap on the amount of games someone can play each season and have it where that player can't be regulated until he's played the cap amount of games, then that's a more fair way. So really anyone can join at anytime, just will have to wait for maybe two seasons if he can't play his cap amount of games for regulation. In the end if we get enough players we'll probably only have maybe 3 tiers max, which is fine. The number of tiers should depend on the number of players.

If we use a league, players will not have any options as to who they play or how many games they play. As I understand it, all players will play a single game against every single other player in their tier. When that is completed, the 'season' is done. Players cannot play additional games beyond those games within the league, just as sports teams do not get to pick and choose who they play during their season. It's all set out in advance and everyone has a pre-defined schedule.

The other stats which would be nice to are replays maybe linked to recent games played and player stat info of every game played such as name of player, the date, and end result screen shot. At the very least a link were you can add replays of your games for people to download and watch if they want.

The system has a built-in screenshot submission utility to allow screenshots of the final results screen to be submitted when the result is submitted. Replays would require a bit more work, but not much. Just upload them on the Org or somewhere else on the net, and then post a link in the comments section for that game. In order for that to work fairly, we'd need to make replay posting mandatory for the league and have Gamemasters regularly reminding people to upload their replays.

Really it's looking good so far, just wish CA had something like this with *dedicated servers*. :p

Honestly, I think the key to a good league is just going to be getting a solid group of high-quality players who commit to playing regularly. No matter how good the system is, it will never go anywhere unless it's got a serious enough level of competition to draw players in. I'll post a draft of prospective rules later today, so we can start on discussion there. After I do, please direct any players to it that you think might fit that description. It'll be very important to have the rules solid and agreed upon by a good group of the players, so that it's considered a proper high-level competition.

[TinCow]

Urgh... I don't know how I missed this before, but ELO rankings are not used in the Leagues, only in the Ladders. This actually makes sense, because Leagues already have a built-in fairness function in them, because everyone plays everyone else exactly the same number of times, so simply tracking wins and losses is an accurate measurement. ELO is only used in Ladders, because in Ladders players will not end up playing the exact same people, so a different evaluation system is still needed.

[00owl]

It sounds like the leagues are designed for seeding people for tournaments. It is all a giant round robin. Does everyone play everyone else in their league exactly once? Or can that number be changed?

[TinCow]

It sounds like the leagues are designed for seeding people for tournaments. It is all a giant round robin. Does everyone play everyone else in their league exactly once? Or can that number be changed?

Currently, exactly once. There's a function in there for setting a time limit on 'Round 1', and from what I can tell starting a new round should open up a second wave of games. However, there doesn't appear to be a way to actually start Round 2. While researching this, I actually just came across a bigger problem. There's no way for me to choose which players get added into which tiers inside the League; all players are assigned randomly. That right there means we cannot use this system, as it would allow some of the later players to join in on tiers higher than the starting tier. The League feature of this utility is pretty new and is still being developed, so I'll request this as an additional feature. However, I wouldn't count on it being ready in the time frame we need.

I still think the League system is the way to go, but I think we'll have to use the Ladder part of the tool along with some custom organization. That's fine, and pretty easy to do actually. I'll just create one ladder for each tier of the League and will manually do the promotion/relegation at the end of every season. Not difficult at all. That also frees us from the constraint of forced match-making, and eliminates the problem of a non-responsive player holding up a lot of matches. If someone isn't playing, they simply won't gain points. Seems like a better way of implementing what we want, really. It'll still be a league, but with the greater flexibility provided by the ladder part of the utility.

[AMP]

I was trying to think of way to combine the two but do as you like, I was just throwing out ideas.

Playing each player only once and not best out of 3 isn't the best I think and let alone having to play everyone in their tier especially if you have alot of people in the tier. That's why I said a cap amount of players in each tier before that player can be regulated at the end of a season. Having an ELO rating and win/losses is best because it gives an idea if that player has more or less losses/wins against higher or lower ELO opponents and allows people to be ranked by ELO rating in their tier.

Your way works for course, but maybe not the most satisfying way. I'll still play in it reguardless.

[TinCow]

I was trying to think of way to combine the two but do as you like, I was just throwing out ideas.

Playing each player only once and not best out of 3 isn't the best I think and let alone having to play everyone in their tier especially if you have alot of people in the tier. That's why I said a cap amount of players in each tier before that player can be regulated at the end of a season. Having an ELO rating and win/losses is best because it gives an idea if that player has more or less losses/wins against higher or lower ELO opponents and allows people to be ranked by ELO rating in their tier.

Your way works for course, but maybe not the most satisfying way. I'll still play in it reguardless.

The hybrid system I'm thinking of now actually gives us a ton of freedom and opens up a lot of the options you were asking about. Since I'll handle promotion/relegation manually, we can easily add in that minimum number of games played rule.

You're actually getting confused by the 'best out of 3' and 'playing only once' aspects of the system. Each match between two players would be best out of 3. First to two wins claims victory in the match, which counts as a single win on the tier. We can then use the specific result of the three games (either 2-0 or 2-1) as the 'score', which will give additional information about how the match went. A 2-0 win really does mean something different from a 2-1 win, and it'll be nice to have that officially recorded.

Regarding ELO and win/loss, we can have both stats recorded, but we'll have to pick one of the two methods as the official ranking method. Is ELO the proper method for that, or is it win/loss?