Are We Sure the ELO System is Working?

degraafination · July 2014

Because here I am PVPing solo. I queued randomly with three other very well geared players (all of us were in the top 10 pages on the Leaderboard, I believe). Our fifth player was...

Seriously? We're all level 60!

borealis01 · July 2014

Those are actual bots (I spotted at least 10x of those while pvp-leveling my gwf) I reported them as much as I could but seems like they're just letting them run loose, this ones used to screw lowbie parties sometimes with 2 or 3 on a team.

pandora1x · July 2014

Lmao I met the same. But a Guardian Fighter.

What's sad is that the one I met wasn't even a bot, but a player that knew perfectly how to chat.

eldarth · July 2014

Sure ELO is working fine -- just compare your ELO Rating to others ELO Rating.

Oh...

Right...

Cryptic doesn't display the ELO Rating so no-one knows what the he11 is going on.

jonkoca · July 2014

Anyone actually worked out the mechanic behind match result(s) and movement up and down the leaderboard..? I've been seemingly going up and down, mainly down if I'm honest, between 22 and 100 this last week regardless of win/loss and associated score values.

I thought I had it worked out - win/loss ratio + average score per match, but obviously not apparantly.

Re: mysteriously low geared high elo players. If you're a tank with low dps - like my pf HR - then the doubles and triples achievements in domination are difficult. One way is to lose matches like crazy until you fall off the leaderboard, start getting matched with really low gs fresh 60 players, then put your serious pvp gear back on and farm double and triple kills. I'm thinking about it myself, the doubles I'm about halfway done, but the triples... eh, maybe in a few year's time at the present rate.

eldarth · July 2014

jonkoca wrote: »

I thought I had it worked out - win/loss ratio + average score per match, but obviously not apparantly.

No. Gear score, win/loss ratio, average score per match, kill/death ratio -- are all completely meaningless.

Here's detailed information about how ELO ratings work.

Basically, if you win - you gain "some" ELO points depending on the ELO rating of your competition -- if you are way above them, you only gain a tiny bit -- if you are below them you gain a great deal.

mehguy138 · July 2014

No, it isn't.
/thread

jazzfong · July 2014

Avoid queue after daily reset. Wait for 3 hours later and you wont meet any bots. Works for me.

I hate bots anyway, they drag me down from top 30 clerics to page 80++ now despite winning a lot and losing little.

jonkoca · July 2014

Thnx for the link. Makes sense I suppose and explains why an unlucky pug blows my pg number out of the water.

zebular · July 2014

. . . If you believe you've gotten a botter in a pvp match (or anywhere really), please report them through in-game Support. Press H, then GM Help and choose Behavior as the category.

. . . Otherwise, please refer to Eldarth's post for what he speaks is true. To validate his statement, below is a GentleManCrush's statement on the ELO System in Neverwinter:

gentlemancrush wrote: »

Hey guys, I wanted to hop in and address some of the matchmaking questions that have been cropping up recently. I can't really get into the proprietary math we are using to *do* the matchmaking, but I will talk about a handful of the factors that surround how we do the matchmaking.

1. Why do matches sometimes seem so lopsided?
This is a hard question to answer, for several reasons. There are a lot of factors that determine match quality, and the system attempts to weigh all of them and give you the best possible match without waiting forever, which causes you to sometimes be placed in a suboptimal match rather than waiting indefinitely for a perfect match. We have several dials we can turn up and down to attempt to adjust matchmaking, but all of these things carry some degree of risk when we adjust them so we try to tune them little by litte. However we are watching matchmaking quality and are actively trying to weigh that against how long it takes to get and play a match.

2. Why do I sometimes drop on the leaderboard after winning a match?
This one is actually a matter of how frequently the leaderboard is updated. Our leaderboard UI cannot update constantly, so when you leave a match you see it update again. While it is theoretically possible in the math for your rating to go down when you win, it is *incredibly* unlikely. Most often what is really happening is that while you rise a bit from winning a match, other people have risen as well when you are in that match which makes it appear that you fell in the rankings.

3. Why don't you calculate for Gear Score when doing matchmaking?
We looked into it, but the data we have shows that Gear Score isn't a valid metric for matchmaking in our PvP. We have kept track of the average Gear Score of players at various matchmaking tiers and it turns out the difference between the top 10% players and the players in the 50th percentile is less than 2%. Also interestingly, the players with the highest numerical Gear Scores tend to fall close to the middle of the rankings. We have discussed doing some kind of adjustment for a player with no matchmaking history based on their Gear Score, but it would no longer matter after the first handful of matches that were played.

I hope this answers some of the questions surrounding PVP matchmaking.

Chris "Gentleman Crush" Meyer

Safe travels,
Archmage Zebular of Mystryl
PWE Community Moderator

pherrow · July 2014

2. Why do I sometimes drop on the leaderboard after winning a match?
This one is actually a matter of how frequently the leaderboard is updated. Our leaderboard UI cannot update constantly, so when you leave a match you see it update again. While it is theoretically possible in the math for your rating to go down when you win, it is *incredibly* unlikely. Most often what is really happening is that while you rise a bit from winning a match, other people have risen as well when you are in that match which makes it appear that you fell in the rankings.

Still not buying this one...

eldarth · July 2014

pherrow wrote: »

Still not buying this one...

If only Cryptic would display a players ELO rating on the leaderboard it would clarify everything.

osterdrache · July 2014

eldarth wrote: »

If only Cryptic would display a players ELO rating on the leaderboard it would clarify everything.

They cant because it would disprove their statements.
You loose OFTEN elo with winning a match. Thats a fact and has nothing to do with some leaderboard updating or such.

The math behind this system is flawed. But hey, what do the players know, right?

proneification · July 2014

pherrow wrote: »

Still not buying this one...

Because there are countless people experiencing the "I win, I drop" thing. People from the top page, where there are just a few guys that you know above you, and they are offline, yet you win against pugs, and you drop.

So the developer's statement is pretty much nothing, just empty words invalidated by what is actually happening in the game. Here come my assumptions. I might be wrong, but am pretty sure at this point I'm not.

- it's all about team "collective ELO". If your team average ELO is higher than the other team average ELO, and you win, you will drop depending on the ELO difference. If the difference is very small, you might remain the same place. If your ELO is significantly higher (ie premade stomping pugs), expect significant drops on the leaderboard as well when you win.

Players are not dumb. They already try to exploit the system, by running a duo team.

Why duo queue? First, it's a big chance you will avoid full premades. Second, if you get bad ELO pugs and you manage to carry them to a victory against a better average ELO team, you will actually, finally, advance on the leaderboard. If you queue full premade, 90% of your games will be pug stomps that will drop you, and the few games that you get to fight other premade might not mean anything anyway if you have higher average ELO.

Average Team ELO. This is what matters in this game. You wanna go up on the leaderboard, you need to persuade the matchmaking to give you higher ELO teams to fight, and you need to win.

gothashot · July 2014

Thanks eldarth for the link. That was a very interesting read.

Sadly, all those nifty formulas and mathstuffs and we still can't get decent matchmaking.

gothashot · July 2014

I did 4 matches yesterday, all pugs. I, of course, lost all 4 matches. My rank went up.

One match I was top score, the others I was pretty close to the bottom.

I really can't make heads or tails of it.

jonkoca · July 2014

Argh. Now I don't know what to think.

zebular · July 2014

jonkoca wrote: »

Argh. Now I don't know what to think.

The dev's said how it works, the rest is player conjecture and opinion of the system based upon their experiences, understand and misunderstanding of the system a-like. Whether or not there are bugs in the system, or if it is working as intended is really a matter of player reports and the Dev's decisions about that and their own data.

rashylewizz · July 2014

It really is simple, the ELO cannot work since there aren't enough players with adequate data queueing for PVP every single minute.

So what happens is the ELO is trying to find the right match, but a lot of players don't have enough data for ELO but it doesn't want players to wait 1 hour for a pvp match so it decides to pair them up with imbalanced teams.

tang56 · July 2014

magenubbie wrote: »

Prove it or lose it. Let's keep this concrete shall we?

Indeed it does not. But I can hardly believe you PvP lot are so thickheaded that you don't get the logic behind it. 90% of the PvP people are casuals or only in it for the AD. 10% are the people who said thank you to the bots. Those 10% will obviously never rise in ranking when you're fighting the casual and low ranked and roflstomped them. It also makes sense they do rise. They stood up to you and scored.

You think I'm wrong? Alright. There's a simple test you can do to prove me, and everybody else wrong: Take the top 10 of the leaderboard, put 1-5 together and 6-10 together in 2 teams and fight it out. You'll barely see a difference if the system works as they say it does. Maybe the 2 with the highest scores in the match move, but the rest should remain stable.

You seem to have a habit of not reading posts properly.

His complaint was that you lose ranking from winning. Not that you don't go up from winning vs similar ranked people. He is correct, stomping the unfortunate 6k fresh 60s or 14k PvE gear pugs(or anyone else who is under the top 50ish pages usually) with a team of geared and high ranked people will result in a drop.

eldarth · July 2014

tang56 wrote: »

His complaint was that you lose ranking from winning. Not that you don't go up from winning vs similar ranked people. He is correct, stomping the unfortunate 6k fresh 60s or 14k PvE gear pugs(or anyone else who is under the top 50ish pages
usually) with a team of geared and high ranked people will result in a drop.

Since your ELO rating is not displayed on the leaderboard you absolutely can not state that you "lose ranking."
You may be displayed lower on the leaderboard, perhaps due to OTHER people advancing past your ELO rating,
but, once again, since your ELO rating is invisible there continues to be wild mythical flailings that things aren't working.

If you are doing 50mph on the road, and 14 people around you are all doing 55mph -- surprise, you're "losing ground" compared to them.

Demand that Cryptic display ELO ratings -- THEN prove "it's not working."

rashylewizz · July 2014

I also think that Cryptic implemented a script that inadvertently caused players to lose rankings by winning because they wanted to prevent the leaderboard from just being a ranking of who played the most games.

Now, we can see that the leaderboard isn't exactly working perfectly, but that is a reason why some players lost their ranking on the leaderboard even after winning a game.

Either way, the gist of this whole problem is, not enough players queueing leading to the system making tons of compromises.

futuun · July 2014

gentlemancrush wrote:

2. Why do I sometimes drop on the leaderboard after winning a match?
This one is actually a matter of how frequently the leaderboard is updated. Our leaderboard UI cannot update constantly, so when you leave a match you see it update again. While it is theoretically possible in the math for your rating to go down when you win, it is *incredibly* unlikely. Most often what is really happening is that while you rise a bit from winning a match, other people have risen as well when you are in that match which makes it appear that you fell in the rankings.

Yea.. well some time ago I was wondering as it really is.
I found a way to call other leaderboard window with numbers instead "rank".
How it looks? 1st page, last page.
I can check my score after every match and... guess what!
90% of my winning matches in 2-5 man premade against pugs = -0.16 score points.

I think that there is no matchmaking system now before mod 3 matchmaking worked a lot better.

...and Leaderboard ≠ ELO rating.

plavia · July 2014

rashylewizz wrote: »

It really is simple, the ELO cannot work since there aren't enough players with adequate data queueing for PVP every single minute.

So what happens is the ELO is trying to find the right match, but a lot of players don't have enough data for ELO but it doesn't want players to wait 1 hour for a pvp match so it decides to pair them up with imbalanced teams.

sound reasonable
(in the begging when they implement it took almost half day to get into a PVP match)

Are We Sure the ELO System is Working?

Comments