I was in a match earlier and got booted at like 5 mins in, couldnt get back in. i had a 1,000 rating or less for that match because someone else was runnin it.
Good news, everyone: I have added a "late entry / disconnect" detection. Currently it's not yet being used to filter the results, but with the latest client version I collect this information for future use.
There's something bugging me. And please correct me if I'm wrong but the program only logs 5v5 matches, right? Well it's happened quite a bit that I'm trying to get my two matches in for the day on a particular character and either a. I end up in a 4v4 or 5v4 etc. or b. someone logs off in the middle of the match. So I may have an excellent match, with great stats, followed by one that just doesn't show which may also have been a good match or maybe not. So as far as the board is concerned, unless I keep playing now for a third match, which I might not be able to do depending on time and the Q's, my first match is pointless.
I suppose I could change the team detection algorihm to work outside of 5v5 and detect whether it was 4v4 or 5v3 or such, but I'm afraid that the rating algorithm will not really work for that and give weird results. Going from 5v5 to 4v4 or 3v3 changes the balance of healing and damage (you can focus all damage, but you can focus only a part of the heals) and thus the weighting of the different aspects would probably be off.
It is not entirely impossible to do, but unlikely for the near future.
(Note: If someone logs off mid-match or enters late, the match will still count as long as the total number of players is 10.)
Can you change the daily requirement to one match per day?
I don't see this happening. If I reduce the requirement for the daily list to 1, there is a high chance that the top spots will be filled by "one match wonders" with exceptional scores.
Note that you can always see your personal scores for individual matches by filtering the list by your handle and then select a character to see your 25 most recent matches.
On another topic, is there any way to differentiate between assists and direct kills and give points for each?
I'm not entirely sure what you mean by assists? Many assist-related things like shieldstripping via CPB/TachBeam and of course SNB are not visible in the log and thus cannot be used for the rating.
Theoretically one could extract enough information from the log to tell whether a PSW or tractor was useful for a kill, but this is very difficult and considering that SNB-support for a kill has the biggest impact by a wide margin and isn't visible in the log, I don't think that such a detection system would be worth the required work.
In the leaderboard/clients present state, and as someone who PUG's quite a bit, I'd estimate about 25% of my matches do not make to the board due to the 5v5 thing.
I understand about the 2 match minimum. It makes sense. And I think if the program could recognize outside of 5v5 this would be much less of an issue.
Not to boast, but just as an example, I had a totally random 3v3 from the public Q last night that went 15-8 and was all accounts a fun, challenging match. Gareb performed well and had over 2,000,000 in heals and great Utility. It would've been a great match to add to the board.
As to assists, I was talking specifically about instances where SNB and other Sci powers that contributed to a kill getting points. Also, any instance where direct damage done by weapons fire that contributed to the kill.
I'm thinking that the Style penalty point criteria are too subjective in its current form. Without a complete list of consoles, powers, etc that receive penalties someone may inadvertently reduce their score. This is a tough one because what one person considers cheesy another does not. Do you have a complete list of what receives penalties that we can all look at and take a vote on or something? An example would be Subspace Jumper and Singularity powers, I would think they would get a penalty. Do they?
Maybe Style could be replaced by Assists?
I know you said it's only a small amount in the grand scheme but as a competitive detail oriented player, these things are all important.
Cool things that could be added:
-Highest Crit hit of the day and week
-Longest kill streaks
-Top ten team players that had best combined average cross heals/kills
As to assists, I was talking specifically about instances where SNB and other Sci powers that contributed to a kill getting points. Also, any instance where direct damage done by weapons fire that contributed to the kill.
I'm sure Hilbert could probably go in to better detail as to exactly how his assist points are calculated, but one really important thing to remember is that this scoreboard (and anything else that collects info from the combat log) can only count what's listed in the combat log.
For something to appear in the combat log, it has to either do damage or give heals. While things like Tractor Beam, Grav Well, PSW, etc. deal damage and show up in the combat log, abilities like SNB, Sensor Scan, Energy Siphon, etc. don't show; there's currently no way for a combat log parser to know that they've been used. In addition, shield drain effects do not show in the combat log right now.
As a result, Science (and to a lesser degree, Eng) skills tend to be hard to detect. I'd certainly like to be able to gather that information from a match (and I'm sure that Hilbert would too), but unfortunately there's no current method of doing that.
An example would be Subspace Jumper and Singularity powers, I would think they would get a penalty. Do they?
Again, Hilbert can probably explain his style calculation, but it's important to note stuff like the Subspace Jump won't show up in the combat log, because they deal no damage.
Note that this changes very quickly as I test new approaches. In particular the current approach makes comparatively little use of the new team detection feature; this is something I intend to change, or at least test whether a change improves the quality of the rating.
Hilbert should arrange the graph like how the winners are arranged in the podium in an olympic event. The winner should be in the middle, the two runner-ups on his/her side and the last two on either side of them. It will look perfect, IMO
Just enter the name@handle data and it will try to give you a sensibly balanced match. (If it's not 10 players, it might suggest 5v3 instead of 4v4... not sure whether that's a bug or a feature.)
The balancing is done in a super simple fashion:
1) Make sure that the number of sci players in the teams does not differ by more than 1.
2) Attempt to balance DPS and (weighted) HPS among the teams.
Obviously there is much room for modification, i.e. attempting to balance according to the leaderboard scores, or use kill counts, but let's see whether this simple thing already allows for decent balancing.
I have to say that the idea behind this and all the effort is great and you have done a great job hilbert. But I think this leaderboard is unbiased as it is being used mainly by premades in pug stomping. All you have to do is take the last 10 matches and see it's premades against pugs.
I believe using this tool in this way is just a d**k measuring tool. I know of several occasions where some who have the tool have been beaten and yet there is no results on your board.
Don't get me wrong I think it's a fantastic tool, if used by all not just a select few when they feel like it.
I have to say that the idea behind this and all the effort is great and you have done a great job hilbert. But I think this leaderboard is unbiased as it is being used mainly by premades in pug stomping. All you have to do is take the last 10 matches and see it's premades against pugs.
I believe using this tool in this way is just a d**k measuring tool. I know of several occasions where some who have the tool have been beaten and yet there is no results on your board.
Don't get me wrong I think it's a fantastic tool, if used by all not just a select few when they feel like it.
I think you mean *biased*. And I agree, originally I loved the idea, however the board itself greatly discourages pugging in my opinion. You have to be very careful about what matches you take and not even run ACT/Client when you will be pugging if you're to maintain good standing on the board. The problem is it's completely luck of the draw that will determine you're score in a public arena match. Not your skill. This PvP is utterly team dependent so to have a leader board for individuals does not work.
I do like the stats, and the top tens, and it's great to have something interesting linked to our matches, it's just hard to take it seriously unless all you do is TD matches.
There are exceptions to the "premade benefiting most from the board" as there are some captains that just do enough damage solo to always place well.
However this exception is more rare in the case of pure support roles. If you have a very competent team with good survivability, you will need to heal less, thus having a lower score.
Also, there are currently not enough registered members in my opinion, to counter the possibility of scores purposely not being uploaded or manipulated.
It's the best Hilbert can come up with until Cryptic opens up the combat log to show more variables.
On the other note, I personally don't take the leaderboard seriously, I even got number 1 once. It is an ego sizer, which is why I never recorded any of my pug matches.
What's more important is whether you yourself are happy with your current build and skill level and never be content with being top of anything, there will always be room for improvement
Now found frequenting MWO short term and then Star Citizen long term. Raged Quit PVP long ago
- Gohan (House of Beautiful /Sad Pandas)
Maybe it's time to reinstate the exponential dampening factor for low match counts. I initially removed this and replaced it by a hard cutoff for the daily/weekly/monthly lists because it leads to an odd and highly unintuitive disconnect between the average score and the individual match scores, but maybe it needs to go back in to give players an incentive to play lots of matches.
Regarding the "premades vs pugs": I think people are underestimating just how unbalanced pvp queue matches are in general. The average total kill count appears to be something like 18, which means 15-3.
And maybe I need to move to a relative-to-team scoring in combination with match weights, to better reward lost matches where one still was the best player in the losing team.
Is there really enough of us to make any real leaderboard worthwhile? It's a great idea of course but there's too few of us.
Yes i do believe they're are enough PvP players for any real leaderbords, the fact is it may not seems to enough pvp players but the reality is even us PvP players have got caught up in our own PvE grind to get resources to upgrade our toons like Omega, Nukara, New Romulus, Fleet Marks which is all exculsively on PvE queues. (well there is some fleet marks on PvP missions daily)
I bet you if PvP have it's own marks people will be playing on PvP. look at World of ******** for an example there's a reason why alot of players play it's PvP and why only a few people on STO PvP's at any given time... it's because in World of ******** PvP it rewards it players with an currency that can buy PvP tech which in STO PvP have no such rewards except for Dilithium.
the sad fact is Cryptic neglected STO's pvp forcing all players to grind in PvE queue's to get the best tech.
I've been playing WoT(anks) for a change. One of the interesting aspects of the game is the third-party stats tracking for win/loss, etc. (Soon to be supplemented by an official rating system.)
Discouraging to me as a noob, but it makes PvP more interesting by giving me some real metrics to work on.
_________________________________________________ [Kluless][Kold][Steel Heels][Snagtooth] [Louis Cipher][Outta Gum][Thysa Kymbo][Spanner][Frakk] [D'Mented][D'Licious]
Joined October 2009. READ BEFORE POSTING
Regarding the "premades vs pugs": I think people are underestimating just how unbalanced pvp queue matches are in general. The average total kill count appears to be something like 18, which means 15-3.
And maybe I need to move to a relative-to-team scoring in combination with match weights, to better reward lost matches where one still was the best player in the losing team.
Have you given any thought to possibly setting up a specific leader-board that focuses solely on TD style matches?
You could run the leader-board you do now, but having a separate one for TD would likely give you some very different (and interesting) data.
Maybe it's time to reinstate the exponential dampening factor for low match counts. I initially removed this and replaced it by a hard cutoff for the daily/weekly/monthly lists because it leads to an odd and highly unintuitive disconnect between the average score and the individual match scores, but maybe it needs to go back in to give players an incentive to play lots of matches.
Regarding the "premades vs pugs": I think people are underestimating just how unbalanced pvp queue matches are in general. The average total kill count appears to be something like 18, which means 15-3.
And maybe I need to move to a relative-to-team scoring in combination with match weights, to better reward lost matches where one still was the best player in the losing team.
The relative-to-team scoring idea sounds good. The exponential dampening factor also sounds like it could be a good thing...
Thanks for your continued work on this. And wow, it's just nice to have someone that listens to, and responds to feedback in such an intelligent manner. CRYPTIC HIRE THIS MAN OR GIVE HIM SOME MONIES!
I have toyed with different rating algorithms, and some seem to give a better ordering of the players within a team, but I don't like the resulting average result list better than the current one, so I haven't changed the rating yet.
The core problems for a good rating system are the following:
1) You can perform really well, but still score very few points when facing a superior opponent. That makes it difficult to say how big the impact of the kill score should be. (Only using dps favours ineffective faw beamboats, not the more efficient buils that score points with comparatively low damage.)
2) Dedicated healers always perform best-in-team in healing. This is therefore not a reliable metric for healing. And absolute values vary depending on the amount and type of incoming damage.
3) Inserting too much of a "bonus" for good performances on the losing side could lead to a situation where it's beneficial to lose the match 14-15 to get a better score.
Yeah its very clear that rating goes up significantly to like 5k+ when pugstomping whereas good matches will make ur rating a bit lower while you still did equally good, just your opposition is better.
Look I am not sure if this has been said before, but I do not qualify for the Leaderboards, as I am not on Hilbert's Friends list. In fact, he has me on ignore, and routinely tries to taunt me with that fact. Do not know why I should care, but the point I am going for, if it is not obvious enough, is that a list derived by one persons rules, and maintained by one person who is not associated with the development process. The list is in my opinion worthless. As there are some players who are surprisingly good that never really care for the ques. So I personally find the Leaderboard laughable at best. Not due to the fact that I do not appear on it, but due to the fact that his friends are more likely to achieve higher ranks due to his relationship with them.
I'm not on his friend list and most of my matches aren't counted either.
I'm not going to claim that I know how it all works but it's a good solid effort. My understanding was that the leader board needs more data sets. Perhaps reading more or asking questions about what you can do to contribute to the board would be a more meaningful use of time.
__________________________________________
Foundry: Yet Another Borg Mission
It's terrible but easy, and these Borg are way cooler than the mess STO and Voyager left us.
May not actually be "way" cooler or even "slightly" cooler.
Being on my friends list has nothing to do with it. But as long as the rating algorithm is not stable and I'm not sufficiently happy with its quality, I see no reason in massively expanding the amount of reporting players. This thing is still very much in testing and a proof-of-concept. There are already some useful applications like the auto-balancing tool that seems to work fairly well when all involved players have a sufficient number of logged matches, but the general list does not yet have the level of meaning that I would like it to have - it tends to give too low scores for long balanced matches. The special Top10s tend to be a more "realistic" thing - and they are much easier to compute since they do not require fusing healing and damage scores into one number.
Comments
I suppose I could change the team detection algorihm to work outside of 5v5 and detect whether it was 4v4 or 5v3 or such, but I'm afraid that the rating algorithm will not really work for that and give weird results. Going from 5v5 to 4v4 or 3v3 changes the balance of healing and damage (you can focus all damage, but you can focus only a part of the heals) and thus the weighting of the different aspects would probably be off.
It is not entirely impossible to do, but unlikely for the near future.
(Note: If someone logs off mid-match or enters late, the match will still count as long as the total number of players is 10.)
I don't see this happening. If I reduce the requirement for the daily list to 1, there is a high chance that the top spots will be filled by "one match wonders" with exceptional scores.
Note that you can always see your personal scores for individual matches by filtering the list by your handle and then select a character to see your 25 most recent matches.
I'm not entirely sure what you mean by assists? Many assist-related things like shieldstripping via CPB/TachBeam and of course SNB are not visible in the log and thus cannot be used for the rating.
Theoretically one could extract enough information from the log to tell whether a PSW or tractor was useful for a kill, but this is very difficult and considering that SNB-support for a kill has the biggest impact by a wide margin and isn't visible in the log, I don't think that such a detection system would be worth the required work.
http://hilbertguide.com/leaderboard/?encounter=2013-09-08%2013:31:51&
In the leaderboard/clients present state, and as someone who PUG's quite a bit, I'd estimate about 25% of my matches do not make to the board due to the 5v5 thing.
I understand about the 2 match minimum. It makes sense. And I think if the program could recognize outside of 5v5 this would be much less of an issue.
Not to boast, but just as an example, I had a totally random 3v3 from the public Q last night that went 15-8 and was all accounts a fun, challenging match. Gareb performed well and had over 2,000,000 in heals and great Utility. It would've been a great match to add to the board.
As to assists, I was talking specifically about instances where SNB and other Sci powers that contributed to a kill getting points. Also, any instance where direct damage done by weapons fire that contributed to the kill.
I'm thinking that the Style penalty point criteria are too subjective in its current form. Without a complete list of consoles, powers, etc that receive penalties someone may inadvertently reduce their score. This is a tough one because what one person considers cheesy another does not. Do you have a complete list of what receives penalties that we can all look at and take a vote on or something? An example would be Subspace Jumper and Singularity powers, I would think they would get a penalty. Do they?
Maybe Style could be replaced by Assists?
I know you said it's only a small amount in the grand scheme but as a competitive detail oriented player, these things are all important.
Cool things that could be added:
-Highest Crit hit of the day and week
-Longest kill streaks
-Top ten team players that had best combined average cross heals/kills
but at what point would this combat log thingy violate the TOS? i mean, at what point in the data collections is too far?
Do you even Science Bro?
I'm sure Hilbert could probably go in to better detail as to exactly how his assist points are calculated, but one really important thing to remember is that this scoreboard (and anything else that collects info from the combat log) can only count what's listed in the combat log.
For something to appear in the combat log, it has to either do damage or give heals. While things like Tractor Beam, Grav Well, PSW, etc. deal damage and show up in the combat log, abilities like SNB, Sensor Scan, Energy Siphon, etc. don't show; there's currently no way for a combat log parser to know that they've been used. In addition, shield drain effects do not show in the combat log right now.
As a result, Science (and to a lesser degree, Eng) skills tend to be hard to detect. I'd certainly like to be able to gather that information from a match (and I'm sure that Hilbert would too), but unfortunately there's no current method of doing that.
Again, Hilbert can probably explain his style calculation, but it's important to note stuff like the Subspace Jump won't show up in the combat log, because they deal no damage.
http://hilbertguide.com/leaderboard/ranking.html
Note that this changes very quickly as I test new approaches. In particular the current approach makes comparatively little use of the new team detection feature; this is something I intend to change, or at least test whether a change improves the quality of the rating.
It's ok, neither does cryptic.
Do you even Science Bro?
Elite Defense Stovokor
Just enter the name@handle data and it will try to give you a sensibly balanced match. (If it's not 10 players, it might suggest 5v3 instead of 4v4... not sure whether that's a bug or a feature.)
The balancing is done in a super simple fashion:
1) Make sure that the number of sci players in the teams does not differ by more than 1.
2) Attempt to balance DPS and (weighted) HPS among the teams.
Obviously there is much room for modification, i.e. attempting to balance according to the leaderboard scores, or use kill counts, but let's see whether this simple thing already allows for decent balancing.
I believe using this tool in this way is just a d**k measuring tool. I know of several occasions where some who have the tool have been beaten and yet there is no results on your board.
Don't get me wrong I think it's a fantastic tool, if used by all not just a select few when they feel like it.
I think you mean *biased*. And I agree, originally I loved the idea, however the board itself greatly discourages pugging in my opinion. You have to be very careful about what matches you take and not even run ACT/Client when you will be pugging if you're to maintain good standing on the board. The problem is it's completely luck of the draw that will determine you're score in a public arena match. Not your skill. This PvP is utterly team dependent so to have a leader board for individuals does not work.
I do like the stats, and the top tens, and it's great to have something interesting linked to our matches, it's just hard to take it seriously unless all you do is TD matches.
There are exceptions to the "premade benefiting most from the board" as there are some captains that just do enough damage solo to always place well.
However this exception is more rare in the case of pure support roles. If you have a very competent team with good survivability, you will need to heal less, thus having a lower score.
Also, there are currently not enough registered members in my opinion, to counter the possibility of scores purposely not being uploaded or manipulated.
On the other note, I personally don't take the leaderboard seriously, I even got number 1 once. It is an ego sizer, which is why I never recorded any of my pug matches.
What's more important is whether you yourself are happy with your current build and skill level and never be content with being top of anything, there will always be room for improvement
- Gohan (House of Beautiful /Sad Pandas)
Regarding the "premades vs pugs": I think people are underestimating just how unbalanced pvp queue matches are in general. The average total kill count appears to be something like 18, which means 15-3.
And maybe I need to move to a relative-to-team scoring in combination with match weights, to better reward lost matches where one still was the best player in the losing team.
- Gohan (House of Beautiful /Sad Pandas)
Next to no help? Hilbert hasn't gotten ANY help at all from them.
Hilbert, thanks again for stepping up and doing something in your own free time that our developers continually ignore.
Yes i do believe they're are enough PvP players for any real leaderbords, the fact is it may not seems to enough pvp players but the reality is even us PvP players have got caught up in our own PvE grind to get resources to upgrade our toons like Omega, Nukara, New Romulus, Fleet Marks which is all exculsively on PvE queues. (well there is some fleet marks on PvP missions daily)
I bet you if PvP have it's own marks people will be playing on PvP. look at World of ******** for an example there's a reason why alot of players play it's PvP and why only a few people on STO PvP's at any given time... it's because in World of ******** PvP it rewards it players with an currency that can buy PvP tech which in STO PvP have no such rewards except for Dilithium.
the sad fact is Cryptic neglected STO's pvp forcing all players to grind in PvE queue's to get the best tech.
Discouraging to me as a noob, but it makes PvP more interesting by giving me some real metrics to work on.
[Kluless][Kold][Steel Heels][Snagtooth]
[Louis Cipher][Outta Gum][Thysa Kymbo][Spanner][Frakk]
[D'Mented][D'Licious]
Joined October 2009. READ BEFORE POSTING
Have you given any thought to possibly setting up a specific leader-board that focuses solely on TD style matches?
You could run the leader-board you do now, but having a separate one for TD would likely give you some very different (and interesting) data.
The relative-to-team scoring idea sounds good. The exponential dampening factor also sounds like it could be a good thing...
Thanks for your continued work on this. And wow, it's just nice to have someone that listens to, and responds to feedback in such an intelligent manner. CRYPTIC HIRE THIS MAN OR GIVE HIM SOME MONIES!
The core problems for a good rating system are the following:
1) You can perform really well, but still score very few points when facing a superior opponent. That makes it difficult to say how big the impact of the kill score should be. (Only using dps favours ineffective faw beamboats, not the more efficient buils that score points with comparatively low damage.)
2) Dedicated healers always perform best-in-team in healing. This is therefore not a reliable metric for healing. And absolute values vary depending on the amount and type of incoming damage.
3) Inserting too much of a "bonus" for good performances on the losing side could lead to a situation where it's beneficial to lose the match 14-15 to get a better score.
Just an observation.
I'm not going to claim that I know how it all works but it's a good solid effort. My understanding was that the leader board needs more data sets. Perhaps reading more or asking questions about what you can do to contribute to the board would be a more meaningful use of time.
It's terrible but easy, and these Borg are way cooler than the mess STO and Voyager left us.
Being on my friends list has nothing to do with it. But as long as the rating algorithm is not stable and I'm not sufficiently happy with its quality, I see no reason in massively expanding the amount of reporting players. This thing is still very much in testing and a proof-of-concept. There are already some useful applications like the auto-balancing tool that seems to work fairly well when all involved players have a sufficient number of logged matches, but the general list does not yet have the level of meaning that I would like it to have - it tends to give too low scores for long balanced matches. The special Top10s tend to be a more "realistic" thing - and they are much easier to compute since they do not require fusing healing and damage scores into one number.
Also how hard wud it be to add player profiles, like win/lose..avg heal ect
and Hilbert is terrible at pvp anyway his guide is for noobs ... nothing to see here ... move on folks
Player profiles with stats would be awesome! Great idea.