test content
Logo
What is the Arc Client?
Install Arc
I can filter Spam better than you, so let me do it instead, please
l0th4ri0
Member Posts: 589 Arc User
Seriously guys, come on. 95% of zone chat in PE is spam messages from goldsellers. Give me the ability to filter out chat messages with my own personal blacklist for keywords that i see repeated over and over again and the spam stops (for me at least). Store the blacklist on my hard drive somewhere in some new game file or something. I don't care. Just let me filter stuff myself -- I'm playing the game, you guys aren't -- it's not like I'm accusing you of not caring or of being incompetent or anything negative. Just listen to me:
Let ME do it.
I will still report spam when one of the botters changes their message, but then I'll add a snippet of their text to my blacklist and then all the hundreds of clones of the bots who get recycled over the next week are rendered powerless. Trust me, the spammers will NOT be changing their messages enough for me to see 95% spam in PE anymore. It will turn into a rare occurrence that I shrug off as I take a grand total of 20 seconds to fix.
Imagine having users that come onto the forums here singing your praises... imagine it. You can do it. It's just like coding an ignore list, only I can enter custom text into a textbox somewhere after selecting something in options menu or whatever. You can be heroes guys... you can make this happen.
Let ME do it.
I will still report spam when one of the botters changes their message, but then I'll add a snippet of their text to my blacklist and then all the hundreds of clones of the bots who get recycled over the next week are rendered powerless. Trust me, the spammers will NOT be changing their messages enough for me to see 95% spam in PE anymore. It will turn into a rare occurrence that I shrug off as I take a grand total of 20 seconds to fix.
Imagine having users that come onto the forums here singing your praises... imagine it. You can do it. It's just like coding an ignore list, only I can enter custom text into a textbox somewhere after selecting something in options menu or whatever. You can be heroes guys... you can make this happen.
0
Comments
We are searching for slave labor, will pay with food from our farm!
Perhaps you misunderstand, young scathias. I am proposing that users have the ability to text filter their own chat. If you are unable to see the immense utility of such a feature, and the ramifications of what that means for your gaming experience in Neverwinter, then I really don't know what to say.
For example, let me filter out any message with the following characters: .c0^m or /\/\ and that is just about 80-90% of it right there. As soon as some botter changes .c0^m to .c^0m or whatever, then I add THAT to my list, and now the botter is defeated for the next week. Hell they could change their messages every day (they won't, but just for argument's sake) and it will still be better for me to be able to filter out a handful of things when I login (and then never see them again) than the current spamfest that is Protector's Enclave.
I would add it in as a config file on your pc that gets loaded into the game, that way, you can edit it offline and also share it with the community.
I'm willing to wager that it would not significantly impact performance.
It seems so simple that there must be a catch, right? Why doesn't this exist already?
BTW I couldn't agree more with your assessment that this doesn't stop cheating. This doesn't stop spammers from creating accounts and flooding the AH with ill-gotten goods. It doesn't prevent ppl from buying AD from 3rd party vendors. This measure I propose does not fix everything.
But it is time for the guys behind the servers to admit that they are beaten in the war vs. chatspam. It's not that they are bad at their jobs or anything, they just can't take the drastic draconian measures it would require to stop the spamming in PE. Their hands are tied.
Our hands are not tied.
I can take the responsibility for never seeing any more chat messages with the characters /\/\ in them. I will fully assume that mantle and understand that if one of my friends want to write me something with that sequence of characters in it, I'm not going to see it.
We are searching for slave labor, will pay with food from our farm!
Anyhow, @strumslinger and @terramak is there any chance of something like this being implemented? I know for a fact that something similar (albeit not for filtering spam but filtering something else) has worked beautifully in other games, so it would be really nice to have it implemented here.
We are searching for slave labor, will pay with food from our farm!
Such filter on the client side shouldn't have any performance issue.
Nor difficult to implement. Running a single regex check on each message is negligible and the code make those already (profanity filter). - But it's still not viable.
Implementing like the OP mentioned: Creating a whole list of messages as they are on the client and comparing to that can cause performance issues.
But regardless of implementation it's not viable, because of how the bots can permute or swap a single and there are a lot of combinations. Having each client to add every variant.... the bots will win, as scatias said, it's playing tag and it's futile.
Also it has a huge downside, and only if from a support point of view.
I believe most players wont be able to handle that on their own, now imagine a player miscopies a filter or listens to a troll, or just tinkers with the options unknowingly and sets a filter the filters everything (this is a good case) ?
Now their chat wont work, they can't ask for help, they tinker further, they go to the support of forums, and they are not happy.
Imagine that it filters partially and it takes time to realize, now we have an angry mob of players (look at the xbox poeple dont see some of their guildies chat)
What really should be done is a server side black listing / white listing. When people report spammers the spammers should not only be muted but also the message checked, a diff from previous blacklisting should be made and added to a filter (dictionary). This addon (and the original) should be approved by a person, or filtered first and checked for white listing. The messages themselves filtered with a string distance algorithm like q-gram distance for example, Somewhat in the same way spell checking works, in essence we try to spell check / correct the bots new variations to those that we already know and add them.
In any case a solution should be made server side, and not client side, as a client side solution will be irrelevant to the majority to the population as soon as the bots adapt.
Blocking the following two snippets of characters:
.c^0m and /\/\
Blocks 80-90% of it at minimum, maybe all of it. Even the PMs... those are the same guys. Even if they decide to change their message to something completely different with different character weirdness, it would take a few heartbeats to add a sample of it to your clientside blacklist. And then they are gone for weeks at a time again, until the botter decides to change it.
I am in PE seeing messages of ppl constantly -- CONSTANTLY -- rightclick blocking spammers and complaining that they have to do so every time they log in, and then continue to do so the entire time they are in PE running around questing or chatting in zone. Do you honestly think that this measure is too onerous to the playerbase? really? Ppl who spend time to whack-a-mole dozens of spammers can't be bothered to once in a great while type a bit of text into a box one time?
My favorite part of my proposal is that the spammers won't even know what it is exactly you are blocking. Do they need to add another ^ or & or something? Where should it go? How do they defeat 1000s of spam filters all slightly different from each other, and as soon as they succeed they are silenced and back in the dark, figuring out how long to even try to run a message before having to change it, and then it's just gonna get seen ONCE before its gone forever?
Do you honestly believe that this is "tag"? This is extermination, my friend. Apply 1% of the dedication that I see on a daily basis from ppl rightclick blocking spammers to maintaining a blacklist and the spammers are handily defeated.
As for the contention that a tiny textblock filter adds undue performance stress and is too unwieldly to be effective when it fires once in a while, I can't really say anything concrete about exact numbers, but filters would be less than 1k... the sequence I outline above to end all current spam is 12 bytes, if we include the necessary separators. But let's pretend that even a tiny 1k filter is hitting performance visibly -- like there's a slight tug every time you get a message popup in chat. Fine, put a 512 byte limit on the filter, that would be PLENTY enough to still get the job done.
My reply: I could give a HAMSTER where other lists are stored. Give me a clientside filter, please. I was using the ignore list as an example (perhaps a poor one) of how we already have lists that block stuff. It was meant as a general thing and not some technically specific example of how to exactly code my proposal. I already mentioned twice how it could be done clientside. You've surely read it. I can't imagine why in the world the servers would need to get involved in any way. It's my filter on my computer that runs off of my client -- the server isn't needed to provide anything but the original text that it was sending me anyway. When that text shows up in my client, it gets run through a brand-new little snippet of code that calls up a blacklist stored on my computer (unlike the other kinds of lists you mentioned already in use, apparently) and compares the text it just received to disallowed sequences of characters. if there is a match, the text received from the server is not displayed on my screen. Everyone else still sees it... unless they have THEIR filters set and it gets flagged on their clients as well. Only unblacklisted text is seen by the player, even though all of it is sent from the server. The blacklist subroutine hides it from me -- that's all it needs to do.
Do you get it now? It's really not complicated. Local program running in the background of my client protecting me from garbage in the chatbox. end of story.
A system / solution in this case is measured not by how it solves the problem now, but how easily it's defeated.
2. "until the botter decides to change it" This is the operative word, it will take me minutes to write something simple that will defeat your solution, by changing every single post or every hour and so on, they have no reason to change now, give them and they will chage. And it will take me long long time before I run out of permutations, and there are things you can't block. So again every user of your system will complain about "it doesn't work" Where I wrote anything that claims that people will not write ?
I said, people will either not know what to filter, or make mistakes and filter to much.
This is not the same issue....
But you are right, it's a valid issue, and I will expand on it in a bit. Easily, by spamming you with variations, some will go through, some will not, but each time one passes you are the one reacting, you saw a message you must react, but it defeats the purpose, because you will see the message.
It's like forcing you to mark each spam email after you yourself read it.
Also they wont care about what you are personally blocking, the filters are not globally related, they are client, and by your additive system they at first need to send you new variation and then if it passes you will block, so they will just send. Worse, as I said the filters are not related, yours blocked, mine didn't, bots job done successfully. And if you spread the filters and share, everyone will know and the bots will adapt. Security by Obfuscation will be negated by open sourcing it....
Bots need to reach a percentage of the their targets, and the system should defeat that, reactive, personal system does so poorly.
Companies fight email spam for long long time now, I think we can learn few things from what worked there and what failed, and what is applicable. What 1% ? Is this a variation of "87% of all statistics is made up" ?
Put one person to filter and blacklist server side and you solved it. You can dedicate your life as a user to updating your personal filter, but it will solve nothing for the player base, the devs, nor the game.
Let me ask you this, how many players you submitted reports of ? Not the spam one, but the actually report of bad behavior.
I'm sure if you played long enough that you met some racists, griefers, thieves and so on.. but to report them you need to manually copy the handle, and go to the appropriate menu and submit a ticket.
You suggest the same, but worse, for every message, I either need to add it whole, but then I inflate the filter and it's easily defeated by changing one char.
Or I need to stop, think, and extract the offensive part, but here I risk to make the filter to aggressive and it will filter later messages I should receive.
Also you assume that most of the game population will know what part to filter, and to that I refereed in my original post.
1. A filter can't fire "once in a while" you can't know what is offensive or not before you filtered it, the filter must be activated on every single message you receive. A way to lessen this is to create a list of trusted sources, but then still each message must be check vs that list. So in a good case you hit the trusted list, in a bad case (source not in the list) you just added the length of the list to the length of the filter list.
2. The length of the filter is not that important, it is only important in the worst case scenario where the the entire filter matched except the last char. So If the filter is of length n I had to make n comparisons to discard it.
But by your description: lets assume 512 byte per filter. (there is no need, but for the example you provided).
Let me quote you again: Lets assume 5k filters, total length will be 2560000B, 2500KB, 2.4MB
But again the length is not so relevant and actually the message limit is about 200 chars...
The worst case for this system is not when it filters, but when the message is not spam, meaning I've checked ALL the filters and all of them failed.
You have 2 choices, when you create the filter, to copy a large portion of the message, this is the safe way, but easily circumvented by changing any single char. Or attempt to make the minimal filter, but then you risk to filter safe messages.
The ".c^0m" will work, but only as a firing shot, do not assume those that run the bots are total idiots, they will check the system and start to cycle, and unfortunately UTF-8 (or a subset) has enough character to make this a long long fight.
Another downside, for the filter being client side, is that if the bots prepared a list with a few hundred thousand of messages (really not that hard to do with a string of 200 in length and hundreds of char options for each char and some creativity), and will choose messages from that pool, by the nature of the filter some people will block it some will not, so some will report it as spam but some that would have will not, so the actual longevity of the bot account is a bit longer, until it reaches the critical number to get muted. This longevity parameter can be used to further refine the bot and rate the pool and so on.
At the end it's not the processing power it takes, filter size or anything, the problem it's that it may work for you, and another 10% that love to tinker with things like that (that will tenaciously update the filter for every spam that passes). But it wont solve the problem for the general population and has a potential to create more trouble than it solves when people will start to filter half of what they friends say.
Sorry for the wall of text, and mistakes and etc.. English is not my first language.
PS: They are using "c-0-m" right now....
I just have to disagree with your assessment of the enemy. They are not clever. They are not determined. They are sloppy, copypasting unimaginative uneducated slobs who are easily undermined. They use the same messages day in and day out until they change the price of something, and even then the rest of the message is the same.
I could block the following character:
$
And end all spam for myself for a very very very long time, I assure you. Nobody else in normal non-spam chat uses that character enough for it to matter. I think you may be forgetting something called "the fog of war", which is the concept that your enemy does not have perfect knowledge of you. None of the spammers are gonna know that I blocked the dollar sign character. They aren't going to be able to see my filter and adjust their messages. And the second they succeed in breaking my filter I fix it, and then they are left with nothing again, and they don't even know it. Right now the only reason I have spam is because they can send stuff to me and I am powerless to stop them. We all know how they worked around the current system of reporting/squelching. A ten year old child raised on nothing but Kraft Mac'n'cheese and Jerry Springer could have figured it out.
I think that maybe you are so used to seeing spam or just acknowledging its inevitability that you are missing something important here: if these botters were actually intelligent/educated and not just random rice farmers in china with a computer from 1986, they would have real jobs instead of griefing people in a videogame. These guys aren't brilliant, they're idiots.
All I ask is for the opportunity for us to prove it. Give me that filter. Hide it behind a settings menu where it has to be activated with warning signs flashing everywhere "WARNING: TAMPERING WITH THE FOLLOWING LIST MAY ALTER YOUR ABILITY TO RECEIVE MESSAGES IN-GAME", I don't care. I'm not scared.
Even if that length of time is a single day and not the week I've been using as an example in this thread -- it's still something that works, which is a vast improvement over what we have now, which is effectively nothing. I can take a few seconds to pop c-0-m into my filter and not see the spam for a day. I'm willing to bet that plenty of other people feel the same.
Remember how there are 1500 instances of "lair of the month" open because bots are running it? Think of the resources (CPU, RAM, electricity, management software) it takes to run 1500 instances (yes visualization helps but you still need the resources). It is the same people botting these instances that are spamming your chat log (their AD has to come from somewhere after all).
So, think of the average player in neverwinter who seems to have difficulty learning to play the game well, gearing themselves and earning AD. Are they even close to being able to manage a sophisticated set up like this?
This stuff is their real job, they make a lot of money off of it, otherwise they wouldn't be doing it. They have not been changing their messages because they haven't had to. no one uses a complicated system where you need to push 5 buttons to make something happen when you can just push 1 button instead right?
And i am very disappointed that you would simply ignore everything mickey1p00 said off as overestimation. You look at these people as if they are idiots who can't even think enough to drool. And you pretend that they are backwards and using 1986 computers to boot. Honestly, your perception of reality is pretty far off i think.
We are searching for slave labor, will pay with food from our farm!
They could start with the profanity filter, upgrade it to let the players add keywords to it, any message that trickers this spam filter will be send into a separated chat channel/chat window.
Once in a while you could check that channel/window for any "false positive" and adjust your own spam filter to it.
Such a spam filter can't be that difficult or time consuming to make... so what are the Devs waiting for?
And it would force the AD sellers to adjust their messages every time, over and over again... until the message itself becomes unreadable.
Your suggestion will force them to swap more rapidly once implemented, but it's not implemented...
If it will help globally? that's where our opinion differ. Actually I don't see spam at all, I don't have zone/lfg/wtt enabled for over 2 years now (since mod2 I think) I've never played MMOs before, so it took me time to realize that pugging and zone chat is not a good way to progress or meet people.
This is your opinion, obviously mine differ here too, I can't prove you either way, but from what I saw (I was curious long time ago and I looked at how they buy stuff, how they sell and so on, and there were some interesting documentaries about them), they are far from stupid, it's a real business (criminal), totally organized, efficient, and modern.
I hope I'm wrong and you are right, but looks like the spammers are still with us, adapting and making money inspite of the measures taken against them (not specifically in NW) - fools go extinct, they are still here.
I'm not against you having this, I just think there are better options out there that will help all players and not the few that will be in the arms race against the bots, like 'crowd sourcing' - making the /spam flag the message itself and filter on the server side.
Or/and combined with automated systems like variants detection and pattern detection.
Using a single manual system against something that can be automated (the message pool) will not end well, imo.
I guess we will have to agree to disagree. I hope if implemented it will work for you....