For members of Open Salon, probably the most irritating part of the "OS experience" is the amount of spam, especially during the evenings and weekends. In addition to being unwanted, intrusive, and sometimes downright stupid, spam also has other negative aspects. It clutters the database of posts, making searches for good posts more difficult. Spam comments clutter individual posts, sometimes to the point that discussions of the actual post are disrupted. Spam can also contribute to the slowness of the site.
Since many readers rely on the "Activity Feed" and "New Posts" queues, spam comments, spam posts and new spam membership announcements add to the volume of activity, thus allowing less visibility for legitimate posts. I think anyone who has posted on OS has had the frustration of seeing his or her posts fly through the activity and new post queues, propelled by a wave of spam. Probably the main reason people sign up on OS is because, unlike most other blogging sites, on OS their posts can actually be read. As spam steals attention and visibility from the legitimate posts, it directly undermines the reason why people sign up in the first place. In that regard spam is an attack on the OS community as a whole.
So when I was offered the opportunity to be one of a few "spam cops," I immediately accepted. Stated simply, being a spam cop means having access to utilities by which spam posts and entire accounts can be deleted.
Since becoming a spam cop, most of my time on OS has been spent deleting spam accounts. Having spent probably a couple hundred hours deleting spam and closely observing the practices and strategies of spammers, what follows are my personal reflections on the current state of spam on OS.
First, whatever you think about spam, however irritated you might because of it, it is probably worse than you think, probably much worse.
Spam comes in many forms, some of them not easily recognizable as spam, and the sheer volume of it not comprehensible unless you look for it. For example, everyone knows that there are spam posts, and the "watch live streaming video" posts are easily recognizable. But the way the activity queues work, you'll typically see only one post per account. In fact, spammers typically have 3 to 5 of such spam posts per account, and I have seen as many as 25 spam posts generated from the same account in only a few minutes. Unlike the manual tools available to the few spam cops, It's obvious that spammers are using programs or automated scripts. Thus the spam posts that you see flowing through the queues are only the tip of the iceberg.
Everyone can also easily recognize spam comments. But spammers have come to understand that obvious spam comments and spam on cover posts is usually deleted. So they go to the older posts where spam will be less noticed. There are some older posts that literally have 200 or 300 spam comments. And just like the spam posts, these spam comments occupy time and space in the activity queue.
And then there are the odd academic, business, or scientific posts, some of which appear to be made up of random phrases. You won't see any spam in the bodies of these posts, but if you look at the html source code you'll find concealed there a large number of spam links. And like the other kinds of spam posts, each account typically generates more than one of these posts, even though you'll see only one in in the activity queues.
Another kind of spam is what I call the spam link account. These accounts generate no posts or comments. Instead, the spam consists of spam links in the Links section of the main blog page. Since they have no posts or comments, these accounts can be difficult to identify; some members do not even know that such accounts exist.
In fact there are thousands of them, perhaps even tens of thousands. Just as one example -- do a search on the phrase "Interesado en" (Spanish for "interested in). You will find over 6,300 (yes, six thousand three hundred) spam accounts. On February 15 I did the same search, and there were around 5,700 accounts. This means that in the intervening 22 days 600 new spam accounts of this type were created, or around 27 per day. Keep in mind that this is just one type of spam, quite likely created by a single spammer.
When a spam cop observes a spam post, comment, or link account, he or she can delete the account using a special utility. Without going into detail, in my experience a spam cop can only delete an average of 2 accounts per minute. This means that even if a spam cop wanted to clean up the old spam, it would be almost impossible to do so. For example, for a spam cop to clean up the 6,300 "interesado en" accounts would require several weeks of work, 8 hours a day, on his or her own time for free.
Also, the way the queues work on OS, when a spam post is deleted, the post announcement continues to travel through the activity and new post queues. Unless you click on the post you won't know that it was even deleted. Thus even deleted spam posts continue to steal attention from legitimate posts.
Another problem is that spammers work 24 hours a day, 7 days a week. When there are no spam cops logged in, no one is deleting spam (unless someone in OS management decides to delete some. As far as I know most of them do not work during the evenings or weekends).
And finally, as I mentioned earlier, many spam accounts are difficult to identify. In particular, there is no reliable way to identify the spam link accounts, and their "new post" announcements typically clutter the activity queue with impunity.
My best estimate is that around 80 percent of all posts on OS are spam posts. As I mentioned before, you won't see all of these in the queue; in fact, you'll see only the tip of the iceberg. But the spam is there nonetheless.
Also, my best estimate is that around 95 percent of all new accounts are spam accounts. These include spam post accounts, spam comment accounts, and spam link accounts.
People typically think that Open Salon is a blogging site with a spam problem. It is more accurate to say that Open Salon is a spam site that also includes blogs.
The "spam cop" concept was a good idea, but, I believe, is inadequate to the task. We can neither handle the volume of new spam nor clean up more than a tiny fraction of the old spam.
One way to control spam would be to make it financially infeasible. This could be done by charging a small account setup fee, perhaps just $5. This would be prohibitive to spammers, but would be a tiny expense for legitimate new members.
Another way to control spam would be to moderate all new accounts. In this scenario, no new account would be visible until it published a post or comment, and the post or comment were approved. "Approved" here means being verified that it was not spam. This solution would not be perfect; a devoted spammer could make a legitimate post or comment and then follow up with spam. But it would take time to do that, and no doubt would discourage many spammers. It would also mean that legitimate members would have to wait some period of time before their contributions became visible.
The way things currently work, as soon as a spammer creates an account, he or she can generate a high volume of spam, and with no active spam cop there is nothing to prevent that. In addition, a spammer can create one account, 10 accounts, or 6,300 accounts, and it is quite possible that these accounts will be neither detected nor deleted.
Another possible solution would be to provide spam cops with better, more automated tools and reports. I think this is not a very good solution as it does nothing to address the conditions that permit a high volume of spam in the first place. But currently, spam cops are using a manual tool to combat spam that is being created by automated tools. Spam cops simply do not have the "firepower" that would allow them to be competitive with the spammers.
It would also be very helpful if deleted spam posts would disappear from the queues. Currently, when a spam post is published the damage is already done, even if it's deleted.
At various times I have presented these solutions to OS management, but so far have not received a response, either positive or negative.
What You Can Do
If you believe that OS should address the spam problem more forcefully, you might want to contact them to them to let them know your feelings. You may also have ideas for controlling spam that are better than the ones I have presented here. Though my suggestions have not yet moved forward, they have been politely received, and further feedback to OS management on the topic may inspire them to work faster to create a venue that is largely spam-free.
Thanks for reading, and I look forward to your comments.


Salon.com
Comments
I'll also send a note to the editors to take a look at this post, if they haven't already, and to voice my support for a paying system. I wonder how many spam cops are now working on OS, outside of any staff involvement. The fact that they got some volunteers speaks to their acknowledgment of just how large a problem it is.
I wonder though, if having all those spam links posted, both visible and invisible, actually help generate revenue for OS via page loads for passive advertising.
Rated with appreciation.
♥R
Before Open Salon I used to have a blog on one of the blogging sites -- can't remember which one -- and I paid $5 a month for that. And in the year that I had the blog, I think two or three people read one of my posts.
People sign up on OS because, unlike these other blog sites, their posts actually have a decent chance of being read. So there is a real service being provided here, and it seems to me that a one-time $5 fee would not be unreasonable, especially if it largely eliminated the spam. Anyone who didn't want to pay the fee could always have a blog on one of the other sites, and publish posts that probably no one would ever read.
bbd writes: " . . . It's appalling what some of the links are--one doesn't even have to click on any to see how vile some of them are just by reading the url titles. . . . "
Yes, many of the links appear to refer to porn sites, some even to child porn.
bbd: "I wonder though, if having all those spam links posted, both visible and invisible, actually help generate revenue for OS via page loads for passive advertising."
I suppose that's possible, but I would guess that a high volume of spam would tend to repel people from the site. I mean, if you happened by chance upon OS, and saw that four out of five new posts were "live streaming video" spam, you'd probably go somewhere else.
I have never been a champion of too-easy entry anyway. I don't think the site should be closed, but I do think that it doesn't hurt it to have barriers to entry. The reason for this is that the necessary rate to achieve is enough to keep us in a healthy steady state. If we're committed to ever-increasing the number of people posting, while never increasing the size of the cover page, then we're committed to decreasing placement of any individual on the cover because there's no other way to make that equation work.
Good question. I also have never understood that, especially when the spam is ridiculous. I mean, when I read something like "We have Ugg boots, you will be happy feet, and we have service to happy the most," it's not like I'm going to rush out and send $50 to Quong Xian in Fujian, China.
If OS doesn't want the solve the spam problem, the least we could do is to pay for English lessons for the Chinese spammers.
By the way, I just got a spam private message from some lady who saw my profile and wants to begin a romantic relationship with me. Let's just say that she sent it to the wrong person, and she no longer exists on OS. One of the benefits of being a spam cop.
Mumbletypeg: "OS looks immensely better than it did a few months ago in terms of spam, but I didn't realize it was volunteers who made it so."
Yeah, the spam cop program really has had an impact. It was a good idea, and whatever other measures might be taken, we'll always need some people around who can go after the spam that does get through. My point is that we need something more, something that would change the balance strongly in favor of the legit members and against the spammers.
But we do need to give credit to OS management for the measures they have taken, in particular the spam cop program and the large reduction in comment spam. We need to build on those and attack spam at the source, by not letting the spammers get accounts in the first place.
I tip my hats to the spam cops but this is not working and something needs to be done. If nothing else the slow 'loading' of pages here drives me to insanity.
I do hope something changes Mish.
Mish, I don't love you as much, I think it's the beard!! :D
Anyways, other sites have controlled their spam without instituting a Pay to....and without having to bring on volunteers to police the spam, cause, no, even with 100s of Spam Cops(I doubt there's that many out there on here, heck ,I know of two! And maybe one diseased wombat!!!) it's beyond controllable!!!
Also, instead of diddling with the 4hour - 12 hour feed, fix the server's speed and maybe the Spam problem!!! Probably sit down with folks such as Cat and yourself, who have fought the battle, others as well, learn and build, get better!!
Then maybe the revenues from the 5.3 million hits will come rolling in!!!
Just saying....a cat's 5 cents of information!!!
**Wanders off**
`
I have no notion. I am so ignorant.
Gadget contraptions bewilder me.
`
Let's get a Bulldog. Over dinner
discuss the texture. Measure
the size of a pooches stool
Last I knew there were four people who have The Power of spam cop. My participation in that role has been inconsistent. Sometimes I get really pissed and fall upon the spammers like the Wrath of God. Other times I look at the continual river of spam, become discouraged, and think "why even bother?" As I said before, the spam cop program is a good idea, but really needs to be part of a larger, more aggressive strategy.
As femme forte noted "The problem really needs a more comprehensive solution."
I like the concept of "the fence." Open Salon is like a little outpost in the wilderness of the internet. Currently, anyone can walk in and crap all over the place, and all the spam cops can do is make them leave and clean up the mess. That's because there is no fence around the outpost, no "barrier to entry" as Kent Pitman calls it.
Because there is no fence, even after we make them leave they keep coming back. I have deleted up to ten spam accounts for the same individual in a single evening. We delete someone, and two minutes later he's back on with a new account. Some of these guys actually create multiple inactive accounts, just to have them on hand in case they need them. So you delete one account, and poof! -- one minute later the guy is posting spam again on one of his standby accounts.
nanatehay writes: "I've long suspected that things are even worse than they appear . . . "
Take last night for example. In one 15 minute period there was one new legitimate post and 34 spam posts. The spam came from 4 different spam accounts, one of which had published 25 spam posts in the space of a few minutes. What people saw in the activity queue was 4 spam posts, when in reality there were 34.
It's an old photo. I shaved since then, so you can go ahead and love me more.
Tink: "Anyways, other sites have controlled their spam without instituting a Pay to....and without having to bring on volunteers to police the spam . . . "
Yes, there must be a more technical solution, but I'm not a tech guy so I don't know what it is or how much it costs.
Some sites have fallen into the 'Moderation' right at the step of sign up. It's fairly cheap, it's been suggested early in the beginning of this 'cluster' of Spam, which actually began awhile ago on Open(well over a year and getting close to two....)
I'm not sure why the 'Wait till we update Salon....' is the attitude though around here, you're not gaining a thing with the Spammers, causing issues inside the site, and well, it's just bad business practice to allow it to continue.
I have a pretty good theory on the "Whys" Spammers do their thing, it's called, "Search Engines", they think, and with good cause, that the search engines pick up their crap, so if I search for 'Nuclear Fission' I'll get http://www.aspamsite.com in the intial search results and there's enough dumm dumms to click on the site.
Course, Open is a good friend to Google search, most the time, you can search for a topic and 99.9 percent of the time, open.salon will be in the first few pages of results.
Throw out a big enough net, or spam comments/articles, and wham, HELLO HITS!!!
Good times.
Also, maybe the slowing down of the response of open.salon is a trick by the big wigs to make the Spammers go away, 5 to 10 minutes between click and response is making me want to wander off back to Yahoo! Groups!!
~:D
As for your proposition, I am already donating to OS by giving my intellectual (okay I'm stretching it) content away, but I would HAPPILY pay $5 or $10 to open an account. And I would gladly pay another $20 or so a year to be rid of the Googlebots and Adsense.
I concur with people above about how long it now takes to access articles and read PMs. Earlier this afternoon, I sometimes got error messages when I tried accessing OS articles, because it was taking too long to access the webpage. This does not foster increased readership…
I gather that my own fave solution, IP blocking, isn't technically feasible or effective.
It's indeed a great pity that all those at Salon and OS who are scratching, when they need to be solving, are not willing to take aggressive steps to rid OS of vermin. WHAT ARE YOU WAITING FOR?
We have suggested hundreds of solutions. And the only one we can see that's been accepted is to hand off responsibility for spam tracking to member volunteers!
It's obviously a much larger problem than 95% of us can even appreciate, but it is amenable to resolution through a number of more sophisticated approaches - WHY WON'T YOU STEP UP? It's going to be a great loss to Salon, and it's ownership group, when OS deteriorates to the level of the lowest common denominator.
It's a SPAM site, with a few thousand people who mistakenly think it's a BLOG host. Kerry? time to address the State of the Site again. Where do I send my $5?
The "why" gets me most of all. 24/7, even automatic, requires maintenance, for no apparent gain ( I'm assuming no-one on OS clicks on spam ) - so, what's the "why" of spam ?
Any prospective advertiser will want to see a demographic break-down, so it can't be that.
In 14 months I've been blithely free of spam on my posts, but I detest its presence in the feed.
Thank you for your efforts.
If TPTB don't like that idea I'd like to know why.
I still believe it was the advent of Adsense, back in June 2009, that attracted the hordes of spammers, and I'd be curious to know if my impression is accurate. I fell all over myself trying to warn everyone at the time (or, er, I should say Cindy did) but it did absolutely no good.
Whenever there is anything worthwhile and good, someone with the soul of a merchant will come along and try to "monetize" it. This attitude is a blight on humanity.
The "numbers" created by the spam traffic seem to be welcomed by Kerry, as noted in his comparison between Feb. 2010 and Feb. 2011.
That's an issue I had not thought about. If there were a $5 fee (or any fee) then it's literally not free. To the extent that new members are drawn here by "free blogs" or other similar searches, I suppose that might have an effect -- though $5 is about as close to free as you could get.
It's a topic I know nothing about -- how people find out about OS. I'm wondering -- in the signup process, are new members asked "how did you find out about OS?"
It is now 12:30 at night where I live, and I just finished off the day by deleting six more spam post accounts, two of which had five or more spam posts.
I also deleted one spam comment account that had left 250 (yes, you read that right, two hundred fifty) spam comments. Most of these were from last August, and I came across him by chance.
I'm signing off now, so as far as I know the spammers will have free and unfettered access to the site for the next few hours. Such is the current system.
Thanks for explaining how it works, and doesn't work as well as we'd all like. I was wondering.
I have another angle on a solution.
Assuming the spammers post here because it is effective on some level, let's find those OS readers who are buying the knock-off Gucci and Nike products and beat the crap outta 'em.
I'm not really in touch with other spam cops, which is to say that I haven't regularly communicated with the other ones about the spam cop program. From what I hear Catherine has been quite vigilant in deleting spam, and I have read a number of very positive comments in other posts about her efforts.
My point is that the volume of spam is so large that it is beyond the efforts of a few volunteers to control it, regardless of how dedicated those volunteers might be.
That's why spam has to cut off at the source, by placing some kind of barrier -- moderation, signup fee, or whatever -- between spammers and the site. Once a barrier is in place THEN spam cops could delete whatever small amount of spam might leak through the barrier. In that context a spam cop program would make sense.
I'm not saying that the spam cop program was a bad idea. It was a good idea, but now we know it's limitations, and that's good information. Now it's time to move beyond the spam cop idea and have more aggressive and systematic controls in place.
It is sad that it comes down to that, though.
The only concern I have is that if they made the fee too high a lot of people might leave. There's great variety here and that's one of the things I find most appealing, sometimes it's like opening a little treasure chest. Perhaps others don't feel the same way and don't care as much about the variety. I don't know it seems to make things very hard for the real writers and the bloggers alike.
Perhaps that's the problem for management. Maybe at some point they'll divide this into two sites, one for those who are professional writers and one for everyone else.
I don't know if there's a way for them to moderate new accounts where you submit the first post for approval and then have people volunteer to review them at 30 & 90 days (or something) based on account opening date. I worked for a start up dot.com for a few years, it was a nightmare. Good luck and thanks again for the post and the police work.
Attempt at humor has passed. I really do value this site and wish we could do something to relieve the precious front page space of all the 'spam-ertising.'
The automation of tools for deletion is a top priority.
How about putting a gamer FPS interface to shooting down spam posts and comments? Perhaps OS could waive the $5 fee for OS'ers who volunteer to pick up a plasma rifle once a week...?
Captcha
http://www.captcha.net
I am slightly exaggerating, but the Captcha method of requiring a person to enter a Captcha code prevents all automated scripts from creating accounts, publishing posts, or posting comments, without a live human entering two words first to prove they are a human.
If this does not entirely prevent spam accounts/blogs/comments, it would at least act as a huge dam to slow the gush to a trickle.
And nobody pays to start an account.
And an awesome thing about reCAPTCHA is that each time a live person enters the two words to verify they are a human in order to create an account, publish a blog and post a comment, they are helping to digitize books, newspapers and old time radio show scripts.
http://www.google.com/recaptcha/learnmore
aren't we off the market now though? I'd pay 5 a month, with the stipulation that we need a pool for people who can't afford that. Perhaps OS could hold it's own as a subscription service? We won't pay for newspapers anymore :D but we'll pay to hear our own voices.
Amen to that! I read your post, after reading your other post about getting ragged on by someone. All I know, is I was finally able to get back onto OS. It has been months! I gave up when it constantly timed out. Thank god for you, and whoever the other "spam cops" are. Without you, this place would still be inaccessible to most of us.