Fork in the Road

C'mon, you can take it.
FEBRUARY 15, 2009 8:33PM

Breaking News: OS Ratings Are A Crock

Rate: 29 Flag

Have you ever come across an obviously well-considered and constructed post here on OS, such as, say, this from Mick Arran, a thoughtful analysis of President Obama's possible policy shifts on Social Security, or this one on the DNC last summer from JTDtheGarlic.  These two examples, like many posts here, are original, insightful and thought-provoking.  And yet they each garnered maybe a 3 rating and a handful of comments at best.

 And you think, wow, if efforts as original, insightful and thought-provoking as these can only get a 3, what does it take to get much more than that ... a doctoral dissertation?  A Pulitzer candidate?

How about a barking dog or two?  Worth at least a 20. 

Now I like trolls and cake photos as much as the next guy.  But are they 3 times better than this essay on what one should do with one's life by Booknut?

Or maybe all it really takes is a shameless plea for rating, without actually writing anything at all, like this one, which is currently at 88 (!) and counting.  Maybe the sparkly font did it.

Perhaps you too have wondered what the logic is here ... is quality really a determinant of a post's ratings?

So I decided to put the quality theory to the test, by posting the highest quality writing possible, to see what ratings they might get.  Not talking about anything I personally have written --- I cannot possibly compete with pink trolls.  Instead, I turned to poetry.com's list of the World's Greatest Poetry and chose several as posting fodder for a little rating experiment --- namely, how does the world's greatest writing rate on OS?

First, I selected this beautiful poem by Len Roberts on moments with his young son.

Truly a touching piece.  Those of us who parent or have ever looked into the trusting eyes of a two year-old will surely understand.  And rate.  Right?

Wrong.  It got a 2.  Here it is.  And no fair rating it now.

Okay, maybe I'm just a sap for kids.  Let's step it up a bit.  And why mess around ... let's go for the top this time.   Shakespeare. Who's better than  Wild Willy hisself.

So I posted William Shakespeare's Love Sonnet #116.  Surely even the hardened cannot resist the tender genius of a Shakespearian love sonnet?

They can.  Both the hardened and all else can and did.  Sadly, Willy could only gain a measly 2 rating.

So what am I to conclude about the relationship between quality of writing and the OS rating scheme?  Hence my headline.

Your tags:

TIP:

Enter the amount, and click "Tip" to submit!
Recipient's email address:
Personal message (optional):

Your email address:

Comments

Type your comment below:
they are a crock..
OS is a lot like high school. There are people who always get noticed.. and a lot going on under the radar. It's really unfortunate.
Glad I could be the first on the rating action here. It's unfortunate that talented folks don't get the credit they so rightly deserve, but the Main page is so fast moving, that all you can do is kickdown a great title and hope for the best. And, if your not an adorable Troll or the Lovely T&D, it's hard to gain readership without the ever popular blog-whorishness. There are people, however, like Dorinda Fox that add read worthy links at the end of each blog, and it's much appreciated. Well, done, now to check out Mike Aaran and JTDtheGarlic.
There's obviously something you're missing here. the ratings are a product of the writers popularity, or...the exposure as an EP.

From what I can tell, quality, generally, is a small part of the equation. This sort of clique grading system in on every blog site there is. There's several writers here whom are constantly thunderously rated and commented on that I just plain don't get all the fuss. But that only means I don't get it. I don't get lots of things.

There's EP's that stay on the front page for days that are barely mediocre. From all the really fine posts available, I don't get that either. But that's the way it is.

But I think the real issue is, that here in OS so many good posts get lost in the churn. They really need to develop a bet system so that t
Open Salon is a cafeteria. I appreciate what you are saying, but you can't apply science to a cafeteria. If the ratings claim to prove anything objective, silly. I don't think OS makes such a claim.

(rated)
Charlie, an ingenious test and to think Shakespeare only received a 2. Something tells me, Toto, we're not in the Globe Theatre anymore.
I rate other members' posts, but I never notice the ratings mine have garnered. I couldn't care less. A comment saying that you enjoyed, laughed at, or learned from my words is far more gratifying than a rating of 80, or an EP, for that matter.
I posted some of my favorite poems by my favorite poet and with very relevant content to today's problems by Langston Hughes, it got 3 ratings. Poetry does not fly well on here. It's not the poster or the content, it's the subject matter. It's not "sexy".
So I'm somewhat in agreement with you.

(rated)
Well now what about the quality of the person actually performing the rating?

I would much rather have 1 rate from the Dali Lama and 1 from Malcolm X (of course they blog under nom de plumes to protect the idiots) for my grand measly total of 2.

That 2 would far and away beat the 88 rates from the teeming masses of Britney Spears listening skidmarks on the underwear of humanity.

Yeah blog rates are like High School. But if you are the dork sitting in the corner writing poetry on a yellow legal pad, the two kids who do come up to you, look at what's on the page, sink into thought for a minute and say "wow that's cool" have already far surpassed my own ability to empathize with my fellow misunderstood souls.

Just like when I waited tables, pumped gas into airplanes and did competitive comedic improv: It doesn't matter what they say about you ... it's the TIPS that count:)

And Charlie, only a bitter divorcée after an 8AM double shot of cheap gin would tip Sonnet #16.

Having worked in Public Radio for a long time I have stood witness to average citizens whipping out their wallets when they are moved by excellent writing.

There is the same thing happening in blogs. I am sure ( or at least I really really really hope) that there are plenty of writers/bloggers who rake in the tips and get crap for ratings. They may have had the sublime fortune of never having seen a troll cake...and I hope they laugh all the way to the bank making $1000 on a blog rated 2.
maybe people like sparkly fonts better than well crafted essays.
Hey! What's wrong with a three or a two!
To be fair, you should post Shakespeare with nudity.

People rate according to entertainment value and to what catches their eye. Kerry's picks are not a search for literary value, either. So yes there are many cases of pearls before swine. Do you have a solution?

I love Freaky and don't underestimate the talent behind that shtick. Open means open and that's a good thing. I certainly have had my eyes opened as to what can be done on a posting. But in the end you just have to find some opinions you trust and not worry about what gets rated as "great writing".
I never got Freaky, until I was on here for a bit. Freaky is the added potato in a too salty soup. I wish I knew how to put that clearer.
We have ratings here? OMG!
Thanks for mentioning Mick Arran. He is very good. I have rated his work but sometimes feel less than adequate to comment. I am so sorry Shakespeare was dissed ;0)
You nailed it. OS ratings have little to do with the quality of the posts. That little experiment of yours was very clever. RATED! I am also going to bookmark this page on my Mozilla browser.
Freaky is one cool and not overly obvious moderator with a wicked sense of humor. And she notices if one needs a pick me up or was ignored by readers.

I love Freaky.

One night during a live blog of the debate for an hour the whole front page was Freaky. That was funny.
People just wanna have fun. Yes, we want to be educated but short and sweet is the way to go to get ratings for news based stories.

Freaky Troll is awesome, she's like a female Cartman, she's awesome.
The rating system here is equivalent to bumper stickers: incomplete communication. There is no way to judge the quality of a post's content from rating alone. Nor can you judge quality of comment discussion which sometimes is better than the actual post. For instance, peak at Dr. Amy Tuteur's recent post about Science v. Religion.

I'm hopeful this is an issue OS will resolve in time, but I'm not sure how other than category ratings. Even then, the subjective nature of what one person finds entertaining, insightful, heartwarming, etc., may prove as meaningless as the current implementation. Like others, I think finding a way to keep recent posts on the FP would be more beneficial. Frankly, RSS seems to be the best recourse and it is provided.
One has to work to get noticed, and get on a few people's friends list, who will then often check out your blog. The feed is so quick that unless you're glued to the computer 24 hours a day, taking only the occasional pee break, you'll miss 99.99% of the stuff. And the EPs - a mystery as to their selection, and I can't imagine Kerry (and his staff of ??) actually scan all the posts...

So, it isn't a matter of quality...
Point well made Charlie. Hopefully OS will eventually develop a better classification system of posts so we'll know where to look. Good stuff gets lost here everyday. And some bloggers are just hoggers - but what's a guy to do?
Rated
If you want your post to get rated, put "hot lesbians" in the keywords.

Remember back in the days of the early internet where you could put metatags in your webpage? If you put terms that got lots of searches in there, you'd goose traffic.
I totally get what you're saying! I mean, hardly anyone read my masterpieces on Bon Jovi and Rick Springfield. What the hell is wrong with these Philistines!
Must agree with all
Lunchroom ratings are bogus
Corndog falls in snow
It is an open market here, open to all.
I don't care one way or another if I get rated or not.
I read blogs and comment. Sometimes I forget to rate. It is at the top and sometimes I forget to scroll back up and rate it.
There is a lot of great work here. Many good writers. Often some gets by.
I think Freaky is in a class by herself.
I read all her posts and laugh.
I conducted an experiment and proved, at least to my satisfaction, that if one is interested in the front page, ratings and lots of comments, it is possible to do so right out of the box. Check my very first posts, and you'll see what I mean.

How was it possible to get such reaction right out of the box? It helps to write reasonably well, have a modicum of wit, and offer a little entertainment value. And certainly, a salacious title helps-- just ask Dr. Amy. It also helps to "work" your audience, as any successful entertainer will tell you -- though Dr. Amy seems a little slow to pick up on that.

My posts were intended as a satire on Dr. Amy's pomposity and to provide a few days entertainment. I think they succeeded on both counts -- as did they many who read and commented.

However, my purpose on Open Salon is not ratings or a front page, but simply to be able to reach an audience of intelligent, thoughtful people, and that Open Salon has provided. Do I wish that more people read more of my other offerings? Certainly. And let me quickly add that I'm not admitting I have another identity since that would be a violation of TOS.

But this is at least as much a social network as it is a forum for writers and other artists, and as such, it is and will remain to some extent a popularity contest. So if you want to win election, work the crowd. Otherwise, do good work, and if it really is good, eventually a few good people will find you.
maybe with all the things going on (lay-offs etc.) people just want to have some fun, read some humor. but then again, people with talent should not be overlooked and part of the reason why they are not being read or rated is because of the OS format. today's cover has been the cover for two days now. what's up with that?
Dontcha know? Salon downsized, oh, sorry, rightsized its staff to make it more competitive with the other companies in its space and had to cut the reviews they do over the weekend in order to accomplish that.

Totally tongue in cheek. I have no clue what Salon is doing as far as staffing goes.
it's practically a full time job to get noticed on OS ... unless it's a "cut open your vein" topic. then a post might a little notice for a little while. But ratings? It's a job. People put a lot of time into reading and commenting on not only others' posts, but their own as well, keeping themselves in the feed as much as possible, especially keeping their posts in the feed. And people ask to be read, ask for comments, feedback, ratings ... really good stuff gets lost. and stays lost if the author doesn't revive it, even commenting on it himself. some people are here, believe it or not, just to write. true, everyone wants an audience. but ratings are gravy. If ratings are important, ya gotta work it. What puzzles me more is when bad writing gets on the front page, or gets an EP ... that I can't figure out. I can see the appeal of certain subject matters, but if it's poorly executed, I can't see bringing it out front. Then again, quality is not easy to define in an open environment.
I haven't been here long and there's certainly a learning curve to all of the commenting and rating. Like I think maybe I didn't respond to enough comments on one of my posts. I noticed that some people take great care to get back to everyone who comments. But that seems to require being available during a window when the comments accumulate. I also find that with only a certain amount of time to spend, I can't comment as much as I would like on some posts. Sometimes I rate and don't comment.

The political stuff is a little dicey--even the pretty good stuff. Whether people comment or rate those might depend on their political leanings. Some that I read I don't rate or comment on. I don't come here to get into political debate--but that's just me.

The most important thing though, is that I have read some very good writing here and have also been entertained by some really creative people. Freaky--who I, like other new people, didn't quite get at first--is truly creative.

Which is why posting a Shakespeare sonnet wouldn't get me to rate or comment. I've already read those; they are on my bookshelf with many, many other previously published works. I don't come here for those.
When I started posting, I got a few looks and was delighted. I then checked out the work of those who rated me and in most cases, added them as friends so I would see their work. I then looked at their friends lists and checked out work of other writers and found more whose work interested me. By commenting and when appropriate rating this stuff, I found more people checked me out. "Hey, someone new jumped in here. Let's check this guy out, see if he's any good." For my purposes, OS works best as neighborhoods. My friends often recommend new writers. All this requires a lot of time reading on my part. But as my Mom said, if you want a friend, be a friend. Some refer to these arrangements as "cliques," which I think is inappropriate as cliques are designed to keep people out. What they do provide is a group of 15 to 30 people who read my offerings. I am happy with this arrangement and stopped worrying about getting on the cover long ago.

If there is a writer you believe is worthy of attention, recommend this writer to your friends. You can also send these posts to the attention of the Daily Scrawl or Monte Canfield, who publish weekly lists of such suggestions in their blogs.
Just noticed both above mentioned lists are on the feed now. If you wish to help out an unnoticed writer, read, rate and recommend.
Thanks, everyone. Okay, have to admit a couple things here:
• Was secretly rooting for Shakespeare to get low ratings, suspecting that might be the case.
• Have no particular ax to grind against Freaky, or for her, for that matter; was just grabbing posts that seemed somewhat content-free and one of hers seemed to fit
• Love that people got the high school analogy right away, even though I buried it in tags
You might find other analogous experiments interesting. Peters and Ceci resubmited published psychology articles to various journal, changing the author affiliations from big-name to no-name schools (if I remember correctly); there was an effect.

Peters, D.P., and Ceci, S.J. ( l982 ). A naturalistic study of psychology journals: The fate of published articles resubmitted. Behavioral and Brain Sciences, June, 219-228.

And then, in literature, there's Jerzy Kosinski's Steps. From Wikipedia:

Steps (1968), a novel comprising scores of loosely connected vignettes, won the National Book Award in 1969. In 1975, Chuck Ross, a Los Angeles freelance writer, conducted an experiment with Steps by sending 21 pages of the book to four publishers under the pseudonym Erik Demos. The book was turned down by all of them, including Random House (which originally published Steps) and Houghton Mifflin (which published three of Kosinski’s other novels). Ross revealed his findings in New West magazine four years later. His article includes Kosinski's advice that next time he should offer the entire text. Ross repeated his experiment by submitting the entire text of Steps to literary agents in 1981, with equally dismal results.

On the other hand, it's not entirely a fair test. OS definitely has community aspects, which probably generate some ratings. And as Greg observes, OS may not be the best venue for poetry. Just for the sake of experimentation, I typed "Len Roberts poetry" into the Amazon search box and found that the two hits that come back each have just one (five star) customer ranking. Are Amazon rankings are a crock? Maybe; I don't know.

For a different sort of experiment, we might look at the very top-rated posts in OS. I think I see a lot of high-quality writing.
Rob, that's hilarious about Jerzy Kosinski. Almost enough to shake one's belief in big-time publishing.

Re OS being non poetry-friendly, so maybe I should do another test, this time with prose?
Charlie, you seem to have done fairly well, with a rating to date of 28!!

It is probably true in a lot of places, it is helpful to know and to be known, irrespective of whether we have any content of wider interest in the world. (of course thats not intended to be a backhanded criticism of yourself, merely a general statement that could be applied anywhere).

If one has had the time to become known it creates a greater chance for people to take a quick look at what the person has written. It is part of human nature that many of us are personality motivated. I am a very new member of OS. Therefore, why should I expect a lot of interest from others when there are so many others to choose from, others who you have got to know.
I find your experiment flawed, as I agree with Greg Thomas that poetry in general doesn't fly here at OS. I mean, I love poetry, but that's not why I'm on this site. So it's not hard to believe that no one rated those posts. A better experiment might be for you to reprint a story related to a hot-button topic that people here like to read about (politics, personal story, etc).

I do understand the frustration with the rating system, and how posts get on the Cover in general. I sometimes write posts that I'm proud of, but that get no love from my fellow OS'ers or the Editors. On the other hand, I write posts that feel like sort of throw off posts (i.e. my recent "What to Remember While Watching Dollhouse" post) only to see them show up on the cover and get lots of attention. So, while I welcome the hits, it's still amusing to me that my own sub-par work often overshadows my more quality stuff.

The reason some people get rated highly is because they're popular. I was an early poster, so I was able to have my posts noticed by more people (the feed moved much slower in those days, increasing the chance that my post would be seen by someone who would then subscribe to my blog via the Friends feature). Other posters do a good job coming up with provocative titles that a reader can't help but want to click.

I have certain writing styles and topics that attract me, so when I see those people pop up, I friend them and make an effort to rate their posts, in order to push material I personally support. I like the posters here who make an effort to publicize the work of others, so I always make sure to keep an eye out for those posts as well.