Ratings: the downside

online reputation management

This is the framework all Troppo authors use in their online reputation management (ORM). KPIs are reported monthly. If you notice any Troppo authors going off track, please shoot an email to reputationnaughties@clubtroppo.com.au

This article by Maureen O’Dowd on the stress of getting a good rating as a user of a service like Uber is worth pondering. It’s important what ratings are for. They’re supposed to be for the exchange of information. But sometimes they become strategic – like they are, or can become on RateMyProfessor where professors get threatened with strategic strikes against their ratings if they won’t give students what they want – ie good marks or whatever else they’re after.

There’s another problem. Even when the players are not motivated by strategic considerations, they might not be rating each other in a way which is generating the most important information for those who stand in their shoes and who are influenced by their ratings. I had one bad experience on oDesk where a guy I’d hired got irritated with me because I wasn’t giving him clear directions. For me that was a feature, not a bug. I was both trying to optimise my time but also seeking to just gradually explore whether I could work with the guy – was he resourceful, could he figure out what I was after etc. I also asked him to look up some material on a paywalled site. I told him that he could subscribe free for an introductory period of two weeks.

Anyway, all this ended badly. He thought I was trying to exploit him and was insisting that he pay subscriptions to work for me – which I wasn’t. And he was frustrated that my instructions were not clear and copious. But here’s the thing. I was buying a service from him. So the way I see it, so long as I was prepared to underwrite any horsing around, any incomprehension of his of my mysterious ways, that is so long as I was paying him for all the time he claimed to be taking, then I think it’s not that relevant to my rating as an employer if, on the first project, he found me less clear than he would like. (A lot of jobs on oDesk, and a lot of the jobs people are looking for, involve instructions like “copy the data from this website, and paste it into this spreadsheet from cell A1 down to cell A1000.)

Over time if we develop a working relationship and he continues to find my style a pain in the arse, well then that might be relevant to other contractors. In the upshot, while I’m all for the democratisation/equalisation of the employer/employee | contractor/contractee relationship and see ratings as a very powerful mechanism for doing so (why didn’t economic reform develop in that direction years ago?) that has to be oriented around their respective and different needs. People paying for services generally want quality work and a range of ancillary attributes to do with effective cooperation with the person paying. The overwhelming need for the contractor is to have an contractee who won’t be trying to pressure or short-change them – and in the case of my situation on oDesk who will underwrite their claims to work, however useless, while they work out whether they can work together.


This entry was posted in Economics and public policy, Information. Bookmark the permalink.
Notify of

Newest Most Voted
Inline Feedbacks
View all comments
Crocodile Chuck
Crocodile Chuck
8 years ago

Sounds like a game theory problem, & an inability to find the Nash Equilibrium.

btw, http://davideharrington.com/wp-content/uploads/2013/01/Essay-on-Seatbelts-and-FARS.pdf

Moz in Oz
Moz in Oz
8 years ago

There’s also the question of what the site is for. You’re apparently looking for an ongoing relationship with one person that involves effort and trust on both sides. Your unhappy worker seems to have a much more transactional view of things, and possibly views it more as low-mental-effort, low paid work. Sit in front of the TV and make a few dollars type stuff.

My impression of most of the sites you’re discussing is that they are very transactional, and to the extent that they’re not people will likely move outside the site. I know some Uber drivers are all over the rules by having a second phone and giving that number to select clients – you see “your driver” is online in uber so you hit their unofficial number and work off the books. That way uber don’t get their 30% cut and you both win. That’s a hole in the model that’s very hard to block when people necessarily meet face to face. I think that’s great, it’s got the potential to turn the tables and see uber disintermediated and whining that it’s losing money because people are breaking the rules:)

8 years ago
Reply to  Moz in Oz

Taxis have used that model forever.