Name a worse piece of research: Troppo competition

I am calling on all Troppodillians to nominate a worse research paper than this. From a very quick squiz the people who wrote the paper are against rape.  After an introductory poem the paper begins thus:

Women who are raped or who suffer domestic violence are somehow thought of in the popular imagination as a stereotype. According to this, the women are asking for it, dressed inappropriately, provoking it responsible for it.

No evidence that I could see was cited for this.

The methodology of the paper appears to be this:

  1. Interview women who have been raped within relationships
  2. Begin the paper with a poem
  3. Let yourself go

According to the paper, the overwhelming majority of the women who were interviewed said that the men who raped them wouldn’t recognise what they did as rape. The conclusion?

It appears that there is a disparity between the rights of women as expressed in Australian law and the way women are related to by their husbands, partners and professionals.

So there you go – rape really is on the end of every wolf whistle.  I guess, if I wasn’t tapping away on this keyboard, I could be raping someone right now, and unless there’s something wrong with the research methodology it’s overwhelmingly likely that I wouldn’t even recognise what I was doing as a crime.

I didn’t realise it before, but like so many things, it seems obvious when pointed out.

Subscribe
Notify of
guest

260 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
FDB
FDB
16 years ago

At least they can spell!

Yeah, that is seriously poor. I’ve had to mark far worse, but this is graduates, right? Parts of it read okay, with references(!), but you couldn’t publish that abstract with a straight face, could you?

Guise
Guise
16 years ago

Oh! so many examples. Especially in the climate change denial genre. But for a new front-runner, try this one:

The Gardasil ‘miracle’ coming undone?
http://www.onlineopinion.com.au/view.asp?article=7786

Verily, if I had that author’s e-mail address, I’d be adding it to my spam filter.

John Greenfield
John Greenfield
16 years ago

The depravity of what passes for “research” and “scholarship” among the Gender Studies/Women’s Studies set is matched only by its vacuity.

dr faustus
16 years ago

I actually couldn’t be bothered more than flicking through it, so I don’t know how good the content is, but it is pretty challenging doing empirical research on a topic like this. Any research on sexual assault, for example, is going to have problems with subject selection — those who attend a rape crisis centre are not a random sample and are not necessarily representative, but the same is true of those who report to police.

The actual content of the report looks like it has lots of quotes, which is a decent indicator of face validity in qualitative research. One of the obvious problems is that of external validity. Qualitative research like this isn’t about external validity (being able to generalise the results to the wider population), but about the experience of one subset of the population, but what I have read of the report doesn’t make that clear. I think Nicholas’ quote highlights this issue pretty well.

It’s probably not the worst research I’ve seen in what is a pretty politically-charged field (I used to work in domestic violence research, and that also has a lot of poor quality research), but is pretty much what you’d expect from what appears to be a community agency. Such places often don’t have a lot of resources or research experience, but are interested in making their experiences in working in the field public in some way. The writing style is quite jarring form the perspective of a professional researcher, but again, not outside what you’d expect from such a source.

The best thing about this sort of research report is that it’s a great way of proving that there is still a need for us folk who design and publish research for a living!

conrad
conrad
16 years ago

I think it’s generally called qualitative research

conrad
conrad
16 years ago

Sorry, Dr Faustus, you beat me to it.

lauredhel
16 years ago

You read that report – the whole thing, all the quotes, all the experiences of these women – and all you can come up with is a defensive, miffed whine about how you’re not a rapist?

Wow.

Amanda
16 years ago

Name a worse piece of self-absorbed bullshit troll-blogging. Jesus, it’s not all about you, it’s a report for and by health workers in rural areas not an undergradute essay. “A very quick squiz” will be more than I’ll bother with your posts in the future. FFS.

Nico
16 years ago

A report “for and by health workers in rural areas” should still not use hearsay in it’s conclusions…

Amanda
16 years ago

“Hearsay”? You mean what the interviewees said about their own experiences? Whatevs.

“And now in conclusion after countless hours of interviewing people we shall now discard the results. Soz, forget we said anything.”

In any event, Nick is not objecting to the methodology really because this could hardly be the most egregious example if it, even if the impressions culled from “a very quick squiz” and partly based on the inclusion of a poem were somehow significant, I submit his methodology in coming to his conclusion is many magnitudes worse than even the worst things one could say about this report.

Someone had the gall to write about women that didn’t include specific reference to him, that’s his beef. Nick demands his Gold Star for being a Top Bloke from the raped women of southern NSW and he didn’t get it and spat the dummy.

lauredhel
16 years ago

Nico: A report about the nature of women’s experiences with partner rape should use what, exactly, if not women’s experiences?

Jason Soon
Jason Soon
16 years ago

conrad is right. it’s a qualitative study. it has its uses. think of it as someone doing fieldwork instead of you so you get a feel for what’s going on second hand.

Nick- word of advice. If you find John Greenfield agreeing with you, reexamine your premises.

Melaleuca
16 years ago

Don’t be ridiculous Jason Soon. The opening para, if I may paraphrase, is saying most people think women who are raped deserved it. That is a bizarre and highly inflammatory view that has no reasonable basis in research.

I had to read and hear this type of crap from typically unintelligent grudge-bearing women during my social science degree and pretend to take it seriously. Nick is right to ridicule this nonsense.

John Greenfield might be a paunchy middle aged chap in a silk bathrobe and fluffy bunny slippers but that doesn’t mean he isn’t right once or twice a year.

Melaleuca
16 years ago

“I can even come at the idea that some of these men would have said in their pathetic way that the women were asking for it – with quite a few not actually even thinking that – but just lying to themselves or others to make themselves feel better about what theyd done.”

That’s fair enought too, Nick. But to generalise from “deviants” to the popular imagination is absurd …

Amanda
16 years ago

Nick: It’s a random bunch of slogans.
Reality: It’s 200 pages and I haven’t read it.

I am looking right now at the pages which talk about the way these actions are excused and minimised by individuals and organisations.

Mate, give up. You had a bad morning and posted rubbish in a dickheadish fashion. It happens. Stop digging.

Melaleuca
16 years ago

“Mate, give up. You had a bad morning and posted rubbish in a dickheadish fashion. It happens. Stop digging.”

Did I sit next to you in tutorials?

Amanda
16 years ago

Yeah, I was the fat hairy one in the boiler suit.

Melaleuca
16 years ago

Yes I know. I’m in furious agreement :)

Amanda
Amanda
16 years ago

So the crime against Scholarship committed here is there’s maybe a poorly worded generalisation on the first of it’s 200 pages? Clearly your personal outrage is not absurd at all.

John H
John H
16 years ago

Here’s a useless piece of research that dropped into my email last night. How anyone can earn brownie points off this type of research is beyond my reckoning. Did they factor in The Hawthorne Effect?

Hope Therapy for Depression.

http://psychcentral.com/news/2008/08/19/hope-therapy-for-depression/2778.html

….

Were finding that hope is consistently associated with fewer symptoms of depression. And the good news is that hope is something that can be taught, and can be developed in many of the people who need it, said Jennifer Cheavens, assistant professor of psychology at Ohio State University.

FDB
FDB
16 years ago

There’s a pretty fundamental problem with unsupported claims, tangential to the thrust of the research, in the first line of an abstract.

Rather than poor research, what we have here is poor writing.

The Worst of Perth
16 years ago

I was impressed by your link Melaleuka. I didn’t know I even needed a new vacuum tank.

Amanda
Amanda
16 years ago

Is it tangential to the research? How so? From reading it, it appears completely on topic.

Niall
16 years ago

Frankly, I find most sociologically based ‘research’ to be either over-generalised for fear of appearing accusatory or seriously subjective for fear of appearing over-generalised To each their own, but I prefer to make my own judgements.

Bingo Bango Boingo
Bingo Bango Boingo
16 years ago

Amanda, perhaps you could enlighten us. You seem to have read the paper. How much of the paper is dedicated to the ‘inappropriateness’ of dress within a relationship? How much is focussed on partner-involving rapes that were ‘provoked’ by a woman’s clothing? If the answer us: none, then you will see FDB’s point about tangential openings. I can’t really see how this relatively small issue in the abstract morphed into Worst Research Evar, so if that is the extent of it then Nick’s remarks are unreasonable. But we should at least be able to agree that screwing up the second sentence is not exactly covering yourself in scholarly glory.

BBB

Don Arthur
Don Arthur
16 years ago

Nicholas – I don’t understand why you posted this.

When I looked at the report my first reaction wasn’t to start critiquing the methodology. I couldn’t get past stories the interviewees were telling.

Did you read any of things the 21 women told the researchers? Did they sound like they were complaining about wolf whistling?

Laura
16 years ago

What Amanda said, actually. It’s time to give Club troppo reading a nice, long, rest.

Melaleuca
16 years ago

Let’s get this clear- criticism of the presentation of the report is NOT belittling the experiences of the women who were surveyed. Their stories are horrifying and I feel a great deal of sympathy for them and anger at the perpetrators.

Don Arthur
Don Arthur
16 years ago

And the last two paragraphs of your post?

JC
JC
16 years ago

Don,

Take a look at this sentence:

Women who are raped or who suffer domestic violence are somehow thought of in the popular imagination as a stereotype. According to this, the women are asking for it, dressed inappropriately, provoking it responsible for it.

It’s a more than a little offensive, no? Do you know of any person that would actually think a woman is asking to be raped and that it fits the popular imagination. There is nothing that can be said to defend this statement.

It does the victims a disservice as it belittles them and it would be reasonable to expect any of them to be damn angry with the writer for even suggesting this. It’s wrong and is actually putting the victims down because of its sheer inaccuracy in the way the vast majority of people would at victims of rape.

Amanda
16 years ago

BBB, for a start the various dynamics involved in the commission, experience of and any subsequent outside engagement in “partner rape” cannot be unilaterally divorced from those involved in “stranger rape.” There are others more able than I to cogently detail that. Also: The sentence says “women who were raped or suffered domestic violence” — see the word “or”? Women who are raped (by men, BTW. Apologies if that shocking revelation hurts your feefees), whatever the circumstance, is the topic of the flipping sentence. And if you aren’t hip to the victim blaming that goes on with this issue in all its complex forms then, I can but suggest you make yourself be.

Nick, you do realise that the post we’re talking about is right above these comments, yes?

So when you furiously retrofit in comments what the post was about and what you said people can just scroll up a bit and see that that was not, in fact, what it was about nor what you said at all? You realise this, right? Your claims now its purely about Felicitous Sentence Construction 101 do not fly.

You cherry picked a sentence at the start (one.fucking.sentence), a sentence in the conclusion and ignored everything in middle, including the many things which are directly relevant to your “concerns”. The rest was a frankly weird demand it Be All About You and some victim blaming of your own.

Michael
16 years ago

NG,

You think the study is actually worthwhile, but yours is just a complaint about the writing style?

Then your 2nd last paragraph seems like a bit of self-indulgent nonsense.

We could start a ‘name a worse peice of blog writing comp’ and put you up as the primary case.

JC
JC
16 years ago

Amanda:

To be perfectly honest the rest of the paper is essentially an interview process with the writer doing little else than drawing on the accounts of the victims etc.

“that little sentence” is the conclusion that only a dipstick could reach. Any victim would be perfectly entitled to feel she was figuratively raped a second time as a result, as they were obviously used to advocate something that was patently false.

So yea, words have meaning…. even ” that little sentence’.

Bingo Bango Boingo
Bingo Bango Boingo
16 years ago

BBB, for a start the various dynamics involved in the commission, experience of and any subsequent outside engagement in partner rape cannot be unilaterally divorced from those involved in stranger rape. There are others more able than I to cogently detail that. Also: The sentence says women who were raped or suffered domestic violence see the word or? Women who are raped (by men, BTW. Apologies if that shocking revelation hurts your feefees), whatever the circumstance, is the topic of the flipping sentence. And if you arent hip to the victim blaming that goes on with this issue in all its complex forms then, I can but suggest you make yourself be.

That’s a lot of words to use in not responding to anything I wrote. Well done. I should add that although I’ve pasted the whole paragraph, I didn’t actually read past this bit: “Women who are raped (by men, BTW. Apologies if that shocking revelation hurts your feefees”. From time to time we are all forced to read rubbish written by adolescents, but I figure that on the internet I’ve got options. Bye now!

BBB

Bingo Bango Boingo
Bingo Bango Boingo
16 years ago

I can’t help but think that a title along the lines of “Name a worse piece of abstract drafting” would have saved a lot of heartache here, Nick.

BBB

Michael
16 years ago

Nic,

It’s the first sentance of the Exec Summary. Why not start at the introduction if you find it so unbearable.

It’s a lengthy and useful piece of work, and it seems a bit churlish to suggest it might be the worst piece of research out there, based on two sentances.

Kim
Kim
16 years ago

I can’t for the life of me see a problem with the first sentence of the abstract. If they’d said “discursive construct” or something JG and Kevin Donnelly and all that crew would be screaming from the rooftops.

And aside from the last bit of this post saying much more about your assumptions than any supposed defect in the research, as Dr Faustus said, a piece of good qualitative research attains validity by being faithful to the subjects – reporting their words in depth is good practice because it reduces the interpreter effect of the researcher. The statistics on which the offending “stereotype” is based are plain from all sorts of other quantitative research.

This post is just prejudice.

conrad
conrad
16 years ago

“a piece of good qualitative research attains validity by being faithful to the subjects”

No it doesn’t. The idea is that you interview subjects and extract all the main themes from their stories. These then give you some idea of what the important issues are. As it turns out, if you interview a hopelessly biased group of subjects (as is evident here) or your interviewing technique gives hopeless biased comments, it means that you *haven’t* interviewed enough subjects or done it in a fashion well enough to get a good idea of the area and the main themes. If you then conclude things from your hopelessly biased subject pool, then all you are doing is junk and your study has no validity at all. If you also assume that the area you are in allows subjects to perfectly introspect the main themes, your study will also not necessarily be valid.

Michael
16 years ago

There seems to be some confusion over what this report is about.

It’s not a piece of research for publication in a peer-reviewed journal, but a self published report from a health NGO in rural Vic. For that, it’s not a bad effort.

As for a “hopelessly biased group of subjects”, I think someone is missing the entire point and misunderstanding the relevancy of bias.

Ken Parish
Admin
16 years ago

The opening statement about the stereotype of women “asking for it” certainly provides a misleading impression having regard to the body of the report. As Nicholas suggests, it’s a stereotype which no doubt fits many people’s attitudes towards “date” rape and “stranger” rape but appears not to characterise community attitudes to rape by a spouse.

The report deals with the women victims’ experiences of community reactions at pages 49-60, and I couldn’t find even a single report of anyone suggesting she was “asking for it” or similar. Instead, responses ranged from supportive (a minority) to denial (it couldn’t have happened, you must be misinterpreting or manufacturing the claim for vengeance or some such motive), an attitude that the wife has a duty to submit to sex and the husband a right to expect/take it, or an attitude that although rape might be regrettable the woman should be stoic and put up with it for the sake of the kids and preserving the marriage.

However, although the somewhat misleading opening seems to have alienated Nicholas at the outset, the report itself seems to me a perfectly respectable piece of qualitative research. The methodology looks to be an appropriate way of eliciting the experiences and understandings of a particular group of people (spousal rape victims in Victoria’s Goulburn valley), existing research is decribed thoroughly and both the women and people from responding agencies (especially police) are interviewed thoroughly on an individual basis by 2 researchers and in focus groups, and later counselled/debriefed.

As others have noted, several of the comments seem to misunderstand the nature and purpose of this sort of qualitative research. The comment about a “hopelessly biased group of subjects” is especially misconceived. They are seeking to elicit the experiences and understandings of a particular group of women, so accusations of sample bias are by definition irrelevant. As dr faustus explains in an earlier comment:

“Qualitative research like this isnt about external validity (being able to generalise the results to the wider population), but about the experience of one subset of the population.”

Such research seeks to capture the experiences and understandings of a particular group. The sorts of deep understanding it aims to capture almost certainly could not be examined effectively by quantitative methods. The findings might or might not generalise over time and different places; that coould only be ascertained by replicating the research elsewhere. I strongly suspect that most of them would be reflected elsewhere to a considerable extent.

One of the aspects of the report I found most striking was that, although the responses of friends, extended family, doctors etc were mostly experienced by the victims as negative, the responses of police (particularly specialist crime squad members) were mostly found to be appropriate and helpful. It suggests that the efforts of the Victoria Police to overcome entrenched prejudices have been very successful. It’s an encouraging observation in a report that otherwise mostly makes depressing reading.

Paul Norton
Paul Norton
16 years ago

What is most worrying about the opening post is not that such a frivolous, imperceptive and insensitive post could be written, but that it has been written by someone who I would never previously have imagined to be capable of demeaning himself in this way. I’m forced to conclude that Nick simply wasn’t thinking about what he was writing, and did not seek advice from others before posting.

As for “a hopelessly biased group of subjects”, can one imagine such a comment being made about qualitative research based on interviewing Holocaust survivors or former Gulag inmates about their experiences?

conrad
conrad
16 years ago

Mike & KP,

if the idea of the report is to learn about attitudes to rape, then I’ll stick by the claim that it’s completely biased.

Alternatively, if the idea of the report is to learn about what people who have been raped think about societies attitudes to rape, then it isn’t biased.

Let’s have a look at which of these is true:

Here is the first sentence:
“Women who are raped or who suffer domestic violence are somehow thought of in the popular imagination as a stereotype”.

That’s a general comment about everyone.

Here is the first thing that comes after the general blurb on their population
Recommendations:
Recommendation 1.1 Create public awareness of partner rape through the myriad forms of the media

That’s clearly using the data to make some general claim. What has the normal population go to do with this? It’s a qualitative study that doesn’t generalize.

I’ll go to the next paragraph:

“Name it as rape
Our health professionals, religious leaders, police and legal sector workers need education informing them that partner rape is a crime, and ongoing professional development about how to help people affected by it.”

I’d better watch out for those religious leaders (of which we have no sample that was interviewed), who probably think rape is just a big joke (Hilali excluded).

I could go on here, but many aspects of the report use languages as if they are talking not about the perceptions of people who have been raped attitudes to rape (which is what you learn from the qualitative data), but as if the results were general community attitudes. That’s clearly sloppy work if that is the intention.

lauredhel
16 years ago

Ken: I think where the criticism has gone off the rails is here: the opening statement isn’t meant to be a summary of the conclusions of the report. Just as in the majority of health sciences abstract formats (and, as has been noted, this is a long report, not a brief abstracted peer-reviewed article), the opening statement is a background statement, not a conclusion. It shouldn’t be taken in isolation and assumed to be such. It needs to be put in context with the rest of the introductory paragraph of background/summary, upon which it makes perfect sense:

Women who are raped or who suffer domestic violence are somehow thought of in the popular imagination as a stereotype. According to this, the women are asking for it, dressed inappropriately, provoking it responsible for it. While this is clearly uninformed, our sample provides yet more evidence that any woman is vulnerable to rape. We do not need to be a certain ‘type of woman, or to behave in particular ways, or to be in the wrong place at the
wrong time.

This introduction is situating the work not purely within the realm of parter rape, but within the larger problem of sexual violence against women.

It’s a largeish piece of qualitative research, with a series of recommendations, which are probably the closest thing it comes to to a “conclusion” as such. There is no soundbite conclusion to wave around and poke sticks at.

If there are people here who truly think there is no systemic problem (clue: “systemic” doesn’t mean “100% of people think this way”) with women in general being frequently thought to be responsible for men raping them, you’re really, really not paying attention. I suggest speed-lurking at a blog like The Curvature and abyss2hope for an education in the matter, with a wealth of examples from government and judiciary on down. Cases where a judge considered the issue of a frilly bra and G-string to be relevant when summing up a case of rape of a preteen girl, the prosecutor who labelled a raped ten-year-old girl “naughty”, the list goes on. Feel free to cruise unmoderated rape related threads at non feminist sites to get an idea of what people in general are saying about rape and responsibility.

It is also true that DV victims are blamed in other ways. The first thing people tend to ask is “Why didn’t she leave? Why did she put up with that?”, shaking their heads. The first behaviour examined is the behaviour of the woman.

If you choose to characterise the vast number of people involved in this sort of thinking as individual “deviants” … not much I can do about that, except say that I strenuously disagree – your definition of “deviant” is clearly far more all-encompassing than mine. There’s no need to delve into remote history for datapoints – just this week in the UK, the Criminal Injuries Authority said that it had been cutting criminal injuries compensation as a routine for women who had been drinking at the time that men raped them.

For an Australian example of a victim-blaming construction: “Police say sex attacks in the CBD could be reduced if revellers took care not to drink too much and not wander off with people they have just met.”

lauredhel
16 years ago

Another way of approaching this: perhaps Nicholas could give us some examples of what he would consider to be good qualitative research into women’s experiences of partner rape, so that we might compare and contrast.

Still haven’t heard back on what the penultimate paragraph was about. Perhaps if Nicholas wishes to retract it, he could strikethrough?

pbrosnan
pbrosnan
16 years ago

“interviewing technique gives hopeless biased comments”
Hopelessly malodorous statement …