18th April blog comments.

1. http://columsblog.wordpress.com/2012/03/25/making-the-case-for-more-field-experiments/#comment-81
2. http://relativelydazed.wordpress.com/2012/03/20/alliteration-hell-yeah/#comment-51
3. http://psucb0.wordpress.com/2012/03/25/can-correlation-show-causality/#comment-58
4. http://skakov87.wordpress.com/2011/12/09/is-it-dishonest-to-remove-outliers-andor-transform-data/#comment-42

Case Studies

This is the last stats blog of the semester (and hopefully my life), so I thought I’d write it about something I actually had an opinion on to start with! Case studies!

There’s been quite a few blogs about case studies and whether they’re any good, and I think they’ve taken a fair beating. Personally, I think a lot of research would either not have started or wouldn’t be going anywhere if it wasn’t for case studies. Of course, they have their disadvantages (like everything does);

• Results from one case study can’t be generalised to a wider population
• Generally too unethical to replicate
• No control over environment, the case is studied purely in hindsight

Saying this, the disadvantages of a case study can often turn into an advantage;

• studying one person/case means you can’t generalise the results, but it does mean that you get more detailed data
• A lot of case studies can’t be replicated because it would be too unethical, but with case studies, we can still see the effects of a particular event that occurs naturally but would be too unethical to create. For example, sticking a metal pole through someone’s brain is an ethical no-no, but luckily for us (not so lucky for him…), this is exactly what happened to Phineas Gage (http://metablog.borntothink.com/wp-content/uploads/2011/07/1994-Damasio-Descartes-Error.pdf), and it gave researchers at the time something to go on when looking into what different parts of the brain are responsible for.
• We have no control over the environment (because we don’t know what’s happening at the time, we just hear about it afterwards), but this means that it is completely natural, so in a way doesn’t that make the results a little bit more generalisable?

Obviously, there are endless discussions about what’s good and bad about case studies and which is more important (do you want detail or generalisability?), but in my opinion, the benefits far outweigh the costs. Case studies have the ability to spark new research or give us more insight into topics that are too unethical to study, and for the control that case studies lack, follow up experiments based on said case study can make up for! Win-win for everyone, I suppose.

One case study that particularly got me on this pro-case study rant was the study of the ‘wild child’, Genie (http://www.youtube.com/watch?v=5vnSUfZhZNQ). Anyone who did A-Level psychology probably already knows the details, but the general principle is that a child was brought up without any human contact – she couldn’t walk properly, speak and seemingly had no emotional attachments to anyone. The conditions she was brought up in were too unethical for any lab experiment to replicate, or even be remotely similar to, which is why I think (in this case especially) case studies are so useful. I found the study really interesting, and its worth having a look at if it’s the sort of thing you find interesting and you’d like to avoid all those assignments we have due in…

 

 

Comments for Blog 3 (14th March)

1. http://alreadyconscious.wordpress.com/2012/03/11/can-correlation-show-causation/#comment-30
2. http://suuzblog.wordpress.com/2012/03/10/bias-in-psychological-research/#comment-65
3. http://relativelydazed.wordpress.com/2012/03/09/its-not-the-size-of-your-blog/#comment-47
4. http://psucb0.wordpress.com/2012/03/11/is-there-anything-that-cant-be-measured-by-psychologists/#comment-48
5. http://nirapsy.wordpress.com/2012/03/11/we-are-the-same-just-different/#comment-46
6. http://lisamarieoliver.wordpress.com/2012/03/11/are-case-studies-beneficial/#comment-36
7. http://jameezio.wordpress.com/2012/03/11/the-psychology-of-exams/#comment-50

Correlation Doesn’t Mean Causation

Even when two variables are strongly correlated, it still doesn’t necessarily mean that one causes the other; it could mean that another variable (a confounding variable) causes both. A very original example: increased ice cream sales may be highly correlated with sunburn, but neither one causes the other – the sunny weather is most likely to be the cause of both. This is known as the Third Variable Problem. It is a problem in research for obvious reasons – it can distort the results and increase the likelihood of a Type 1 error (stating that there is a significant effect when there isn’t). The real life implications of this problem, in my opinion, are more serious.

One study (Lawlor, Smith & Ebrahim, 2004) found that there was a significant decrease in the likelihood of Coronary Heart Disease (CHD) for women taking Hormone Replacement Therapy (HRT), suggesting that HRT protects against CHD. But, after investigating this further, they actually found that the socioeconomic status (SES) of the women was what affected their chances of developing CHD; the women with a high SES did more exercise and had a better diet, which decreased their chances of developing CHD. After looking into the relationship between HRT and CHD further, they actually found that HRT increased the likelihood of CHD. Obviously, if this study was taken at face value and not investigated further, it would seem that HRT reduced the chances of CHD, when in actual fact it was the opposite!

The ideal solution to the Third Variable Problem would be to eliminate the confounding variables altogether. Realistically, though, this isn’t possible. Can you ever be certain that you’ve eliminated ALL possible confounds? Or are there likely to be confounds that you hadn’t even considered? Maybe, even if you’re aware of the confounds, it isn’t possible to remove them – in which case, the best option may just be to be aware of the effect of the confound and keep it in mind. This is also the conclusion that Lazarus, Delongis, Folkman and Gruen (1985) came to when studying stress.

 

Lazarus et al: http://psycnet.apa.org/journals/amp/40/7/770/

Lawlor et al: http://ije.oxfordjournals.org/content/33/3/464.short

Comments for Blog 2 (22nd Feb)

1. http://columsblog.wordpress.com/2012/02/19/the-advantages-and-disadvantages-of-case-studies/#comment-59
2. http://jameezio.wordpress.com/2012/02/19/case-studies-worthwhile-or-a-waste-of-resources/#comment-38
3. http://joestatsblog.wordpress.com/2012/02/19/sona-system-the-best-way-to-collect-data-for-dissertations/#comment-42
4. http://psud43.wordpress.com/2012/02/19/advantages-and-disadvantages-of-case-studies/#comment-44
5. http://suuzblog.wordpress.com/2012/02/18/can-correlation-show-causality/#comment-55
6. http://klbpsych.wordpress.com/2012/02/19/can-correlation-show-causality/#comment-51
7. http://mdscurr.wordpress.com/2012/02/15/why-we-should-not-offend-animals-by-saying-their-results-cannot-be-generalised/#comment-53

The File Drawer Problem

The File Drawer problem basically claims that our view of a certain topic is distorted due to the research that is (or, more to the point, isn’t) reported. The idea is that when a study has significant results (i.e. shows a relationship between two variables), it is reported but when the results are insignificant, the study is ignored and locked away never to be seen again. Rosenthal suggested that 5% of studies are reported with a Type 1 error (saying there is a relationship when there isn’t) and the other 95% show insignificant results, so are stored away and never reported.

For obvious reasons, this is a huge problem as if it is genuinely as serious as Rosenthal suggests, our view of certain concepts in psychology would be completely distorted. For example, if for every 10 studies examining the difference in aggression between girls and boys in primary school, 9 found no difference but one found that boys were more aggressive, the studies with the significant findings would be reported. So, if there were 100 studies, 10 of them would support the idea that boys are more aggressive than girls, and these would be published, so we would consider this to be significant evidence supporting a theory. However, there are 90 ignored studies which found no difference in aggression between boys and girls. If ALL of these studies were reported, our view on aggression in boys and girls would be very different – we’d be more likely to assume that there’s absolutely no difference whatsoever (because 90% of studies say there isn’t), rather than assuming boys are more aggressive because ALL the studies that are published say so. Shadish, Doherty and Montgomery investigated marital and family therapy research and, after asking for researcher’s non-reported results, estimated that there are as many unpublished results (in the ‘File Drawer’) as there are published. Results like this haven’t just been found in psychology, Csada, James and Espie found similar problems in Biological research.

The obvious fix to this problem might be ‘report everything’, but in my opinion, this would result in an excessive amount of research which may have been left unreported for reasons other than ‘the results weren’t significant’. So, maybe there’s no way around this File Drawer problem, and it’s just something that we should keep in mind when making a decision about something based on the research we read about it.

Rosenthal: http://psycnet.apa.org/journals/bul/86/3/638/

Csada, James & Espie: http://www.jstor.org/pss/3546355

Shadish, Doherty & Montgomery: http://www.sciencedirect.com/science/article/pii/0272735889900135

 

Blog comments for 8th Feb

http://suuzblog.wordpress.com/2012/02/04/the-stats-of-valentines-day/#comment-43

http://statsblog2011.wordpress.com/2012/02/06/ethics/#comment-31

http://psychosomething.wordpress.com/2012/02/05/my-super-sweet-16-child-abuse-on-television/#comment-28

http://iamjackscompletelackofsuprise.wordpress.com/2011/12/09/do-ethical-guidelines-restrict-research-possibilities/#comment-22

The effect of historically relative ethics and diagnostic criteria

These two articles are what got me thinking about this whole topic (which was lucky, really, because there was a blog due…)

“Ann Widdecombe Defends Gay Conversion Therapy”

“The therapist who claims she can help gay men go straight”

Basically, Lesley Pilkington tried to turn a gay man straight using therapy and everyone was outraged…and then Ann Widdecombe defended her and everyone was doubly outraged. Lesley Pilkington is 60 years old and Ann Widdecombe is 64 – both brought up in a time where being gay was considered a psychological disorder and it was perfectly normal for psychologists to try to ‘cure’ it, in the same way psychologists try to cure psychological disorders now. In the 1970s, homosexuality was taken out of the DSM and is no longer considered a psychological disorder. Obviously, there is still some controversy, a lot of older generations still aren’t accepting of homosexuality, but generally speaking, in the present day, being gay is considered a completely normal and healthy way to be (and so it should be).

So this is all fine and well and simple to understand; the DSM folk saw the error of their ways and fixed it, and now everything in the DSM is DEFINITELY a psychological disorder, so no more trying to ‘cure’ people of things they don’t need to be cured of. Maybe? My paranoid little mind has reached an entirely new level of paranoia here, so excuse the drama, but what if we still consider some things psychological disorders when they’re actually not? Consider Schizoid and Schizotypal personality disorders, which are basically defined as being ‘odd’ or ‘strange’.  Do we really have the right to try to change it? Just because we don’t consider it to be ‘normal’? Are we trying to play God, believing we have the right to make people however we want them to be? Or do we not understand enough to be trying to change people? I’m sure when people were trying to ‘cure’ homosexuality, they weren’t doing it to be harsh and unaccepting, they just didn’t understand, considered it abnormal and tried to fix it, much in the same way that psychologists are trying to cure psychological disorders now.

There really is no definitive argument for this, because there’s no way to know whether the definitions of disorders that we have now are right or just historically relative, which means that the research we do may be considered completely ethical now, but in 50 years time, it might not be. Only time will tell, it’s just interesting to see how much has changed and how much could potentially change in the future. Maybe we’ll all be judgemental, narrow-minded old people in 60 years time!

It’s surprisingly hard to find examples of the studies which tried to ‘treat’ homosexuality, but here’s a few I’ve found to do with aversion therapy:

http://www.sciencedirect.com/science/article/pii/0005791672900717

http://psycnet.apa.org/journals/abn/81/1/60/

http://bjp.rcpsych.org/content/115/529/1417.short

Week 11 Comments…Merry Christmas Julie!

1. http://psud74.wordpress.com/2011/12/04/must-the-need-for-documented-ethic-procedures-hinder-research-progress/#comment-19
2. http://alreadyconscious.wordpress.com/2011/11/25/is-it-dishonest-to-remove-outliers-and-or-transform-data/#comment-23
3. http://psud6d.wordpress.com/2011/12/09/milgram-poor-show-or-a-step-in-the-right-direction/#comment-29
4. http://statisticalscrutiny.wordpress.com/2011/12/09/must-the-need-for-ethics-procedures-hinder-research-progress/#comment-22
5. http://statssam.wordpress.com/2011/12/09/does-the-need-for-documented-ethics-procedures-hinder-research-progress/#comment-24

Internal Validity vs. External Validity

Essentially, is the control of the experiment (internal validity) more important than the ability to generalise of the results (external validity)? In my opinion, the right balance or combination of the two types of validity is more important than a study being high in one but low in the other.

For a study to be considered high in external validity, it has to be as close to a real life situation as possible, in order to generalise the results of a study to real life situations. For example, if you wanted to know how parents react when their children misbehave, you might secretly film households to see how parents react when their kids start misbehaving. But, there are obvious limitations to this – in this study, you have no idea what is actually causing the parents response to the child’s behaviour, you can assume, but you can’t be certain. In other words, you can’t control confounding variables in observational studies, which is a huge threat to internal validity.

A study is high in internal validity when it is controlled for all possible confounding variables, so that the only factor which affects the dependent variable is the independent variable. For example, you have parents bring their children to the lab, assess and control the parents mood, and any other things that might be affecting the reaction of the parents (work stress, tiredness etc), then see how they react when their child starts misbehaving (they’re kids, they will inevitably misbehave, it’s just what they do). This is obviously ideal, because you know you’ve controlled everything, so the reaction from the parents isn’t a result of their bad mood or crappy job or anything else. With an observation, there’s a million other things which could be taking place which you aren’t aware of  (the parents shouting at the kids might actually be a reflection of the parents’ bad mood, rather than a reflection of their general response to their child’s bad behaviour).

So, there are obvious pros and cons to having a study high in internal or external validity, and unfortunately, having a study high in one means its low in another (because life’s just cruel like that). So, the best way to tackle this is just to meet half way – control your study enough that you can be relatively certain nothing else is effecting the dependent variable, but also keep it as realistic as possible, so the results can be generalised to real life situations (it’s better to be able to say ‘this is how people will act when this happens to them’ rather than ‘this is how people will act when this happens to them…if they’re in a highly controlled lab environment).

I think Milgram’s obedience study is a perfect example of the right balance between internal and external validity (if you have somehow been entirely absent for the last year, or you just want to refresh your memory a bit, I’ll leave a link to the abstract for this study at the end). It was high in internal validity, because it was a lab setting, with all the wonders of controlling confounding variables, so the behaviour of the participants was attributed to nothing other than the fact they were basically just doing as they were told, regardless of the outcome. But, at the same time, it was also high in external validity, because although the situation wasn’t actually real, the participant fully believed that it was which means that they were acting how they would do if they were put into a situation like this in real life.

Milgram: http://psycnet.apa.org/journals/abn/67/4/371/