Professor wants data to look better than it is, is this ok?

Moderators: statman, Analyst Techy, andris, Fierce, GerineL, Smash

Singalore
Posts: 6
Joined: Tue Apr 07, 2015 11:33 am

Professor wants data to look better than it is, is this ok?

Postby Singalore » Tue Apr 07, 2015 11:41 am

I hope this is the right place to post this question.

I'm doing my thesis based upon the data of my professors research study. I want to look into wether 2 different psychological treatment conditions affects depression compared to a control group, how they differ, and wether former treatment and medication can predict the outcome (probably through logistic regression).

Frequency analysis of all 3 groups shows that only 25% had been cured. However, analyzing only the two active groups cured 38 individuals 27.1% which gives me a "better" result but a smaller sample size.

Any ideas on which path I should choose? Of course I want a better result, but it feels somewhat wrong to go for the smaller sample size. What would you do?
Last edited by Singalore on Thu Apr 09, 2015 1:15 pm, edited 1 time in total.
statman
Administrator
Posts: 2721
Joined: Tue Jun 12, 2007 12:08 pm
Location: Florida, USA

Re: Professor wants data to look better than it is, is this

Postby statman » Tue Apr 07, 2015 11:55 am

45 cases is simply not enough to meet both "power" "effect size" needs - Unless, the "population" is that small?

And ...
See the note below

NOTE: Please read the Posting Guidelines and always tell us your OS, the SPSS version and information about your study and data!

Statman
Statistical Services
Singalore
Posts: 6
Joined: Tue Apr 07, 2015 11:33 am

Re: Professor wants data to look better than it is, is this

Postby Singalore » Tue Apr 07, 2015 12:25 pm

Sorry, I'm working on a Mac, OS X Yosemite, SPSS version 22.

i'm not sure that I get what you mean. As far as I can see the power would not be enough either way, even if I'd make my analysis based on only the 2 active groups or f I'd include the control group. Since the study already has completed there is no possibility to increase the sample size. I'm more curious if it's ok to choose to present your data this way. If you've based your study on 3 groups from the beginning, shouldn't you include them all, even if it makes your result look slightly worse?

My total sample size is n= 209.
GerineL
Moderator
Posts: 1477
Joined: Tue Jun 10, 2008 4:50 pm

Re: Professor wants data to look better than it is, is this

Postby GerineL » Tue Apr 07, 2015 2:49 pm

It is most definately not ok to collect data in a third group and not report it because it looks better if you don't report it.
There is no question about that.

See for instance this link:
https://replicationindex.wordpress.com/ ... practices/


Also, don't just look at frequencies but run some actual tests, for instance, run a repeated measures anova and see whether your 3 groups differ from pre- to post-test.


[edit]
I just wanted to edit this message to make it abundantly clear that it is absolutely not ok to do this. This is what's wrong with psychology, according to many people. Go to any psychological conference and you will probably find on or more talks (or at least mentionings) of questionable research practices. It is a very important point, and I cannot stress enough how much it is not ok to leave out a condition, with no mention of it in a paper, just because it looks good. It is not only questionable science, it also goes against ethics, it is morally wrong, especially (but not only!) when you research interventions!!!!
Singalore
Posts: 6
Joined: Tue Apr 07, 2015 11:33 am

Re: Professor wants data to look better than it is, is this

Postby Singalore » Tue Apr 07, 2015 3:41 pm

Thanks for answering!

Well, that was my initial thought as well. I assume that my professor meant that I should mention it in my report, although he didn't mentioned it. But this makes me even more confident about using all 3 groups in my analysis!
GerineL
Moderator
Posts: 1477
Joined: Tue Jun 10, 2008 4:50 pm

Re: Professor wants data to look better than it is, is this

Postby GerineL » Wed Apr 08, 2015 7:41 am

If you have good reasons to test only 2 groups (theoretically or statistically e.g. the third group does not meet assumptions for the test you want to use), then I think it is ok to test 2 groups, but you should report it, and also report that the result is less favorable in case you do include this third group.
And I think that reviewers (or whomever reads the report) is going to ask you about that.

One more thing: Isn't it a good thing that you don't get results in your control group: I think this is whatyou would want right? That the intervention does something wheras the control group doesn't change?
This is something you should research with statistical models, and cannot answer with only frequencies.

Who is online

Users browsing this forum: No registered users and 2 guests

cron