Research Critique Assignments One skill that should be acquired in upper-level Psychology courses is the ability to read and critically evaluate primary research articles. Students (actually most people in general) often find academic writing dense with lots of jargon and hard to understand statistics. The goal of this class is not to enable you to completely understand everything you read in top journals – articles are written for audiences with advanced training and expertise in an area. However, you should be able to do a surface level reading of any article and be able to understand what they were trying to accomplish, what they did, and what they found, and then draw your own conclusions about whether those findings are useful or meaningful. To help you enhance your skill in digesting research, you will be asked to write a “critique” of a set number of articles during the term. You won’t have to do a critique for every article we read (although it might not be a bad format to follow in your reading). General notes: • There are multiple opportunities to hand in a critique throughout the term. You need to do 8. If you wish to do 9 then your lowest critique grade can be replaced. • Getting these done earlier in the terms is good advice. You could actually read ahead and hand some in well before the due date! • Articles often contain multiple studies. When writing a critique for a paper that has two or more studies, try to summarize across the two. When needed, make sure you reference “study 1” versus “study 2.” • Do not get hung up on the statistics in the results section. Many of these papers will describe statistical analyses that you would not learn until you are an advanced graduate student. Instead, get the key takeaways from the results sections – were hypotheses supported? Follow the format below for writing up your critiques (including numbering and labelling points as indicated). Grading rubric is indicated for each section. 1. Article reference (1 point) Provide the full citation for the article, APA style. Example: Ryan, A.M. (2018). The changing nature of work. Journal of the Workplace, 13(3-4), 147-148. 2. Primary research question(s) and hypotheses (2 points) What are the key hypotheses the researcher sought to test? Or the key questions? Reword the hypotheses or questions rather than just copying; if there are many hypotheses (e.g., more than 5), please summarize the key ones rather than listing all. 3. Rationale for the research (4 points) The introduction to the article gives a rationale as to why this is an important problem to research. The introduction will also give you the logic/theory/arguments for each hypothesis. In your opinion, is there a sufficient rationale for doing this research? That is, do you think this is an important problem to address? Why or why not? In your opinion, did the authors provide good logical and persuasive arguments for their hypotheses and expectations? Was their reasoning clear and easy to follow or complex or non-intuitive? Why did you come to that conclusion? 4. Construct validity (4 points) In any study researchers will discuss the concepts they are linking in their hypotheses (e.g., job satisfaction, pay level, trust) and then will come up with a way to either measure or manipulate those variables in the study (studies) they conduct. Sometimes they do a good job of operationalizing the concept and sometimes they are forced to pick one way of manipulating or one way of measuring that is not ideal. How well did the operationalizations (the manipulation or measure) of key variables in this study map onto the concepts as discussed in the introduction? Recognizing that in any one research study, a choice has to be made about how to study something, what are your thoughts on the choices made here? What other suggestions would you have for how to operationalize variables? If there are a lot of variables in the studies, please focus on the primary ones from the key hypotheses – cover 2-5 at most 5. Internal validity (4 points) One of the biggest questions we have about research studies is whether the design used supports the kinds of inferences made based on findings. Depending on the study design, some of the questions below might be relevant for you to discuss: Was the goal of the study to look at causality – that one variable caused another? Or was it to just show an association? Was it just to describe something and say nothing about relationships? Are the conclusions drawn appropriate? Are there alternative explanations for findings that the authors don’t discuss or don’t give enough credit to? For example, was there a confound with a manipulation? Is there another variable that wasn’t measured that could explain things? 6. External validity (4 points) In any one study, researchers make a choice of who, when and where they conduct the study. Their goal then is to have a finding that generalizes beyond the people and the context of that specific study. Were the participants in this study representative of the group the authors wish to generalize to? Do they acknowledge any limits on generalizing? Are there any concerns about how participants for the study were recruited or selected? Are there any concerns about attrition (dropping out) of the study or missing data? Is the context of the study an unusual one that might put limits on findings? Any concerns about the generalizability of the findings across cultures? 7. Practical implications (3 points) What would a manager or employee in an organization do differently based on these results? What if anything would you do differently in your working life? Are the findings practically useful in some way? 8. Theory and research implications (3 points) After reading the study, what is one idea for future research that you have on this topic that was not suggested by the authors? Provide a specific research question or hypothesis. Grading of critique will be based on above points plus 5 points on your quality of writing and thoroughness. Total 30 points.
