Challenging aspects of critical thinking : A mixed-methods study of students’ test results, students’ reasoning, and teaching strategies

Abstract

There is a growing focus on critical thinking throughout the education system. Overall, the efforts that have been made to improve students’ critical thinking have not yielded the desired results. This could indicate a need for more research concerning which specific aspects of critical thinking that are challenging for students and what barriers students face in this regard. Furthermore, this could also indicate a need for more research on effective strategies for teaching critical thinking, and, more specifically, the details that make these strategies effective in some cases and not in others.

The aim of this dissertation is to provide pragmatic (i.e., useful) knowledge of students’ struggles with critical thinking problems. This could hopefully lead to further integration of the insights from critical thinking research into teaching practice. Furthermore, the aim of this dissertation is to investigate whether supposedly effective yet general strategies for teaching critical thinking are effective in the context of lower secondary classrooms.

In this research project, we have used a well-known test of critical thinking skills, together with a modified version of this test, to identify students’ challenges concerning their use of critical thinking skills. The modified version of the test includes written justifications to selected multiple-choice items from the test. This could give insights into students’ reasoning when facing these items and indicate certain skills and knowledge that should be of particular focus in instruction. Moreover, we measured the effect of the ARGUMENT project—which included supposedly effective strategies for teaching critical thinking—on students’ performance on the test.

In Phase 1 of the study in Article I, we quantitatively explored lower secondary students’ test results on the unmodified critical thinking test, and tentatively identified challenging items. The items were then qualitatively analyzed and divided into five categories based on their proposed solution strategies (i.e., from the test manual). According to the analyses, three categories of items were particularly challenging for these students. First, the items that required students to discern between observations and inferences seemed to be the most challenging. Second, many students struggled with the items requiring that test takers recognize a conflict of interest and take that into account when evaluating the credibility of sources and statements. Third, some students also struggled with the items requiring that test takers recognize that certain methods of observation are better than others. In Phase 2 of the study, we administered the modified test which asked for written justifications to selected multiple-choice items from these tentatively identified difficult categories of items. We have not seen any previously published studies that have used this method. The results from the modified test support the hypothesis that these items are challenging, and, importantly, that the challenges relate to the required critical thinking skills.

In Article II, we conducted a thematic analysis of the written justifications from the dataset we collected in Phase 2 of Article I. We identified six overarching general themes of reasoning encompassing 21 sub-themes. In sum, more than a quarter of the responses expressed strong inductive logic yet contained incorrect reasons because the premises used were either based on alternative evidence or were made up by students who introduced elements not originally included in the context of the items. Only a few responses did not express strong inductive logic. Most of these were responses from students who seemed to believe that an inference is just as, or more, believable than an observation. We discuss potential barriers to critical thinking that students seemed to face when working with the test items, and how these barriers relate to skills, dispositions, knowledge, and motivation.

In Article III, we conducted a quasi-experimental study comparing the gain in critical thinking test scores of the lower-secondary students in the ARGUMENT project with a control group. Teachers in the schools within the ARGUMENT project worked with researchers to develop and implement inquiry-based teaching methods with a focus on scientific argumentation and critical thinking in the context of socioscientific issues. The project aspired to implement general strategies for teaching critical thinking which have been found effective in previous research. Students in the three treatment schools and the three non-treatment schools improved their critical thinking scores significantly from pretest to posttest. However, we did not find a difference in the gain in scores between the two groups. The article discusses potential reasons for this, including the theoretical rationale used in the research and the degree to which the implementation of the project aligns with this rationale. Importantly, we also suggest that the strategies for teaching critical thinking found in the literature could be too general. The article proposes potential avenues that should be explored in future research. In particular, the discussion of the results indicates that there is a need for more detailed insights into the characteristics of the types of authentic inquiry, dialogue, explication of critical thinking principles, and teacher training that are effective in improving critical thinking.

The findings and tentative conclusions from the first two articles could contribute to the literature on critical thinking instruction by providing preliminary insights into which aspects of critical thinking that might be particularly difficult for secondary students, as well as how these students reason when faced with critical thinking problems representing these aspects. With time, especially if future research is able to further validate these conclusions, these insights could indicate which aspects of critical thinking that should be the focus of instruction. Moreover, the insights from Article III might aid instruction and design of other projects with similarities to the ARGUMENT project. Further research on how to explicate critical thinking principles, for example through dialogue and scaffolds for inquiry into authentic issues, could then make use of the insights (from the first two articles) into which aspects of critical thinking that should be the focus of such explication.