COS 27-7
Exploring students' mental models of matter and energy transformation through lexical analysis of written assessments
Constructed response assessments, such as writing, allow students to express their knowledge in their own words. Students are able to present several ideas or concepts, including responses that contain both correct and incorrect or naïve ideas. Thus, constructed response assessments may allow better insight into students’ mental models of ecological concepts. In large enrollment courses, written assessments are seldom used as they are more time consuming to administer and grade compared to multiple-choice assessments. The objective of this study was to examine the use of a lexical analysis approach to analyze and provide feedback on student writing on the transformation and transfer of matter and energy in living systems. Students in a large enrollment introductory biology course were prompted to write explanations for 1) the limited number of levels in a food web and 2) flow of matter and energy in an ecosystem. I collected and analyzed responses from 170 students. I coupled lexical analysis and statistical analysis to uncover ideas revealed in student writing. First, I used lexical analysis to extract key concepts from student writing. Then, I used these extracted concepts as variables in a cluster analysis to determine the groups of students’ responses.
Results/Conclusions
Lexical analysis identified several ideas in student responses including discussions of energy loss, laws of thermodynamics, trophic levels (e.g. primary consumers), and physiological processes (e.g. digestion). Cluster analysis of students’ explanation of the limited number of levels in a food web produced 4 clusters of responses based on the association of ideas within the responses. Although most students (80%) identified the loss of energy at each trophic level in their explanation of the limited levels in food webs, the depths of student explanations varied. For example, one cluster contained student responses that mentioned inefficient transfer of energy but gave no further explanation, while another cluster used the second law of thermodynamics to explain the inefficient transfer. Analysis of the students’ explanation of flow of matter and energy in an ecosystem revealed that some students had mixed models that included correct ideas, such as the loss of heat from the system, as well as incorrect ideas, such as the conversion of energy to matter (15%). These results demonstrate how lexical analysis can be used to provide instructors with feedback on their students’ thinking about ecological concepts and the various mental models that students are forming while learning the material.