Beyond development and dissemination: The Diagnostic Questions Clusters (DQCs) project as a model for transforming introductory level courses

Background/Question/Methods

Although most faculty would agree that students learn better in science courses emphasizing student-centered, interactive instruction and conceptual understanding, transmission style teaching focusing on “facts” remains the norm, especially at the introductory level. This disparity may exist because we lack effective models for change for college science teaching. In the DQC project faculty use research-based Diagnostic Question Clusters (DQCs) concerning transformations/pathways of energy and matter to assess whether students apply principle-based reasoning, e.g., use Laws of Conservation of Matter and Energy, to explain ecological and biological processes. DQCs are given pre-post instruction to assess students’ comprehension. Faculty learn to use these questions and associated active teaching strategies plus interpret students’ extended responses in a robust faculty development program. We examined pre-post gains in introductory students’ understanding of concepts related to energy and matter using Effect Size (ES) for 19 faculty from two groups participating in this voluntary research. This index describes the magnitude of the difference (if a pre/post improvement is large or small) and is independent of sample size. Calculated using Cohen’s d equation, ES is the mean post minus mean pre students' proficiency value/pooled standard deviation for DQC data from one course. We applied Item Response Theory (IRT) based methods to estimate students’ proficiency.

Results/Conclusions

Twelve faculty from the two groups saw significant pre-post DQC gains in ES; in contrast, for seven faculty pre-post ES changes were not significant. Faculty whose students showed significant gains in conceptual understanding actively worked together to disseminate their efforts in posters, talks, and papers; we propose that they see themselves as change agents for introductory biology and thus participants in the reform process. A common factor for teachers whose students did not significantly improve was that the energy/matter DQCs were less relevant to their course. We suggest that funding agencies have put too much emphasis on development and dissemination of instructional materials and too little on research on models for the effective spread of already available research-based instructional strategies. Critical aspects of the DQC faculty development model include research based questions that reliably assess student comprehension pre/post instruction, formative use of these questions by faculty, active teaching targeting diagnosed problems in understanding, long term work (over several years) with faculty, and very active communities of practice leading to faculty members’ dissemination of their own experiences and findings.