Conditions and Consequences of Classroom Assessment

Project description

Formative assessment focuses on the central idea of eliciting relevant information about the students’ levels of understanding and making use of that information by adapting teaching and learning processes. For example, it is thus possible to provide informative feedback on how to overcome the discrepancy between a learning goal and the current level of understanding. Formative assessment consists of two central elements: the precise diagnosis of students’ understanding and the utilisation of that information by provision of supportive feedback.

The first funding period (2007-2009) was concerned with the development and scaling of tasks and problems that were meant to be used in the subsequent experiment and in the intervention study (1). The aim of the second funding period (2009-2011) was to study the impact of “feedback” as a central element of formative assessment. An experiment was conducted to investigate how various types of written feedback affect students’ cognitive and motivational processes (2). In order to replicate the findings of the scaling study and the experiment in an ecologically valid setting, an intervention study was conducted. In this field-study also conducted in the second funding period, teachers were trained to diagnose their students’ understanding and to provide supportive feedback (3). In the fourth and last study, the transfer study, teacher trainings from the intervention study were enhanced and intensified to investigate the impact of formative assessment trainings on teachers’ general pedagogical knowledge and pedagogical content knowledge (4).

(1) the scaling study

In the first funding period, tasks and problems were developed in order to enable a detailed assessment of “technical“ and “modelling“ competencies. The tasks dealt with the ninth grade teaching units “Pythagorean theorem” and “linear equation systems”. The scaling study was aimed at probing and scaling the constructed test items to enable a detailed and curriculum-aligned diagnosis of student understanding. Additionally, resulting item parameters could be used in the subsequent experiment and the intervention study. In the first funding period, one of the central elements of formative assessment was focused, i.e. “diagnosis”. In addition, data of the scaling study was used to answer discrete psychometrical research questions. It could be shown that content domains, cognitive domains and content specific cognitive domains are empirically separable. Finally, student and teacher questionnaires were used to study teachers’ general assessment practices in math instruction.

(2) the experiment

The aim of the second funding period was to investigate the cognitive and motivational effects of the second element of formative assessment, i.e.  “feedback”. Based on the research literature, multiple feedback characteristics were identified that are assumed to support cognitive and motivational processes, and so called process-oriented feedback was developed. In an experiment, written process-oriented feedback was compared to a grade-oriented type of feedback and to a type of feedback based on competence levels concerning its effect on achievement and interest development. The results of the experiment show that under laboratory conditions, process-oriented feedback is particularly supportive. Students perceived process-oriented feedback to be more useful than grade-oriented feedback which in turn resulted in higher levels of achievement and interest development.

(3) the intervention study

Findings from the scaling study and the experiment were used in a quasi-experimental intervention study to evaluate the effects of formative assessment on learning processes in a more ecologically valid setting. Forty-one teachers were randomly assigned to one of the experimental conditions, (1 control group and 2 intervention groups) realised through teacher training. In a first training, all teachers were provided with subject-specific content concerning the predesigned teaching unit “Theorem of Pythagoras” consisting of 13 lessons. Teachers in the two intervention groups received an additional training on how to implement a formative assessment tool at three time points. Teachers in the second intervention group participated in a third training on oral feedback. The formative assessment tool used for diagnosis and feedback was based on the findings from the previous experiment. The so called “diagnostic task” consisted of two parts: the students’ solution to the mathematical problem(s) and feedback fields, to indicate strengths, weaknesses and strategic hints. First analyses show that students in the intervention groups perceived the process-oriented feedback to be more useful which led to improved self-efficacy and resulted in higher levels of achievement and interest development.

(4) the transfer study

The aim of the transfer study is to evaluate whether teacher’ assessment competencies can be fostered through teacher development programs on formative assessment. Therefore, teacher trainings from the intervention study were enhanced and intensified. Sixty-seven teachers participated in this development programme. First results show that the training improves pedagogical content knowledge about formative assessment. Further analyses will evaluate the effects on general pedagogical knowledge.

Selected Publications

Besser, M., Leiss, D. & Klieme, E. (in press). Wirkung von Lehrerfortbildungen auf Expertise von Lehrkräften zu formativem Assessment im kompetenzorientierten Mathematikunterricht. The effects of teacher trainings on teachers' expertise concerning formative assessment in competence-oriented mathematics. Zeitschrift für Entwicklungspsychologie und Pädagogische Psychologie, 2.

Bürgermeister, A. (2014). Leistungsbeurteilung im Mathematikunterricht. Bedingungen und Effekte von Beurteilungspraxis und Beurteilungsgenauigkeit. [Assessment in mathematics education. Conditions and effects of assessment practices and exactness of assessment.] Münster: Waxmann.

Bürgermeister, A., Klieme, E., Rakoczy, K., Harks, B. & Blum, W. (2014). Formative Leistungsbeurteilung im Unterricht: Konzepte, Praxisberichte und ein neues Diagnoseinstrument für das Fach Mathematik.[Formative assessment in instructional settings: concepts, practice reports and a new diagnostic instrument for math education] In M. Hasselhorn, W. Schneider & U. Trautwein (Hrsg.), Lernverlaufsdiagnostik (S. 41-60). Göttingen: Hogrefe.

Harks, B., Klieme, E., Hartig, J. & Leiß, D. (2014). Separating cognitive and content domains in mathematical competence. Educational Assessment, 19, 243-266.

Harks, B., Rakoczy, K., Hattie, J., Besser, M. & Klieme, E. (2014). The effects of feedback on achievement, interest, and self-evaluation: the role of feedback’s perceived usefulness. Educational Psychology, 34(4), 269-290.

Klieme, E., Bürgermeister, A., Harks, B., Blum, W., Leiß, D., Rakoczy, K. (2010): Leistungsbeurteilung und Kompetenzmodellierung im Mathematikunterricht. [Assessing achievement and modelling competencies in math instruction] In: Klieme, E., Leutner, D., Kenk, M. (Hrsg.): Kompetenzmodellierung. Zwischenbilanz des DFG-Schwerpunktprogramms und Perspektiven des Forschungsansatzes. 56. Beiheft der Zeitschrift für Pädagogik, Heft 2/2010, Beltz-Verlag, Weinheim und Basel.

Rakoczy, K., Harks, B., Klieme, E., Blum, W. & Hochweber, J. (2013). Written feedback in mathematics: Mediated by students’ perception, moderated by goal orientation. Learning and Instruction, 27, 63-73.

Alumni

Birgit Harks

Annika Bürgermeister

Malte Klimczak

Cooperations

Werner Blum (University of Kassel)

Dominik Leiss (Leuphana University Lüneburg)

Michael Besser (Leuphana University Lüneburg)

Funding

German Research Foundation, within the priority program "competence models"

Project co-ordination

Project co-ordination is shared between DIPF Frankfurt and University of Kassel: Prof. Dr. Werner Blum  and Dr. Dominik Leiss (Kassel).

Project Management

• Prof. Dr. Dr. h.c. Eckhard Klieme
• Prof. Dr. Katrin Rakoczy

Project Details