PISA-LDW – Assessing Competencies for Learning in the Digital World

As part of the innovative domain in PISA 2025 "Learning in the Digital World" (LDW), the project researches and develops innovative approaches to the diagnostic use of behavioural data from digital learning and assessment environments.

Project Description

How do students learn to solve problems in the digital space? How can learning in digital environments be observed and assessed? How are core competencies of digital learning effectively supported? To answer these and other questions, innovative approaches to the diagnostic use of behavioral data from digital learning and assessment environments have great potential. The central goal of this accompanying research on the innovative domain in PISA 2025 "Learning in the Digital World" (LDW) is the development and empirical validation of such approaches.

As an innovative PISA domain, two areas of competence are to be covered that are of particular importance in digital learning contexts: The first is the competence to model complex systems and solve algorithmic problems with the help of digital tools. In doing so, the students go through a process of self-directed learning and "learn" how to solve the problems. On the other hand, the students' ability to self-regulate this learning process is to be recorded. This means how they monitor, control and adapt this process in (meta-)cognitive, behavioral, motivational and affective terms. For this purpose, behavioral data from the processing of the tasks will also be used - these are obtained by presenting complex problems to the students on the computer in a simulated and interactive learning environment. The students work out solutions step by step - using the learning tasks, learning materials and aids offered in the learning environment. How they accomplish this and how successfully they do so is the subject of the measurement.
For example, in one learning unit, students are asked to create a program by combining commands that are available as blocks in order to control an game piece (see turtle in Fig. 1). The game piece should be made to collect the blue object and then move to a target position. The students can check their programmed solution again and again with the help of a test option, receive supportive feedback and further improve it using learning materials.

Fig. 1. Mockup of a sample task (train turtle) in the style of LDW units.

Mockup of a sample task


The indicators obtained from the behavioral data to capture the competence areas will be examined to determine whether the individual differences they depict can be interpreted in terms of the constructs in question (construct validation). For this purpose, the correlation structure between learning potentials or learning activities on the one hand and LDW problem-solving competence on the other hand will be examined (nomological network). Learning potentials include, among others, reasoning, prior knowledge, self-efficacy and motivation to learn, learning activities the activities in the LDW learning unit as well as preceding learning opportunities in the (extra)curricular setting.

In the context of the research project, (self-regulated) learning processes in digital environments are thus made empirically accessible, the influence of individual preconditions on the learning and problem-solving processes is investigated as well as their effects on learning success. The knowledge gained in the project contributes to a better understanding of learning processes and how they can be diagnosed in digital learning environments.

Project Objectives

The project has the following objectives:

  • Development of indicators to capture the LDW construct domains.
  • Validation of test score interpretations
  • Research on learning processes in interactive learning environments

Funding

Institutional Budget and Third-Party Funding:

  • Logo Bertelsmannstiftung

  •  Logo OECD

  • Logo Centre for International Student Assessment

Cooperations

This project is being carried out in cooperation with...

Project Management

Prof. Dr. Frank Goldhammer

Project Team

Project Details

Status:
Current project
Area of Focus Education in the Digital World
Department: Teacher and Teaching Quality
Unit: Technology-Based Assessment
Education Sector: Primary and Secondary Education
Duration:
09/2021 – 08/2024
Funding:
External funding
Contact: Dr. Leonard Tetzlaff, Academic Staff

TBA