A key element of Cambridge Assessment’s test validation procedures are the regular 'health checks' we give our Cambridge English Qualifications to keep them fit for purpose in today’s changing society. Periodic reviews are built into the ongoing quality assurance process to ensure that the skills and knowledge measured in the exams are still appropriate and relevant to the contexts in which they are used.
In Revising the A2 Key and B1 Preliminary Reading exam, Marianne Pickles, Tony Clark and Mark Elliott from our Assessment, Research and Thought Leadership, and Validation and Data Services departments, report on how one of these reviews led to exam revision. The changes were prompted by two main considerations. First, the introduction of upwards certification, where candidates who perform highly in a Cambridge English exam can be granted a certificate a CEFR level above, meant that in each exam it was necessary to include descriptors of such a level. Second, the move towards greater standardisation of exam structure in order to create a 'family resemblance', which would help both test takers and teachers progress along the Cambridge English Qualifications learning ladder.
The review of the exams was conducted using a combination of expert judgment and statistical analysis of how tasks performed. For example, the B1 Preliminary Writing sentence transformation task, when analysed statistically, was found to actually rely on lower-level reading processes, so the decision was made to convert it to an open cloze reading task with extended text. This had the added advantage of bringing consistency of structure to all five levels, as A2 Key, B2 First, C1 Advanced and C2 Proficiency all feature such a task.
Other changes were the inclusion of a longer text for expeditious as well as careful reading in A2 Key, and, crucially, statistical analysis to investigate potential bias had shown that Reading/Writing part 6 – a vocabulary/spelling test – behaved differently for test takers with different first languages, so this task was retired altogether.
You can read more about the thinking and processes behind the revisions to A2 Key and B1 Preliminary Reading tests in the full Research Notes article.