Have you ever wondered what happens behind the scenes in the creation of a Cambridge English Qualification? Test development is a lengthy process from inception to the moment it goes ‘live’.
First of all, experienced item writers produce test items according to established test specifications. Next, the items are vetted by external chairs to ensure they don’t have any bias towards any particular group or gender. Approved test items are then piloted to make sure they work for different nationalities. Finally, statistical analyses are run to check that the items discriminate between higher and lower performing test takers.
Even when the test is running successfully, we don’t want to be complacent. One of the key points in the life cycle of a test is a periodic ‘health check’ to ensure it continues to cover communicative language skills for a fast-evolving society. And of course, any changes made must safeguard the validity of the test and the reliability of the scores.
In Revising the A2 Key and B1 Preliminary Speaking exam, Kathryn Davies and Nick Glasson from our Assessment department report on how a periodic health check prompted changes in these exams’ format and task order. A review of feedback from over 500 stakeholders and testing experts highlighted several potential areas for revision.
In the case of A2 Key, there was concern that the information exchange section did not always encourage test takers to take turns, or allow stronger candidates to demonstrate their best performance. The task was subsequently revised to align more closely to the CEFR level descriptors, and allow test takers to demonstrate more fully their ability to manage simple exchanges of information on familiar topics. These changes elicited test takers to more authentic, personalised communication at a basic level, and the format change created a family resemblance between the A2 speaking and successive Cambridge English Qualification test formats.
With B1 Preliminary, the focus for revision was how the linking from the long turn to the extended discussion might limit performance of stronger students. Another point raised was the lack of examiner support in the extended discussion, as B1 test takers may struggle to manage the interaction. Finally, several experienced examiners commented that this discussion would follow more logically after a collaborative task. As with A2 Key, a final objective was to create more of a family resemblance between each Cambridge English Qualification from A2 to C2, to lower test-taker anxiety and to help teachers preparing learners of different levels.
To know more about the exam review process, changes made to the two speaking exams and the rationale behind them, read the full Research Notes article.