Skip to main content
Cambridge
  • Products and Services

    Products and Services

    • Products and Services

      Our innovative products and services for learners, authors and customers are based on world-class research and are relevant, exciting and inspiring.

    • Academic Research
    • English Language Learning
    • English Language Assessment
    • Educational resources for schools
    • Bibles
    • Educational Research & Network
    • Assessment Research
    • Cambridge Assessment International Education
    • OCR
    • Cambridge Assessment Admissions Testing
    • Cambridge Assessment Network
    • Cambridge CEM
    • Cambridge Partnership for Education
    • Cambridge Dictionary
    • The Cambridge Mathematics Project
    • CogBooks
    • Bookshop
  • About Us

    About Us

    • About Us

      We unlock the potential of millions of people worldwide. Our assessments, publications and research spread knowledge, spark enquiry and aid understanding around the world.

    • What we do
    • Our story
    • People and planet
    • Governance
    • News and insights
    • Legal
    • Accessibility
    • Rights and permissions
    • Annual Report
    • Contact us
  • Careers

    Careers

    • Careers

      No matter who you are, what you do, or where you come from, you’ll feel proud to work here.

    • Careers
    • Jobs
    • Benefits
Change Language

Cambridge English Language Assessment

Main navigation

  • Products and Services

    Products and Services

    • Products and Services

      Our innovative products and services for learners, authors and customers are based on world-class research and are relevant, exciting and inspiring.

    • Academic Research
    • English Language Learning
    • English Language Assessment
    • Educational resources for schools
    • Bibles
    • Educational Research & Network
    • Assessment Research
    • Cambridge Assessment International Education
    • OCR
    • Cambridge Assessment Admissions Testing
    • Cambridge Assessment Network
    • Cambridge CEM
    • Cambridge Partnership for Education
    • Cambridge Dictionary
    • The Cambridge Mathematics Project
    • CogBooks
    • Bookshop
  • About Us

    About Us

    • About Us

      We unlock the potential of millions of people worldwide. Our assessments, publications and research spread knowledge, spark enquiry and aid understanding around the world.

    • What we do
    • Our story
    • People and planet
    • Governance
    • News and insights
    • Legal
    • Accessibility
    • Rights and permissions
    • Annual Report
    • Contact us
  • Careers

    Careers

    • Careers

      No matter who you are, what you do, or where you come from, you’ll feel proud to work here.

    • Careers
    • Jobs
    • Benefits
Change Language
English Language Assessment
  • Home
  • Why choose us?

    Why choose us?

    • Learners

    • Who accepts our exams?
    • Visas and immigration
    • Study abroad
    • Cambridge English journeys
    • Organisations

    • English for higher education institutions
    • Our exams and tests
    • Verify results online
    • Setting your requirements
    • Exam security and quality
    • Employers
    • Research and Consultancy

    • English Research Group
    • English language consultancy
    • Impact monitoring and evaluation
  • Exams and tests

    Exams and tests

    • Cambridge English Qualifications Cambridge English Qualifications
    • Schools
    • General and higher education
    • Business
    • Pre A1 Starters (YLE Starters)
    • A1 Movers (YLE Movers)
    • A2 Flyers (YLE Flyers)
    • A2 Key for Schools (KET)
    • B1 Preliminary for Schools (PET)
    • B2 First for Schools (FCE)
    • A2 Key (KET)
    • B1 Preliminary (PET)
    • B2 First (FCE)
    • C1 Advanced (CAE)
    • C2 Proficiency (CPE)
    • B1 Business Preliminary (BEC Preliminary)
    • B2 Business Vantage (BEC Vantage)
    • C1 Business Higher (BEC Higher)
    • Linguaskill Linguaskill
    • IELTS IELTS
    • OET OET
    • Useful links

    • Test your English
    • Cambridge English Placement Test
    • How to register
    • Find an exam centre
    • What to expect on exam day
    • Study English in the UK
    • See exam results online
    • Cambridge English Scale
    • International language standards explained
    • Information for preparation centres
  • Learning English

    Learning English

    • Free activities
    • Exam preparation
    • Test your English
    • Parents and children
    • Games and social media
    • More resources
  • Teaching English

    Teaching English

    • Teaching qualifications
    • CELTA
    • DELTA
    • TKT (Teaching Knowledge Test)
    • Professional development
    • Cambridge English Teaching Framework
    • Cambridge English Trainer Framework
    • Become a Cambridge English Assessment Specialist
    • Resources for teachers
    • Webinars for teachers
    • Information for preparation centres
  • Blog
  • News
  • Events
  • Help

    Help

    • Lost certificates
    • Enquiries and appeals
    • Special requirements
    • Special considerations
    • Malpractice
    • Complaints
    • Corporate social responsibility
  • Test your English
  • Find an exam centre
  • Who accepts our exams?
English Language Assessment

You are here:

  1. Cambridge English
  2. English Research Group
  3. Validity and validation

Validity and validation

  • Validity and validation
    • Familiarisation

Familiarisation - making it work for us

This short paper discusses how Familiarisation procedures as described by the Manual could work for a large scale assessment body such as Cambridge English.

The Manual perceives the Familiarisation procedure as an ‘indispensable starting point’ before a linking exercise to the CEFR can be carried out effectively. It goes on to say that ‘an account of activities taken and results obtained is an essential component of the validation report’. It advocates that ‘participants in the linking process [should] have detailed knowledge of the CEFR’ (2003:1).

There appears to be two underlying assumptions in the above statement: (a) linking examinations to the CEFR is a one-off, time-bound process; (b) participants involved in the linking process are a limited number of people with specific skills perhaps not general within the wider organization.

Let’s consider the first assumption in light of the Cambridge English test development and review model. The model has an ongoing iterative cycle from perceived need to test design to trialling to administration to post exam review (for full discussion of the model see the Cambridge English test development cycle in Weir and Milanovic 2003). The cycle allows for changes in learning, pedagogy and assessment trends as well in the targeted candidature to be incorporated into an examination. Using the Manual terminology, we can relate the test design phase to Specifications, the trialling phase to Standardisation (training and standard setting), the administration and post exam review to Empirical validation. Therefore, by default the linking process is perceived as an on-going activity rather than as a single activity at a given time.

Several issues are highlighted by the second assumption: who are these participants, how many, how familiar are they with the CEF on a familiarity-unfamiliarity continuum, should we take their assessment experience into account or should we start with an assumed zero baseline of knowledge? The picture gets complicated when the participants involved in ensuring that an examination is related to the CEF span across the organization and beyond. Let us look at some figures before we discuss how familiarisation is dealt with. Within Cambridge English, there are 33 subject officers, seven exam managers, eight research & validation officers, and four research & validation managers who work with a network of stakeholders on test specification, item writing, test construction, performance scale construction and on the application and use of performance testing scales. The network includes 30 chairs and 115 item writers, 15000 oral examiners, 700 team leaders, 65 senior/regional team leaders, and 500 writing examiners who do not necessarily reside in the UK and whose roles and responsibilities are at times interchangeable, e.g., a chair of one exam could be an item writer for another (figures are true at Dec 2007). Experienced members of this large community certainly have a close understanding of Cambridge English exam levels. For example, item writers providing materials for a particular examination are familiar with how to interpret the level, e.g., in terms of text difficulty, linguistic features, genre choice etc. Item writer guidelines and examination handbooks provide detailed information on text selection and item writing at a certain level. Through their work with a certain proficiency level, many will also be familiar with the ALTE Can Dos (which Cambridge English, as an ALTE member, helped to develop and validate) and, of course, have come into contact with the CEFR.

The questions we must answer, then, concern the purposes that familiarisation with the CEFR should serve, and the approaches that we should take.

Current processes and documentation specify and operate exam constructs and levels very effectively. They already make use of many things now incorporated into the CEFR but familiar for many years – the Waystage and Threshold specifications, for example. Further explicit reference to the CEFR is being introduced into processes over time where this serves to complement or clarify. Modification of a standard is one possible purpose. The CEFR may also impact at other points in the test cycle, without explicit reference in operational documentation. For example, current studies of the constructs of Reading, Listening, Writing and Speaking, in which a socio-cognitive model is used to match test tasks to CEFR level descriptors, may motivate changes to tasks as exams are revised or updated.

For Cambridge English, Familiarisation with the CEFR should be seen as a part of consolidating and building on existing knowledge. At the same time, the prominence of the CEFR raises the need for a more general awareness-raising, of particular importance for staff just entering the organisation.

Using activities and materials described in Chapter 3 of the Manual, a set of familiarisation procedures are made available to internal staff and to our network and are being implemented where and when applicable as follows:

  1. as part of a one-day workshop with pre and post activities
  2. as part of commissioning letters to stakeholders involved for example in performance testing
  3. as part of the induction and training programme for new staff, e.g. induction worksheet
  4. as part of a longer training course - a total of 18 contact hours (planned activity)
  5. as a seminar with pre- and post-activities, for example during annual meetings with Chairs (planned activity)
  6. as part of Cambridge English seminar series (planned activity)

These activities will be preceded by a short exercise to ascertain level of familiarity with the CEFR. 

References

  • Council of Europe (2003) Relating language examinations to the CEFR. Manual; Preliminary Pilot Version.
  • Weir, C J and Milanovic, M (Eds) (2003) Continuity and Innovation: The History of the CPE 1913-2002, Studies in Language Testing 15, Cambridge: Cambridge University Press/UCLES.

On this site

  • Home
  • Why Cambridge English?
  • Exams and tests
  • Learning English
  • Teaching English
  • News
  • Events
  • Help
  • About us
  • Who accepts our exams?
  • Agents
  • Consultancy
  • Research
  • Contact us
  • ALTE logo
  • English Profile logo
  • Cambridge University Press & Assessment
  • BSI Logo

View Related Sites

  • Cambridge English main site
  • Cambridge English helpdesk
  • Cambridge English Schools
  • Cambridge English Online
  • Cambridge English for Centres
  • Preparation Centres Online
  • Results Verification Service
  • Occupational English Test
  • ALTE
  • English Profile
  • Cambridge University Press & Assessment
  • Cambridge Assessment Admissions Testing
  • Michigan Language Assessment
  • Cambridge Assessment Japan Foundation


© Cambridge University Press & Assessment

  • Terms & conditions
  • Data protection
  • Accessibility statement
  • Statement on modern slavery
  • Safeguarding policy
  • Sitemap
Back to top
We use cookies. By continuing to use this website you are giving your consent for us to set cookies Dismiss