Looking to create more value in your exam program? A program evaluation can be completed at any stage of the exam development process. For those starting from scratch, it provides an invaluable opportunity to build a solid foundation for your exam program. And if you’re looking to grow your program, this process enables a period of reflection to ensure a test still serves its main purpose: assessing a desired set of skills in a fair, valid, reliable, and secure way.
Why complete a program evaluation?
- Maintain a defensible and reliable exam
- Ensure that exam content reflects the intended purpose of the exam
- Provide investment rationalization to stakeholders
- Support an accreditation application
When you’re considering making changes to your exam, it is vital to understand the nuances of your program, such as the strength of your candidate volume or the depth of your item bank, in order to maximize the benefits of a new design.
Delivering one test form to 50 people a year is very different from administering five exam forms to 14,000 people, which is also very different from delivering an exam with a more complex item-selection design, such as adaptive or LOFT (linear-on-the-fly testing). Completing a program evaluation allows you to understand how the item development and psychometric processes would need to evolve to support a larger exam program. For example, if you deliver a paper-based exam one day a year and want to deliver the exam in a computer-based testing format that leverages on-demand delivery dates, a formal evaluation can identify (1) the psychometric techniques that would enhance the security and comparability of scores across multiple exam forms, and (2) the item development processes that support those psychometric techniques.
Similar to how you would do a home inspection or schedule a doctor’s appointment, it is just as valuable to complete an evaluation as you build your program as it is to routinely re-evaluate your long-standing program. Everything might be running smoothly, but evaluating it gives you the transparency to identify any weak points that might need an adjustment in the future.
So what exactly is involved in a program evaluation?
- Contact your Pearson VUE program manager to arrange a meeting with our leading psychometricians.
- Describe the exam program and your goals.
- Provide written documentation regarding your processes (item inventory and development, non-disclosure agreements, and style guide, for example).
- Our subject matter experts will review and report on potential improvements or suggestions.
Whether you currently run a computer-based exam or you are thinking about transitioning from paper and pencil, there is tangible value in integrating program evaluations into your process to ensure that your greatest asset — your exam — is performing how you expect it to. Put your organization’s best foot forward when certifying the future leaders of the next generation.