Features

I.C.E. Virtual Exchange Preview: Debunking Myths About Linear-On-the-Fly Testing

Among those who operate a certification examination, recent challenges or curiosity could prompt a question like, "If there is a smarter way to operate, will those changes fit our program?"

Presenters Kelly Zhao, a psychometrician from PSI Services, and Jessica Rapey, senior manager of professional assessment for the Commission on Dietetic Registration, plan to answer this question among others during the 2020 I.C.E. Virtual Exchange. During a session entitled, “Debunking Myths About Linear-On-the-Fly Testing: What to Prepare When Moving Into a “’LOFT,’” these presenters will describe how a program's fixed-form operations were assessed and how those results affected linear-on-the-fly (LOFT) implementation. If you share concerns about subject matter expert performance, hitting project milestones, budgetary impact or item exposure, then this session could be a good fit for you.

We spoke with Zhao and Rapey to learn more about their session.

Why is this topic important to those who run credentialing programs?

Adopting an unfamiliar method like LOFT can be a complex, labor-intensive and scary undertaking. These risks can be minimized after an assessment of LOFT readiness and evaluating the implementation roadmap. This session will describe the psychometric assessment of fit between LOFT and a credentialing exam program. A credentialing program manager will describe the experience of switching from fixed-forms to LOFT, including key differences. Session attendees will be invited to share their questions and their own potential myths about LOFT.

What highlights do you hope attendees will take way from your session?

We hope attendees will get the specific information they need, or questions they should ask their psychometrician, about their credentialing program while considering LOFT. We also plan to address areas a credentialing program may have to change to implement LOFT.

What characteristics help LOFT fit well with a program?

  • Higher candidate volume
  • Higher number of fixed forms developed each year
  • Limited number of pretest items approved after each testing cycle
  • Type of items
  • Details on the exam blueprint and other specifications
  • Higher number of items in the bank with known statistical performance
  • The policy on retakes and recertification
  • Comfort level with item/exam exposure or test security concerns
  • Opportunities in a typical budget and project timeline
  • Adaptability of subject matter experts to the training and workload