Measuring the Value of Certification: Part 2 — An Example

The primary objective of a value of certification (VoC) study is to understand and quantify the value that is placed on certification by various stakeholder audiences. In part one of this two-part series, I described key steps and considerations for conducting a VoC study. In this article, I will share highlights from a VoC study conducted by the National Board for Certification in Occupational Therapy, Inc. (NBCOT^®) that followed the steps outlined in part one.

NBCOT’s Value of Certification Study

NBCOT is the certifying body for occupational therapy professionals in the U.S. For the purpose of this article, the results of the VoC study will be combined for its two certification programs — the Occupational Therapist Registered (OTR^®) and the Certified Occupational Therapy Assistant (COTA^®). The results followed the same pattern and did not differ significantly by program.

NBCOT used survey methodology to assess the perceived VoC of three stakeholder groups: current certificants, certificants who had reinstated their certification after letting it lapse and employers. Shared below is a subset of the key findings from the study.

When interpreting the results of the study, it is important to note that initial certification by NBCOT is a requirement for licensure in all 50 states, Washington, D.C., Guam and Puerto Rico; however, certification renewal is not required for licensure renewal, except in South Carolina. Therefore, a primary focus of our study was certification renewal since it is largely discretionary.

Certificant Survey Highlights

NBCOT invited 188,631 certificants via email to participate in its online survey. A total of 21,641 individuals provided at least a partial response to the survey, representing a response rate of 11.47%. The sample was representative of the demographic and professional characteristics of the certificant populations, and although we examined results by demographic and professional group, a discussion of these analyses is beyond the scope of this article.

We asked certificants to rate the extent to which they agreed with a series of statements about the value of NBCOT’s national certification programs to the occupational therapy (OT) profession. Overall, respondents held very favorable views about our certification programs (see results below).

Table 1. Perceived Value of NBCOT’s Certification Programs Among Certificants

Note: Percent agreement is based on “strongly agree” and “agree” responses on a five-point scale ranging from “strongly agree” to “strongly disagree.”

Next, participants were asked to rate the extent to which they valued tangible benefits of NBCOT certification. Of all the benefits presented, certificants appreciated the ability to use the OTR or COTA credential the most, followed by having access to the NBCOT Navigator and the free certification renewal units it affords, and having access to the ProQuest research database to find evidence-based research. A majority of respondents also agreed that the time and cost required to renew their certification was reasonable, although the results also suggested that they would like to receive more benefits for the cost.

Certificants were asked a series of questions designed to assess their attitudes toward being certified and expressed the highest level of agreement with the statements in Table 2.

Table 2. Perceptions of Being Certified: Highest Rated Statements

Note: Percent agreement is based on “strongly agree” and “agree” responses on a five-point scale ranging from “strongly agree” to “strongly disagree.”

Certificants expressed the lowest level of agreement with the statements outlined in Table 3. The lower levels of agreement with these statements were not surprising given that OT is a licensed profession in which initial certification is required to gain entry into the profession.

Table 3. Perceptions of Being Certified: Lowest Rated Statements

Note: Percent agreement is based on “strongly agree” and “agree” responses on a five-point scale ranging from “strongly agree” to “strongly disagree.”

The results of the certificant survey were informative, and have provided valuable insights for NBCOT’s quality management, marketing and strategic planning initiatives. The findings sparked conversations around questions such as: How can NBCOT increase the VoC? Is NBCOT offering the right benefits to certificants? How can NBCOT leverage the value that certificants perceive? NBCOT staff also used the results for various marketing and outreach efforts, including student presentations, certificant newsletter articles, mailers to renewal candidates and social media posts.

Reinstated Certificant Survey Highlights

NBCOT was also interested in hearing the perspective of certificants who had reinstated their certification after letting it lapse (i.e., not renewing during their renewal year). A survey was emailed to 2,375 certificants who had recently reinstated their certification, and 419 individuals (17.6%) provided at least a partial response. The sample included a range of demographic and professional characteristics and was generally representative of the certificant population.

In an effort to gain broader insight into perceptions of value, the reinstated certificants were asked about the reason for letting their certification lapse (Table 4) and the reason for reinstating it (Table 5). Those who selected “other” as the reason for letting their certification lapse primarily cited life events or crises (e.g., having a child, illness or death in the family, divorce, the pandemic). In the “other” category of reasons for reinstatement, recurring themes included participants who were reinstating after an accidental lapse, a desire for higher status, improved job opportunities and the fact that being certified is required by their state, employer or educational program.

Table 4. Reasons for Letting Certification Lapse

Note: Participants selected all responses that applied.

Table 5. Reasons for Reinstating Certification

Note: Participants selected all responses that applied.

NBCOT asked the reinstated certificants if they intended to renew their certification at the next opportunity, and 92% said yes, 7.5% said they were unsure and 0.5% said no. When asked to explain their decision, many responses mentioned the perceived value of certification, including factors such as employers desiring or requiring certification, feeling proud to be certified and enjoying the benefits of certification — such as NBCOT’s free competency assessment tools that offer credits toward certification and licensure renewal.

The recently reinstated certificants were also asked the same questions about the value of certification as the full certificant sample, and the pattern of results was very similar. This was somewhat unexpected, since those who had let their certification lapse might be expected to perceive less value in being certified. However, these reinstated certificants felt generally positive about the personal and reputational benefits of being certified, as well as the tangible benefits of renewing their certification. Table 6 summarizes this sample’s top five responses to questions about the perceived benefits and costs of being certified.

Table 6. Perceived Benefits and Costs of Certification and Certification Renewal

Surveying recently reinstated certificants allowed NBCOT to gain a deeper understanding of this important stakeholder group. The results of the survey revealed many areas of opportunity, including clarifying misunderstandings (e.g., explain the difference between licensure and certification), increasing the tangible benefits offered and finding new ways to communicate value. The findings also underscored the importance of employers in encouraging lapsed certificants to reinstate. In the final phase of their VoC study, NBCOT sought to learn more about how to communicate value to the employer community.

Employer Survey Highlights

NBCOT developed a separate survey designed to assess employers’ perceptions of the value of the organization’s certification. For the purpose of comparison, the employer survey overlapped considerably with the certificant survey, although it was tailored towards the employer audience. The online survey invitation was emailed to a list of 2,968 employers compiled from a campaign to obtain employer information directly from certificants. A total of 559 employers provided at least a partial response to the survey, representing a response rate of 18.83%.

OTs work in many different practice settings, including hospitals, schools, skilled nursing facilities and rehab clinics. As such, employers from a range of demographic and professional backgrounds were represented in the sample, with other professional backgrounds primarily including physical therapy, speech-language pathology, education, psychology and nursing. Since OTs are often part of an interdisciplinary team that may be led by someone with a different professional background, this posed a challenge for our VoC study, as employers from other fields would be aware of license requirements, but completely unaware of the full value of OT certification.

Employers rated the extent to which they agreed with the same six value statements shown to certificants regarding the benefits of having a national certification program for the OT profession. Agreement with these value statements ranged from 75%-85%, demonstrating that employers place a high level of value on NBCOT’s certification programs.

Table 7. Perceived Value of NBCOT’s Certification Programs Among Employers

Note: Percent agreement is based on “strongly agree” and “agree” responses on a five-point scale ranging from “strongly agree” to “strongly disagree.”

When asked to rate the overall importance of certification for OT professionals on a scale from 1 to 10, the average rating was 7.5. Employers who placed higher value on the NBCOT certifications were more likely to require certification renewal for their employees and more likely to consider certification when making hiring and promotion decisions.

NBCOT also inferred value through questions about organizational support for certification. For example, most employers (73%) indicated that their organization did not cover the cost of employees’ certification renewal fees, although 18% said that they covered the full cost and 5% reported covering part of the cost (the remaining 4% were not sure). A majority of respondents (56%) indicated that OT employees are required to maintain their certification as a condition of employment in their organization.

Employers were also asked two open-ended questions that yielded interesting data. The first question was: From your perspective as someone who employs or supervises OT professionals, what is the biggest benefit of having employees who are NBCOT certified? Responses included:

• More engagement in and commitment to the profession
• Greater professionalism
• More up-to-date resources on evidence-based practice and on the profession as a whole
• Access to online tools
• Name recognition, marketing
• Standard national level of preparation

The second question asked respondents to share what they thought would make NBCOT certification more beneficial to them as an employer. Among those who responded, several key themes emerged, including:

• Make certification renewal mandatory or required for licensure
• Provide more education to employers about why certification is important
• Educate the public about benefits of certification
• Improve recognition for certificants and the employers who employ them

Although building and maintaining a list of accurate contact information for employers is challenging, the benefits of conducting VoC research with employers are numerous. Not only did the survey provide valuable information about an important stakeholder group, it also served as an opportunity to create awareness of NBCOT certification among employers who may not be familiar with it (i.e., those who are not OTs). NBCOT plans to continue to build its employer database and conduct employer focus groups to further understand employer perceptions of the VoC.

Insights and Lessons Learned from the Study

Demonstrating the VoC is especially challenging for organizations like NBCOT because the fact that initial certification is required for licensure and certification renewal rates are high makes finding a comparison group of uncertified individuals very difficult. Nonetheless, through careful planning, NBCOT succeeded in learning about the perceived VoC among its certificants and those who employ them. The results of the surveys were very encouraging and showed that NBCOT’s key stakeholders perceive a considerable amount of value in its certification programs and hold the OTR and COTA credentials in high regard. The results of the survey also highlighted several growth opportunities, both in terms of communicating the VoC and improving the value proposition for employers. Educating employers regarding the incremental value offered by certification renewal beyond licensure renewal is one such area of opportunity for NBCOT.