Benchmark Survey in Translating Credentialing Examinations: An Opportunity to Learn, Improve and Explore Language Barriers

By Isabelle Gonthier, PhD, Meazure Learning-Yardstick, Shahid A. Choudhry, PhD, MPH, National Board of Certification and Recertification for Nurse Anesthetists (NBCRNA), and Edwin N. Aroke, PhD, CRNA, University of Alabama at Birmingham School of Nursing

Credentialing organizations are increasingly sensitive to the need to embrace diversity, equity and inclusion, both as it relates to credential holders serving diverse populations and in servicing and addressing the needs of credential holders and individuals seeking credentialing. There are different aspects of diversity, and one of them is language. Language barriers have been identified as the single most significant obstacle to nursing students who speak English as a second language (ESL).¹ These students struggle with reading and comprehension, as they have to read in English, may translate into their native language and retranslate concepts back to English.^1-3 Inadequate reading, verbal, oral and writing-dependent learning can lead to failure in nursing school and on the National Council Licensure Examination for Registered Nurses (NCLEX-RN).^4-6

In 2019, the National Board of Certification and Recertification for Nurse Anesthetists (NBCRNA) charged the Evaluation and Research Advisory Committee (ERAC): ESL Subcommittee to investigate the performance on examinations of individuals who speak English as a second language with limited proficiency, and considerations for translating credentialing examinations.

The ESL Subcommittee partnered with SeaCrest Company on an exploratory study to identify current practices and collect baseline trends affecting the translation of credentialing examinations. Specifically, a benchmarking survey was administered to understand:

1. Policies and procedures related to managing examinees who speak English as a second language.
2. Examination translation operational considerations and resources needed.
3. Language(s) offered for examinations and other materials.
4. Translation models and methods used for the development and implementation of translated examinations.

Key Results and Observations From the Benchmark Survey

The exploratory benchmark electronic survey was designed to gather baseline data on organizational considerations and implications related to translating examinations and other materials for examinees who speak English as a second language. Participant members of various credentialing organizations in the United States and Canada were invited to participate in the study. The survey solicited information about the profession they serve, as well as translation considerations. The research questions for the survey are included in Table 1. 

Table 1: Organization of Research Questions

Participants from 27 credentialing organizations (N=2 licensure and 25 certification organizations) completed the survey. Among the credentialing organizations that participated in the survey, 63% (N=17) of the respondents were from the health care sector. The remaining organizations were from administrative/management, education, events and meetings, fitness and wellness, safety and other industries.

Most participating credentialing organizations offer their programs in the United States (93%, N=25) and Canada (70%, N=19). The number of credentialing programs administered by each organization ranged from one to 20, with an average of five programs per organization. All participating credentialing organizations offer examinations in English. Other languages used in testing include Chinese/Mandarin, Spanish, French, Arabic, Greek, Hindi, Japanese, Korean, Portuguese, Turkish and others.

Of the participating credentialing organizations that completed the survey, 22% (N=6) indicated that they translate their examinations into another language. As part of the survey, organizations that offered examinations in multiple languages were asked about their translation methodologies and psychometric testing. Most credentialing organizations reported that they followed the National Commission for Certifying Agencies (NCCA) standards to translate examinations. These standards, more specifically Standard 16 Essential Element B and Standard 21 Essential Element C, outline the requirements for proper examination design to ensure equivalency of content and comparability of examination results.⁷ Credentialing organizations that follow and adhere to the NCCA standards indicated that most use forward and back-translation* (N=4) and back-translation** (N=1) techniques. Three participating credentialing organizations currently translate their examinations using a translation company and a bilingual subject matter expert. Others use a certified translator (N=2) and a bilingual expert panel (N=1). None of the credentialing organizations responded that they pilot-tested translated examination forms or translated examination questions for their respective programs.

Perceptions, Benefits and Barriers to Translating Examinations

Organizations offering a translated version of their examinations indicated that candidates generally react positively to offering of the translated examinations. Specifically, credentialing organizations described a positive or very positive reaction (N=4), or neural/mixed reaction (N=1), to having a translated version of their examinations.

Regarding the fairness related to the offering of the translated examinations, perceptions were varied. Some credentialing organizations indicated having a few concerns (N=6), where a few expressed concerns about the validity of the examination (N=4), outlined concerns about impact on the test construct (N=4), and identified concerns about other non-translated languages (N=1), where the reminder responded: “I don’t know (N=5), and (N=7) selected “Other” (N=7). The “Other” category included comments regarding concerns with the accuracy of translated content, concerns with addressing dialects and concerns related to examination security.

As part of the survey, credentialing organizations were asked open-ended questions on the benefits and barriers experienced in translating examinations for their programs. Figure 1 summarizes the feedback collected regarding those benefits and barriers. Additionally, the participating credentialing organizations were asked open-ended questions regarding additional potential interventions to address barriers unique to ESL candidates. Responses outlined potential interventions such as:

• The translation of non-examination materials
• The verification of educational credentials via a third party
• The performance of examination review to address the potential use of idiomatic expressions or regional phrasing or terminology
• The completion of an accessibility review using a validation scale

The provision of extended time as a testing accommodation.

Figure 1. Benefits and Barriers to Translations of Credentialing Examinations.

Finally, some of credentialing organizations (N=5) indicated that they provide translated materials, such as candidate handbooks, preparatory materials (i.e., sample items, practice tests), website content (i.e., certification program information), marketing and promotion materials (i.e., brochures) and preparatory courses.

Discussion

Initial findings from this survey were accepted for publication,⁸ and the results suggest that a majority of credentialing organizations do not translate their examinations into another language. These findings are especially significant for a profession like nursing, considering that language barriers have been identified as the single most significant obstacle to nursing students who speak English as a second language.¹

One of the concerns relates to cognitive load. Students may first translate the information into their native language, think of a potential answer in their native language, back-translate the answer into English, and then consider the options provided. This forward and back-translation process may significantly increase the amount of time necessary to complete a timed certification examination. While the provision of extended testing time may help level the testing conditions, it must consider the purposes of the examinations and potential legal implications.   

Additionally, this may increase the amount of stress associated with testing, which may affect performance. These language factors may introduce construct-irrelevant variance into test scores, impacting those scores' validity for their intended use. As a result, some academic institutions have advocated extra testing time for students with English as a second language.⁹ However, providing extended testing time may raise concerns about fairness and create challenges related to eligibility for such a time extension.

If credentialing organizations embark on a multilingual credentialing program, collecting and analyzing demographic data — including race/ethnicity, native language and English language proficiency — to measure and determine the need for translating examinations, along with a cost benefit/effectiveness analysis, is paramount. Additionally, it would be judicious to use a standards-based process that utilizes specialized professionals and best practices to ensure psychometric equivalence and legal defensibility in all examination versions, as outlined in the NCCA standards. The translation may help ensure that candidates are tested on their intended competencies, not their language proficiency.

In the end, the consideration of translating examinations into different languages, based on the need of the certificant population, may assist in fostering a diverse, equitable and inclusive workforce of providers who speak English as a second language and may help reduce disparities.

*The forward and back-translation method begins with a version in a set language, translated into another language, followed by a translation back into the original language to be compared to the original version.

**The back translation method is where a translated version is re-translated back to the original language and compared to the original version.

References

1. Olson MA. English-as-a-second language (ESL) nursing student success: A critical review of the literature. Journal of Cultural Diversity. 2012;19(1).
2. Sanner S, Wilson A. The experiences of students with English as a second language in a baccalaureate nursing program. Nurse Education Today. 2008;28(7):807-813.
3. Sanner S, Wilson AH, Samson LF. The experiences of international nursing students in a baccalaureate nursing program. Journal of Professional Nursing. 2002;18(4):206-213.
4. Stewart B. Enhancing success in bsn nursing education for minority nurses. ABNF journal. 2005;16(1):8.
5. Moore BS, Clark MC. The role of linguistic modification in nursing education. Journal of Nursing Education. 2016;55(6):309-315.
6. Donnell WM. A correlational study of a reading comprehension program and attrition rates of ESL nursing students in Texas. Nursing Education Perspective. 2015;36(1):16-21. doi:10.5480/13-1212
7. National Commission for Credentialing Agencies. National Commission for Certifying Agencies Standards for the Accreditation of Certification Programs. Washington, DC; 2014.
8. Aroke E, Pereira S, Leonard C et al. Translation of Credentialing Examination for Providers with English as a Second Language: An Exploratory Study. American Association of Nurse Anesthetists Journal. Accepted for publication. 2021
9. Miller LA. The effects of extended time on reading comprehension performance for English as a second language college students: Is there a need for accommodations?. School of Psychology, Syracuse University; 2014.