We often receive questions about exam scoring — how exams are scored, how passing scores are determined and what various scoring terms mean.
If it seems complicated, that's because it is ... but necessarily so. Our team of psychometricians work to ensure NHA certification exams accurately assess the competencies required for a candidate to perform career-related skills. Achieving an NHA certification shows employers they have the knowledge and skills to do the job, and certification holders often tell us that it boosts their confidence and pride knowing they are #NHAcertified.
Psychometrics: the branch of psychology concerned with the quantification and measurement of mental attributes, behavior, performance, and the like, as well as with the design, analyses, and improvement of the tests, questionnaires, and other instruments used in such measurement. - APA Dictionary of Psychology
For a deep dive into NHA's scoring methods and reports, refer to the NHA Candidate's Handbook. If you want a crash course, our psychometricians have broken down a variety of factors that go into NHA's exam scoring for you in this post.
1. Can you explain how NHA determines what a passing score is for certification exams?
NHA determines the passing score (also known as cut score) using a psychometric procedure called standard setting. Psychometricians and researchers work with a diverse group of Subject Matter Experts (SMEs) to determine a point on the score scale to make a pass/fail decision on an exam.
There are various standard setting methods to determine the passing score. NHA uses criterion-referenced standard setting, which means criteria is predetermined before students begin the test.
SMEs who take the examination undergo training to help them judge whether a minimally competent person (MCP) can answer a question on the test correctly. If the SME determines they can, then a value of one is assigned on a rating sheet. If the SME determines that an MCP cannot answer the item correctly, then they give it a value of zero.
The SME ratings on all questions are summed and the average of all SMEs' scores are calculated. The result of this calculation serves as the recommended cut score (passing score) for the exam.
To help the SMEs make a good judgement on the difficulty of the test items or questions, the SMEs review each item's difficulty statistic after the initial cut score recommendation. This statistic is computed by calculating the percentage of people who answered an item correctly.
For example, if an item is answered correctly by 95% of test-takers, then the item is considered very easy. If only 10% of the test-takers answer an item correctly, then the item is considered very difficult. The SME’s professional opinion and the item difficulty index allow the SMEs to make a better judgement on the ability of an MCP to answer an item correctly.
After this concludes, the SMEs rate the items again and alter their ratings based on the item difficulty statistic shared with them. Ratings are averaged once again, and the new total serves as the recommended cut score. SMEs may elect to do this for a third and final time, or set the cut score based on these results.
2. How can the test plan help you prepare for an exam?
Anyone taking an NHA certification exam for the first time should begin by reviewing its test plan.
A test plan or a test blueprint is a public document that testing programs publish to help guide candidates.
A test plan usually identifies:
- Time allotted for taking the exam
- Types of questions on the exam
- Delivery method
- Major content areas (domains)
- Tasks and knowledge statements associated with each domain
- Number and percentage of items on each domain
- Total number of exam questions
On some exams, the number of items on each domain is the same and domains are equally weighted. On others, the number of items on the domains vary and the domains are not equally weighted.
Reviewing the above information and having prior knowledge of what content areas will be on the test helps candidates prepare, reduces anxiety and helps set candidates’ expectations.
Preparing before exam day is key to doing well. Taking practice tests allows candidates to identify their strengths and weaknesses. For example, a candidate who does not do well on a set of questions can review the questions and map them to the specific tasks or knowledge statements on the test plan, as well as identify the content area(s) where they need the most improvement.
3. When a new test is available, it sometimes requires provisional scoring, which delays receiving results. What is provisional scoring? Why is it important?
Provisional scoring, also called pilot scoring, is a task that bolsters the validity of a test. It is usually conducted after all questions, on a new exam, pass all content, editorial, and psychometric reviews and before a cut score study takes place on a test.
Once all reviews are completed, a provisional test form is assembled. Prior to releasing the provisional test form, testing programs notify candidates that they will receive final results after the provisional scoring period is complete and the cut score study is conducted. This often takes up to eight weeks.
Provisional scoring is important because it allows psychometricians to examine performance and analyze the data to provide quantitative feedback to the test development team.
The test development team summarizes this feedback and holds a meeting with a group of SMEs in order to review the items flagged for questionable performance. SMEs then review flagged items to confirm the content of the item, the correct response, and select the final set of scored items. SMEs also review and discuss the candidates’ qualitative feedback on some of the questions on the exam.
4. What else should instructors or students know about scoring?
The scoring method and scores are important to the validity of a certification examination. These are explored in great detail in the NHA Candidate’s Handbook.
It's important to note that all NHA certification exams are scored using the same scoring method. Questions are scored as either correct or incorrect — a correct answer is assigned a score of one and an incorrect answer is assigned a score of zero. A candidate’s total score on an examination is the sum of all scores on scored items on the exam.
We hope this helped you make sense of the complicated process that is test scoring. Since 1989, we have awarded more than 900,000 allied health certifications (that's a lot of scoring!). We're constantly working to ensure our exams are relevant and meaningful, and our talented team of psychometricians are a big part of that effort.
