NBME Self-assessment Tests [GUIDE] - USMLE Forums
USMLE Forums Logo
USMLE Forums         Your Reliable USMLE Online Community     Members     Posts
USMLE Articles
Go Back   USMLE Forums > USMLE Step 1 Forum

USMLE Step 1 Forum USMLE Step 1 Discussion Forum: Let's talk about anything related to USMLE Step 1 exam

Thread Tools Search this Thread Display Modes
Old 09-04-2011
USMLE Forums Master
Steps History: Step 1 Only
Posts: 576
Threads: 25
Thanked 522 Times in 263 Posts
Reputation: 532
Default NBME Self-assessment Tests [GUIDE]

NBME Self-assessment Tests

The NBME self assessment test is another of those tools that I feel is responsible for the increasingly high scores in the USMLE. It is a great tool that allows the test-taker to assess his or her readiness for the exam. To a certain extent, if used properly, it is also a useful tool for the examinee to target a certain score, even 99′s and achieve it.

How the NBME self assessment tests were designed.

The questions for the NBME self assessment tests were taken from a pool originally designed for the shelf exams of American Medical Student. As such it covers the most important concepts that AMGs are required to know before passing to the next level. It covers the same topics that will be covered in the actual USMLE examination.

The NBME self assessment tests are very good at predicting the score you will get at the USMLE. In fact I read somewhere (unfortunately, I can’t seem to find the site now though) that at least for Step 1 the 1st exam set has a correlation as high as + 0.89, while the others range from + 0.72 to + 0.85. If you go to most of the forums, you will find that the correlation for most examinees is very good.

Difference between NBME and USMLE.

Although the topics covered in both the NBME and the USMLE is more or less the same, there is a tremendous difference between them.

First, the NBME questions are much, much more straightforward than the USMLE questions. The NBME has precious little tough questions. Therefore a low score in the NBME is more indicative of lack of knowledge of the topics covered in both exams rather than a lack of mastery of those same topics. Although the latter exam sets have tougher questions than the first sets, they still are not as tough as the tougher parts of the USMLE, plus, they actually have lower correlation with USMLE scores than the first exam set.

Second, the NBME questions total less than the actual USMLE exam and therefore will cover less lower-yield stuff than the actual USMLE exam.

Third, you can have up to 4 hours to answer the examination. Part of the toughness of the USMLE is the time constraint imposed on the examinee.

Fourth, you can take the blocks at a time of your own choosing, usually during periods where you are at your peak effectiveness. The actual examination does not give you such luxury.

Fifth, you have to take the USMLE blocks, 7 to 8 at one time with minimal breaks, which can take its toll, believe me. By the fifth or sixth block, your mind tends to stop functioning. You have only 4 blocks in the NBME and you can take it at your leisure.

NBME self assessment tests is a good predictor of USMLE performance.

So despite the obvious difference between the 2, why is the NBME such a good predictor of USMLE performance? The answer lies in the fact that the NBME has taken the time to correlate the results of hundreds of thousands of AMGs with their actual performance in the USMLE. This correlation has taken into account the difference in test conditions and actual content of the examination and that this difference although apparent is not very significant as can be shown by the correlation of + 0.72 to + 0.89 between the various NBME exam sets and the USMLE.

Ineffective use of NBME self assessment tests

So the practice of using the “downloaded” version of the NBME tests effectively wipes out the predictability of the exam set itself. It’s effectiveness lies in the statistical analysis they’ve done for the results and not the results themselves.

Another ineffective practice, is studying the NBME test questions to improve your score. This will be effective only if your original review is so poor that you have not covered the required high yield topics tested in the USMLE properly. Otherwise you are better off studying the questions in USMLE world and Kaplan Q Bank to understand how to answer tough questions and low yield stuff that are more likely to appear in the USMLE.

Another common error is repeating the exact same exam set. The correlation holds only if you take the exam once and only once. Repeating it will raise your score in the NBME which will not be carried over to the USMLE. Therefore, I suggest reserving at least one set to be taken only if you are ready to do the USMLE.

Used properly, the NBME tests can be an effective tool for self-review. So do yourself a favor, use the NBME self assessment tests properly as an assessment tool rather than a study tool. And take the online form rather than downloading them, which is also subject to copyright issues. The NBME self assessment site is here.

How well does NBME predict your USMLE Score?

That has been the number one question asked of me since I wrote about this topic. My answer is still the same. Fairly well. Although correlation is never 100% more like 70 to 80%. However, certain development in the past few years have made the assessment tests less reliable for some people.

One of the reasons for this problem is that a lot of NBME questions, especially the earlier forms of step 1 have been discussed extensively in various forums. What’s more some of these posters did not even bother to warn people that what they are discussing are NBME form questions. Going into the NBME assessment tests knowing some of the answers already can invalidate the predictability of those assessment tests. In fact just knowing some of the questions beforehand can also invalidate the results. The reason is that knowing the questions ahead, means you’ve had time to consider the questions and possible answers before, not just the 1 minute or so that you will actually have in a real exam. That can skew your result.

It has also come to my attention that some NBME questions or variations of those questions have come out in online qbanks. That is bad news, too as it has the same effect of invalidating some of the questions in the NBME forms. Even if you did not get the answer, encountering the questions in the NBME form for the second time rather than the first time means you had more than the 1 minute allowed in the exam to think through the questions and look for the answers.

So what are the remedies for these problems that seem to have cropped up recently. Well first is to actively avoid discussing posted NBME questions in the forums. Of course since some posters do not have the courtesy of even warning people about it, avoid participating in any discussions on questions in forums unless you know for sure that they did not come from an NBME form.

Another solution is to try to use the later forms. In Step 1 this would be form 4. 5 and 6. NBME forms for Step 2CK does not suffer from the same problems as those in Step 1 as they are not discussed as often as those of Step 1.

Doing at least 2 forms instead of just 1 as I recommended before may also help. Just make sure one of the form is not the first 3.

What about the “Downloaded” NBME Self-assessment test version.

I can never understand the popularity of the so-called “downloaded” version of the NBME. IF saving a hundred dollars or so is worth scoring low or failing the USMLE altogether, then it is understandable. But ruining your long term career to save a couple of bucks is not a very intelligent move.

In earlier posts, I have said that the value in the NBME does not lie with the fact that the questions mimic the USMLE. In fact, in general, they are much easier than the USMLE. The main value of the NBME forms is that they are fairly reliable predictors of performance in the USMLE due to the correlation they’ve done with NBME results vs. actual USMLE performance.

However, recently, some posters in my blog have commented that they are able to “predict” their USMLE scores, since there are answer keys and correlation tables available with the downloaded version. So I decided to give it another look.

So what did I find out? Well, first, there is a problem with the answer keys. There were some answers that I completely disagree with. In some cases where I myself am not sure what the right answer is, I tried to verify the possible correct answer by researching them and I still cannot decide what the right answer is even after searching through textbooks and the internet. Therefore, there is a question of how accurate the raw score one is getting for each of those NBME forms are. And that is a major problem. In my case, anywhere from 2 to 8 answers in each form fall into this category and for me, an “unknown” of 4 to 16% in the raw scores completely shoots down any chance of actually knowing the exact raw score you should be getting.

A second major problem is that there is only one correlation table in existence, instead of the 6 correlation table I am expecting. That is one per form. There might be additional correlation tables out there, but the one I got is just 1. Why six tables? Because the forms are of different levels of difficulty, you expect the same person will get different number of questions right in the different forms depending on the level of difficulty. So you need a correlation table for each form to make them comparable. Having only one correlation table means we don’t even know to which form this correlation table belongs. See the problem, now.

A third really big problem is that people who used this downloaded forms has a tendency to keep on using them throughout their prep multiple times. Probably because it’s free. In fact, it was justified that since they did not try to look at the answer, they can redo the same form and expect it to still be accurate in predicting their scores. That is actually wrong. Again, one of the reasons why USMLE is hard is the time limit imposed in answering questions. When you go through the same question multiple times, you’ve had more than the 1 minute per question limit imposed by USMLE to think of the answer and therefore will tend to score higher. That skews the predictability of the NBME.

Therefore, again, do yourself a favor and use only the online NBME assessment tests and use them only when you feel you are ready for the USMLE, to confirm your readiness. The “downloaded” NBME forms may seem free, but it’s hidden costs may be greater than you are willing to pay.

USMLE World Assessment Tests

I’ve had more time to gather information about this relatively new resource. So far based on talking to my students, posters in my blogs, people who have emailed me and reading various forums, my conclusion is that the UW assessment tests is just as good as the NBME assessment tests, so far. Although there have been some observations that UW tend to be overestimate your scores in comparison to NBME, this does not seem to happen in all cases and the score difference is not too big. So all in all, I believe the UW assessment tests have enough track record by this time that we can safely say, they are fairly accurate in predicting USMLE scores. But as in all assessment tests, correlation is never 100%, therefore expect some deviation from predicted scores in the final result.

How to use the NBME Self-assessment tests ; Best time to take the NBME Self-assessment tests

So when is the best time to take the NBME? The best time is when you think that you are ready for the USMLE. When all parameters indicate that you are probably ready for the USMLE examination, you take the NBME to confirm that you are ready for the examination. In other words, use the NBME as a diagnostic or confirmatory test, instead of a screening test. You can use your performance in USMLE World Q Bank or Kaplan Q Bank as a screening test.

Use NBME Self-assessment Tests to assess readiness, not identify weak points

The practice of many to take the NBME Self assessment tests to identify weak points and measure progress in review is not very practical. First, there are only 7 NBME tests (for Step 1) and second, you can take it only once. Taking it more than once invalidates the correlation and therefore its accuracy in predicting actual USMLE performance. Therefore, what do you do if you are still not ready after taking all six tests? You just used up your most valuable tool for predicting your actual performance in the USMLE.

Personally, I used the NBME exam to confirm my readiness to achieve my targeted score of 99. In preparing for USMLE Step 1, when my Kaplan scores were hitting 84 and above, I felt I was ready for the test. I took NBME form 1 only and 740 confirmed my readiness for the USMLE exam since I was aiming for a 99. 600 was enough for a 99, but I would have postponed if I got less than 660. Why? Since I decided I want only a 99, I was giving myself a 10 point leeway (600=240 and 660=250) to make sure I make it to a 99 no ifs, ands, or buts. The same thing holds true for Step 2 CK, I took the exam when my one and only NBME assessment test scored 700, way above what I needed for a 99.

Timed NBME Self-assessment Tests for best results

Another common question is whether to take the NBME examination timed or self-paced. Since you are trying to test your actual readiness for the USMLE examination, it is best to simulate actual exam conditions. In my case, I took self-paced but finished everything in the 1 hour allotted for it. I may look over the questions after 1 hour, but I will not change them so as not to alter the score predictability too much. Make sure everyone in the house knows not to disturb you while doing this. Although you can pause the exam, it is not recommended. Also I do not recommend you do it in a public library. There was one person who took the USMLE despite getting a low score in NBME self assessment test. He attributed it to the fact that he took it in a noisy public library and “upward adjusted” his predicted score. He failed. So, take it properly and trust the score. One warning, though, correlation is not 100% therefore always give yourself some leeway in case things do not go too well. Never, ever go for the USMLE examination if you scored 400 or less. Your chances of failing is too high and even if you passed 75 and 76 are with you for life.

What about the USMLE World Self-assessment tests

Recently, USMLE World started offering their own assessment tests for US$ 30 each, US$ 15 cheaper than NBME’s assessment tests. The question foremost in everyone’s mind is, can I switch to UW assessments tests instead of NBME’s. How wise is a decision like that?

The reason for using assessment tests is to make sure you are ready for the USMLE and to lessen the risk of getting an adverse score. NBME has proven throughout the years, after thousands of test takers, that it is a reliable predictor of your actual score in the USMLE. It is also backed up by correlation studies that have been published in journals. Now, although USMLE World’s assessment test may also be a reliable predictor of your actual USMLE performance, all we have is their word for it. No study has been cited or provided to independently verify such claim. And not enough empirical evidence from test taker’s experience forums allow us to make any judgment on its accuracy at all. Surveying forum after forum reveals variable experience with the tests as of now.

My take is that since you only have one shot at the USMLE, it is better to go with what is proven and tested rather than take a risk. Although, you can retake the USMLE if you fail, that failure is recorded and counted against you. If you pass, the score is with you the rest of your professional life. If you chose to, maybe you can use the UW assessment tests as a screening tool. Just be sure to use the NBME assessment test, whatever form to confirm your readiness before actually sitting for the USMLE.

I found this very helpful over the web, so maybe it can help someone... =)
By AskDoc from AskDoc Usmle Blog.
Reply With Quote Quick reply to this message
The above post was thanked by:
akua (09-04-2011), CleverFOX (04-18-2015), Dr_Laura (09-04-2011), dr_lizard (09-04-2011), drmirzanaveed (09-04-2011), Haneshkumar (09-04-2011), ispy (09-04-2011), nativedoc2 (09-04-2011), steptaker89 (06-15-2014), Tantawy.emad (08-31-2014), tootsie (09-04-2011)

Old 09-04-2011
USMLE Forums Scout
Steps History: 1+CK+CS
Posts: 78
Threads: 11
Thanked 24 Times in 19 Posts
Reputation: 34

@rulz : Thank you for this.. this is an eye-opener.. really, you must think i'm so ignorant because I thought NBME was just another question collection.. apparently it's so much more.. I'll make sure I take your advice on this one.. Thanks again.
Reply With Quote Quick reply to this message
Old 09-04-2011
USMLE Forums Master
Steps History: Step 1 Only
Posts: 576
Threads: 25
Thanked 522 Times in 263 Posts
Reputation: 532

Always a pleasure to help you guys! Yes, a lot of people think of NBME as you, but im glad now you can see this as it actually is. =} a lot of questions are answered on this post, hope it helps others as well.

See tou in the journey.
Reply With Quote Quick reply to this message
Old 04-18-2015
Thrax_usmle's Avatar
USMLE Forums Scout
Steps History: 1+CK+CS
Posts: 91
Threads: 21
Thanked 16 Times in 12 Posts
Reputation: 26
Default One question

Does pausing too many times during the nbme exam or changing answers too often, affect the overall nbme score? Does it mean, you get a lower score?
Reply With Quote Quick reply to this message


Quick Reply

Register Now

In order to be able to post messages on the USMLE Forums forums, you must first register.
Please enter your desired user name, your email address and other required details in the form below.
User Name:
Please enter a password for your user account. Note that passwords are case-sensitive.
Confirm Password:
Email Address
Please enter a valid email address for yourself.
Email Address:
Medical School
Choose "---" if you don't want to tell. AMG for US & Canadian medical schools. IMG for all other medical schools.
USMLE Steps History
What steps finished! Example: 1+CK+CS+3 = Passed Step 1, Step 2 CK, Step 2 CS, and Step 3.

Choose "---" if you don't want to tell.

Favorite USMLE Books
What USMLE books you really think are useful. Leave blank if you don't want to tell.
Where you live. Leave blank if you don't want to tell.


Human Verification

In order to verify that you are a human and not a spam bot, please enter the answer into the following box below based on the instructions contained in the graphic.

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Similar Threads
Thread Thread Starter Forum Replies Last Post
NBME tests! ferdenand USMLE Step 2 CK Forum 2 10-16-2013 10:22 AM
Step 2 CK Self Assessment Tests meanmr.mustard USMLE Step 2 CK Forum 1 05-26-2011 11:08 AM
NBME and UW Self assessment scheduling conspicous USMLE Step 1 Forum 1 02-15-2011 02:12 PM
I want details about NBME tests MedicalDR14 USMLE Step 1 Forum 1 10-13-2010 01:24 PM
About the official NBME Tests & Forms StepTaker General Topics Recommended Threads 0 08-01-2009 01:22 PM

RSS Feed
Find Us on Facebook
vBulletin Security provided by vBSecurity v2.2.2 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.

USMLE® & other trade marks belong to their respective owners, read full disclaimer
USMLE Forums created under Creative Commons 3.0 License. (2009-2014)