The Good, the Bad and the Ugly. (A thought on a good USMLE-type question)

[DocSikorski]

Hello,

I am getting ready for step 1 examination, maybe way too very long, its been a little bit over a year already. During this time I was able to get many question banks (qbanks) that claim to get you exposed to the exam type questions.
Usually there would be about 2000+ questions in every qbank.
We get some pure "regurgitational questions", where you face a direct question and supposed to give a direct straight-recall answer. Other questions are more conceptual, multi-level questions. These would give you some description of a patient, his history where you are likely to diagnose the patient first, than come up with the most common treatment of choice and maybe even think of the side effects to come to the right answer.
All of this have been already described and chewed multiple times in many currently popular books such as First Aid and Kaplan Lecture notes though.

When I do question banks, most of the time i can see how many people got the same question right, so I was able to grade all the questions in 3 types - easy, conceptual and weird.

Easy questions are those that most of the students get right - 80-90%

Conceptual tend to be answered by 40-60% of people.

Weird questions are those that are answered by 10-20% of students.

We all heard, that the real exam is a "fair game" and that all the questions are really doable. What does that mean?
While listening to dr.Goljan lectures I have heard him saying that the good question is not the question that many students get wrong. When most of the class got the question wrong - than this is just a bad question or you are a bad teacher.

What is our teacher? most of the students will say FA or KLN. Compared to a regular medschool class, we (IMG's or those who are getting ready for Step 1) can chose our teacher on our own. So if most of us (80-90%) who study for Step 1 prefer the same teacher, it means that previous students recommended this teacher to us. Of course we only listen to those students who passed the test...

So even if we have the same (good) teacher, we can still be devided in 2 groups - those who start the "class" and those who are about to take the exam. We all do qbanks and we can see other students "scores" (percentage of those who got the question right).

What does this mean?

Question banks are not a predictor.
Questions are devided in 2 groups (generally) - good (easy + conceptual) and bad (weird and random facts outta the blue)
My "take home" message from this post is - study smart: master the conceptual questions and keep going over easy ones... These are the fair game and most likely be asked on exam

[DocSikorski]

I have covered 55% over with Kaplan Qbank right now.
My cummulative is 70 (though I do subjectwise)
How I would rate Qbank? 50/50 some questions are high yield, some questions are weird, some are very unfair. Qbank does not have a single no-brainer question (I haven't seen so far).

Kaplan qbank has a call to be a conceptual-based question bank.

I can see what is the thought behind these questions and I can see one major similarity in all of them: weak connection line between question and an answer. This is bad.

A good example will be a question that recently was posted to this forum:

65-y old guy had suddenly developed pain around umbilicus that moved to right lower quadrant.

This is the classic presentation of acute appendicitis in every surgical textbook e.g. Sabiston. But the answer was - diverticulitis.
So I guess we should have based our diagnosis on the age of the patient due to Kaplan. This is obviously wrong and unfair.

But the point that Kaplan is trying to stand - they are trying to "overtrain" you, so you always have to think one step further or do one extra calculation. It is a good call, but to reach it - a better job should have been done. So at the end of a day it's more of a "Russian Roulette" type question bank - you have to guess between 2 (or even 3) answer options.

Don't expect high % performance scores with Kaplan. Its claimed to be harder, than Usmle World, but it is what it is. A study tool, where sometimes you study and remember stuff because you disagree with the question and do some research after. It is time consuming.

Even if you do High Yield mode only, you still get the same unfair questions but for classical diseases that tend to appear on boards mostly.