Tuesday, August 5, 2014

Board passage rates and residency program quality

I am 3rd year med student thinking of surgery programs to apply to next year. I came across this document on the American Board of Surgery website. It shows the qualifying exam and certifying exam first-time pass rates for general surgery residents between 2008-2013.

Is this a worthwhile marker for evaluating surgery programs and identifying top programs? Also, is first-time pass/fail on the QE/CE a measure of preparation to practice as a general surgeon upon graduation or is it poorly correlated with a graduate's ability to function independently?

I have a moderately competitive Step 1 score and I want to choose a program that would make me a general surgeon without needing to take a fellowship upon graduation.

[Note: The email was edited for length.] 


Thanks for asking a couple of really good questions. The answers are not black and white. On page 23 of its requirements, the Residency Review Committee (RRC) for Surgery states that board passage rate is one measure for evaluating program effectiveness, and "At minimum, for the most recent five-year period, 65% of the graduates must pass each of the qualifying and certifying examinations on the first attempt."

A quick look at the board passage rates in the ABS document shows that 31% of programs, most of which are community hospital-based, did not reach the magic 65% quota.

Two years ago, I blogged about the two major reasons why the 65% board passage threshold on the first try discriminates against community hospital programs. You can read the full piece here, but briefly one issue is that university hospitals attract smarter residents who are better test takers, and the other is that smaller programs are statistically more likely to have test result outliers.

I am unaware of any correlation between passing the boards on the first try (or the second or third tries) and a surgeon's ability to practice independently. In fact, another post I wrote discussed a paper that surveyed 4882 surgical residents. It found that community hospital trainees were more satisfied with their operative experience and more confident that they could work independently than those who trained in university programs.

But there is another consideration. Fair or not, programs with first-time board passage rates chronically below 65% are at some risk for both RRC probation, which is detrimental to recruiting, and possible discontinuation. It is difficult to quickly turn around a low board passage rate because the number of graduates is small in most community programs and the stain left by a failed first-time taker lasts five years.

So what should you do?

Perhaps you should play it safe and apply to community hospital programs with adequate first-time board passage rates.

Comments from surgical educators, current residents or recent graduates of surgical training are welcome.



6 comments:

  1. Board passage aside, my anecdotal experience working with residents at community and university programs finds community-trained surgeons are better prepared for practice after residency. But, this observation may simply reflect the intention of residents to practice immediately, versus pursuit of fellowship training. Fellowship, unfortunately, is becoming the place to finish training not completed in residency.

    ReplyDelete
    Replies
    1. Chris, I agree. Some say as many as 85% of graduating chief residents are doing fellowships.

      Delete
  2. Possibly a stupid question, but um ... why would a surgeon need to pass an exam solely for surgery? I'd rather them prove they could do a lap choly or the like, think on their feet: criteria which are much more needed in a surgeon than a board test. Not saying they don't have a place, but I dont care if my surgeon can pass a test: I need to know he can fix my body part and get me alive out the end and that's not exactly a lot of test material in that.

    ReplyDelete
    Replies
    1. It's not a bad question. The first part of the two-part general surgery board-exam is a multiple choice test. Part 2 is an oral exam which does test thinking on one's feet and decision making. Here's a more detailed explanation http://skepticalscalpel.blogspot.com/2014/04/why-arent-all-board-recertification.html.

      There are about 1200 new surgeons graduating from residency every year. It would be impossible for members of the board to watch them all operate. The program director of the residency has to attest in writing that the graduate is competent to both take care of patients and operate.

      Delete
  3. When making a tough decision all statistics are fair game for consideration. I remember on a residency interview asking a faculty member why their pass rate was so low (roughly around 60% if I remember correctly), and I was told "Some people don't want to read the book." I didn't care much for that answer, even if it was true. PDs should feel more pressure not to let someone progress through a program who won't have a fighting chance to pass the boards - that's their job description.

    ReplyDelete
    Replies
    1. Josh, thanks for commenting. You have a point, but please understand that it is complicated. Like most people, we hate to admit we made a mistake and selected the wrong person. When you cut a resident loose, it can be hard to find a suitable replacement. Bringing someone in at the PGY-2 or 3 level doesn't always work out. Then you are really stuck.

      Delete