Skip Navigation
Department of Health and Human Services
Back to Issue Cover

Comparative Effectiveness Research and Decisionmaking: Part II of a Conversation With Dick Justman and Alan Rosenberg

Comparative effectiveness research provides evidence to support clinicians’ decisionmaking with their patients who face multiple options for the testing or treatment of health conditions. But what role does it play among health policymakers? Michael Fordis, M.D., of the John M. Eisenberg Center for Clinical Decisions and Communications Science, David Hickam, M.D., of the Scientific Resource Center, and Stephanie Chang of the Agency for Healthcare Research and Quality sat down with two health-system administrators — Dick Justman, M.D., the National Medical Director of UnitedHealthcare, a national health service delivery company, and Alan B. Rosenberg, M.D., Vice President of Medical Policy, Technology Assessment and Credentialing Programs for WellPoint, Inc. We discussed the ways in which the results of Comparative Effectiveness Reviews are used in their work. This is the second part of their interview. The first part can be accessed on the EHC Program Web site.

Stephanie Chang: Do you have any sort of parameters set around subpopulations when making decisions?

Dick Justman: To give you an example, the CMS (Centers for Medicare & Medicaid Services) National Coverage Decision with regard to artificial lumbar disc [replacement] singled out people over 60 years old and said this decision applies to those people. It gave no guidance whatever with regard to people under the age of 60. One could argue that the evidence with regard to artificial lumbar disc [replacement] is flawed and meager with regard to all populations. In other words, there might never be populations where the evidence is convincing enough that the particular technology would be safe and effective for that subset of the population.

Alan Rosenberg: Generally, we do look for information on subpopulations, or the population as a whole. And if there is evidence that something works better for a subpopulation, you would say it’s medically necessary for that subpopulation but not for the population in general. We would say it meets medical-necessity criteria for that subpopulation and is not medically necessary for general populations. So, yes, we do evaluate based on evidence and levels of evidence for subpopulations. Also regarding your point, Stephanie, there might be an individual who may do better with a treatment when a specific complicating factor may exist. That is part of why WellPoint provides both for an exception process and for an independent appeal process.

Dick Justman: Alan raises a great point. We all know that breast cancer screening is something that has a beneficial effect on health outcomes. We also need to acknowledge that while mammography is the general standard, it is not the most sensitive imaging study. In other words, you could actually have small tumors that would be missed on a mammogram. MRI (magnetic resonance imaging) is certainly more sensitive, but it’s also considerably less specific. So, should there be coverage for MRIs for breast cancer screening? And if so, would that be for the entire population, or would that be for a defined subset of the population? And if for a defined subset, what would that defined subset be? Those are the kinds of questions we have to look at.

Stephanie Chang: And I would say, you specifically are both policymakers in large organizations. Keep in mind our audiences may be smaller hospital administrators or people who really maybe don’t have the resources or just haven’t thought as clearly through the use of evidence in policy decisions.

Dick Justman: Let’s use the example of robotic surgery, because this question comes up all of the time. If you are a hospital, should you purchase the DaVinci® system for use by your physicians? If you’re a physician, should you use a DaVinci system? And if you should, are you going to be paid for using it? If you are paid for using it, are you going to be paid a differential for using it? If you are a consumer of health care and somebody tells you that you need a radical prostatectomy because you didn’t follow the rules that Alan and I just laid out and you ended up going to a urologist rather than to a family physician or a general internist, should you have an open radical prostatectomy? Should you have a laparoscopic radical prostatectomy? And if you’re going to have a laparoscopic [procedure], should you have it done with a robot, or does it even make any difference? Those are the kinds of questions that we would look at.

I think the answers to those questions would probably be somewhat different for a purchaser of health care, for a hospital, for a physician, or for a consumer. In other words, is there a reason for using a robot? Sometimes there is, and sometimes there isn’t. I’ve talked to urologists who tell me that doing a radical prostatectomy is a physically tiring procedure. And one of the advantages of a robot is that, in fact, there is less physical fatigue for the operating surgeon. From the standpoint of both the physician and the consumer, there is a shorter operating time. There is less bleeding. It is reported that there is less infection. There may be a shorter hospital stay. If that’s the case, then these would all be advantages depending upon who you are. The question is, are there advantages from the standpoint of a payer? I believe that there are. Are there going to be more later complications? Are there going to be rehospitalizations? Are there going to be other surgical procedures? From the standpoint of a radical prostatectomy, there’s also the question of nerve sparing. Is nerve sparing going to be easier to do with a robot or less easy to do with a robot? These are all questions that need to be addressed. We are not able, as a payer, to say there’s insufficient evidence for us to make a decision. We have to make the decision based upon whatever evidence there is. As Alan pointed out, the vast majority of situations that I deal with have to do with technologies that are sufficiently recent. I’m not going to have access to a prospective large, statistically robust, randomized, controlled trial that goes out to 5 years. I’m going to have to make a decision based upon considerably less robust evidence.

Michael Fordis: And Dick, when you make decisions with limited evidence, what factors might cause you to revisit your decision? I mean, it would be obvious if a large trial comes out. But are there other things that you would follow internally that might not be related to what’s coming out in the literature?

Dick Justman: Absolutely. Let me give you an example. You know the gold standard for a woman who wants permanent contraception is tubal ligation — a very safe procedure, but it’s done in a hospital as an outpatient [procedure] and requires general anesthesia. There is a relatively new device called a fallopian tube occlusion device, which can be inserted in a physician’s office with minimal sedation. It requires a hysterosalpingogram to confirm placement. But how does this compare with the gold standard, tubal litigation? The first time we looked at this, there was no evidence to review. If you’re talking about permanent contraception, you’d like to know if there are going to be late failures or if there are going to be complications, such as tubal perforations. So, our initial answer was that there’s insufficient evidence to say that it’s safe and effective. We’re not going to cover it.

We had lots of conversations with the device manufacturers, and they were very collaborative discussions. They said, “What do you want?” And I said, “What I want is a randomized controlled trial.” And they said, “That will never happen because women are not going to randomize themselves into having a procedure done in the office or having a procedure done in the hospital with regard to anesthesia. The trial itself would never be completed.” I said, “I need to know that it’s safe. I need to know that it’s not going to cause infection, catastrophic bleeding, perforation, intractable pain, et cetera. And I also need to know that these women are not going to become pregnant.”

They were able to accumulate sufficient data for us, though not randomized, over a reasonable period of time. These data showed that there was a certain percentage of women who had to have this device removed because of pain, but in relatively small numbers. There were no catastrophic bleeding episodes that required emergency hysterectomies. There was no overwhelming sepsis that required hospitalizations. There were no deaths. And more to the point, over the period that they looked at this, there were no pregnancies. So they said to me, “Well, Dick, there are no pregnancies and, based upon what you said, it appears to be safe. So, what do you think?” I also learned at that point that there are certain markets in which we have enrollees where the only way you can have permanent contraception for a woman is through this device — gynecologists simply don’t do hospital-based tubal ligations any more. Based upon that, in the absence of randomized control trials, we made a decision to cover it.

Dick Justman and Alan Rosenberg both serve as members of the AHRQ Effective Health Care Stakeholder Group.