home email us! sindicaci;ón

Archive for Quality Measurement



Unintended consequences of quality measurement and incentives

by Emily DeVoto

This session was chaired by Thomas Valuck, MD, JD, Director, Special Program Office for Value-Based Purchasing, Centers for Medicare & Medicaid Services (CMS); panelists were Brent James, Vice President for Medical Research and Executive Director of the Institute for Healthcare Delivery Research, Intermountain Health Care; and Tom Sackville, Chief Executive, International Federation of Health Plans and former British Minister of Health.

Here’s a summary of a slide I liked from Brent James:

Three ways to get a better number:

1. Measure, manage, and improve
2. Subuptomize - make one area loook better at the expense of others that are unknown, unmeasured, or outside of scope
3. Game the numbers - move high-risk patients around, cherry-pick patients, etc.

Tom Sackville added a dose of wry British humor to the proceedings, poking fun at the U.S. Congress as well as himself. The British National Health system found that implementing 150-some measures in a pay-for-performance scheme led to a box-ticking frenzy, in which gross overestimates of prevalence of various disorders were a red flag.

Here’s a summary of the Q and A (paraphrased), beginning with questions from the moderator, Thomas Valuck:

1. What are the negative effects of incentivizing?

Brent James replied that the worst thing you can do to a physician is tell him that money is more important than their professional care. We have found that bigger payments started to become perverse incentives. Professional incentives must come first, then you can back them up and add a little money to make professional incentives real.

A question for Mr. Sackville: Did you see anything interesting about the polling questions? How does it compare to NHS system?

Answer: In Britain, they’ve gone much too far down the road of financial incentives. Huge bureaucracy ensued as well. Don’t give people the temptation to fiddle with the rules; you create a very cynical bunch of people out of a very vocational group of docs.

Audience questions: Assuming quality and costs are important, how do consumers use information for decisionmaking? Is it possible consumers use stories, not statistics, because that’s what they’re used to? Emily: Great Question.

Brent James talked about a study in which researchers gave carefully prepared information to patients; the conclusion was that people act on emotion. Data may not be sufficient - e.g., adjuvant chemotherapy for breast cancer - stories are equally important in such cases.

Tom Sackville agreed: “Most people are left unimpressed by statistics.” People judge health service by what happens to them, their families, what people tell them about it. Most people disapprove of HMOs in general, based on stories, he said, but a lot fewer disapprove of their own HMO.

Question: How does panel see P4P functioning, if at all, under consumer-directed health plans? Moderator follow-up: We haven’t had very good access to information, so we don’t know how to use it. Does P4P create a Garrison Keillor effect: is every doctor above average?

Brent James talked about evaluation of patient education around heart disease; education been shown to be a critical element of patient outcomes. Counseling takes 30 to 45 minutes to effect desired outcome. Or, you can give patients a sheet of information. This led James to offer a good example of gaming the numbers: Some clever staff in one hospital overcompensated by delivering an information sheet to every bed in the hospital, whether the condition was relevant or not, and even when the patient was asleep; the hospital got a perfect score for patient education, but distributing the information this way didn’t improve patient outcomes.

James noted that all publicly traded companies have a CPA, and most of the financial measures are integrated into daily hospital operations. Apparently these systems represent an opportunity to support quality measurement.

Question: Please comment on pay for choice, to get consumers to pick high quality doctors.

James: How do tell which providers are high quality? Attribution of outcomes is the problem. Thus, you have an initial technical problem to solve before you can incentivize patients.

Sackville: In the UK, nothing incentivizes doctors to talk to their patients about prevention. Parents are losing control of what kids eat, and there’s a growing obesity problem. Is something going to be done to bring sticks and carrots to this area? We need to get tougher on patients, too. [Emily: Growing obesity problem (my unintended pun, but is there a better word?). Carrots (Sackville’s). Sorry…]

Overall, both of these speakers were pretty frank and realistic about the potential for misuse of quality measurement and pay-for-performance systems. I dropped some text at the end about how to model such systems to avoid these problems, because the presentation got a little rushed at the end and what I wrote down wasn’t completely understandable to me when I went to reconstruct it. So let’s leave that good question out there and see if we can get a fuller discussion.