To a great degree, the evaluation element in PHM comprises repetition of one or more of the assessments done for the initial baseline analysis.  Ideally, this repetition, whenever it is repeated, will identify the changes of concern to payers, at least, while any added tracking of results used to sustain participants in their efforts and changes will do the same for the changes of interest to participants.  But saying this, there will inevitably be the complication of making a “business case” that credibly demonstrates that the PHM strategy overall, and its individual interventions, have been the causes of the changes discovered.

Because the value dimensions addressed in EHM efforts and evaluations are major concerns of insurers and employers all the time, it is likely that they are doing something else to reduce costs or improve productivity and performance, beyond the effects of PHM interventions.  And rarely would insurers, much less employers be willing to devote an entire year’s financial performance to the effects of a PHM investment alone.  So generally speaking, the PHM evaluation should include some effort, at least, to separate out what changes in the evaluation dimensions it addresses that were probably affected by other efforts as well.

One of the best ways to track the effects of PHM is to analyze the tracking data that connects PHM efforts, such as HRA, screenings, and other information that was shared with individual participants in the HRA, coaching interventions used with participants in particular PHM interventions, and participant-reported behavior changes, health status biometrics of self-reports, etc.  If this “value chain” of causes and effects can be shown to be connected as expected, that will make a strong case for attributing much at least of measured improvements to the PHM effort rather than “extraneous” causes.

Results such as “yes/no” behavior changes, (smoking, alcohol/drug abuse cessation) or “degree” changes, (increased physical activity, improved nutrition, longer average time slept per night, lower stress, etc.) should correlate with the amount and intensity of interventions, together with participant participation measures.  In other words, the more the inputs by coaches and participants, the more the outputs in terms of behavior change, either a higher rate of yes/no changes, or a greater degree of change in continuous metrics.

In turn, the more the changes or progress achieved in behavior, the greater should be the improvements in biometrics, risk/impairment factor status, etc. that are supposed to be affected by each. This should also appear as a correlation between the different kinds of measures.  And the measured biometrics, health/risk/impairment factor status should be well correlated with changes in healthcare cost, disability, and workers compensation costs, plus productivity and performance metrics or estimates.

The higher the degree of correlation across these changes, the more likely and credible the causal connection between them is.  Only randomized and limited intervention clinical trials will meet academic standards, but demonstrated correlation should provide stronger evidence than simply records of changes in the outcome value, and certainly be better than the most popular approach, which among employers, at least, is not to measure ROI at all.

To determine ROI, of course, it is essential to factor in all costs as well as financial gains discovered in the evaluation.  These will include all  internal costs for payers who do it themselves, and is likely to include internal costs – for staff efforts, incentives paid, internal promotion of participation, team contests, etc.—as well.  The costs imposed in the form of PHM supplier charges should be the easiest to identify, since they involve direct payments.  When all financial benefits are compared to all costs, the ROI ratios and net earnings can be calculated.

There is a further complication in PHM, however, because, as is the case with some marketing expenditures, some portion of PHM costs should probably be treated as longer-term investments, rather than this year’s costs.  While the published cases of long-term PHM results are few, they strongly suggest that when most of the participants in one year’s PHM efforts remain beneficiaries, members, or employees, the second-year results tend to be better than the first, and the third better than the second.  The only case I know of where results were tracked for longer than three years found the fourth-year results essentially flat compared to the third.

This case, however, involved a cohort of employees, roughly 6000, who were continually employed and continuously participating in the PHM program for all four years.  The pattern they produced, which was savings of $233 per participant in the first year, $375 in the second, $944 in the third, and $950 in the fourth, counted both medical care and disability/workers compensation expense savings, but not the productivity losses most likely associated. [G. Stave, et al. “Quantifiable Impact of the Contract for Health and Wellness” JOEM, 45:2 2003 109-117]

But chances are, when there are high turnover rates among health plan members or employees, the degree of improvement in that case would not be replicated.  “Veteran” participants would too often be gone, instead of in their third or fourth year of improvement.  And “novice” participants would have to replicate lower first-year results when it is their first year, even if the fifth or tenth for the payer in terms of PHM program investment.  But as long as there is less than 100% turnover in the participant population, there seems likely to be improvement in annual results for successive years, at least up to three or four.

Improvements usually occur both because it takes time for participants to gain confidence in the own abilities.  It also may take time for health status effects to occur.  Fortunately for employers, many improvements in productivity occur quickly, so this should bolster clients’ confidence.  One of the factors that take time is the influence that successful participants have on their peers who chose not to participate when first offered.  The examples of successful peers, and the word-of-mouth reports spread by them, can be the most effective, and certainly least expensive approach to promoting future participation and improved results.

In general, it may be the third or fourth year until positive or at least desired ROI levels are achieved.  Marathon Health, for example, a PHM supplier in Vermont, guarantees a 2:1 ROI, and often achieves 3:1, though not until the third year.   The huge increase in the third year cited above may have meant no significant ROI was achBy contrast, HealthMedia is willing to guarantee the results it measures, though not ROI, for whatever period it uses in measuring them should its clients desire (per e-mail from Ted Dacko, CEO). HealthMedia only has two years* history with any of its results, and less than that for interventions introduced more recently, has not yet reported long-term results, but it makes employers who want quick results happy.

Whatever method for evaluation is used, it should meet at least standards for accuracy, reliability and validity, and avoid at least the most common errors due to self-selection bias and regression to the mean.  Measuring ROI is clearly superior to pure credence that PHM works, or even relying on results others evaluate and report.  The huge variety of methods currently in use, particularly in what is measured, together with the unavoidable costs of measuring many important sources of value from PHM investments, as well as the need to be able to take a long-term view of such investments, makes choosing an approach difficult, but definitely worthwhile.