The Bridge

What a product trial scorecard actually looks like

All evidence points to hospitals deploying product trials more frequently than in the past to support sourcing decisions. In last week’s post, we reviewed survey data showing that hospital supply chain leaders are far more active in pushing product review trials than their physician colleagues. But how do hospitals actually evaluate a product trial?

At Tisoni Health, a pseudonym for a three-hospital system in the American Midwest, value analysis committees distribute scorecards to physicians and nurses to gauge whether clinical staff view a potential new product to the system as an improvement over current products. Participants rank the products along several criteria using a scale from one to five and can also write qualitative comments. VAC participants then average the scores that a particular group has submitted to the VAC to simplify in-meeting evaluation of the scorecards.


Our reactions to the scorecard

I immediately noticed a couple things when Tisoni sent me this scorecard. First, it was clearly created by supply chain/procurement executives and offers a window into what these stakeholders value. While there is room left for physicians to discuss clinical nuances at the bottom of the page, the quantitative components of the scorecard are mostly focused on product economics and efficiency instead of clinical value.

  • Only the first item on the list is directly related to product clinical quality.

  • Scan-able labels. This function represents 40% of a product scorecard at Tisoni, indicating that this is a key priority for the organization.

  • The last two items show supply chain’s move into looking at ancillary economic benefits. Patient pain management affects HCAHPS scores and Medicare hospital reimbursement, and efficiency clearly affects throughput and potentially length of stay. There are certainly other ways that products can affect hospital economics that are not included on this scorecard. But, the scorecard does show that hospitals (or at least Tisoni) actively care about a wide variety of product metrics.

The second thing I thought about was the usefulness of the scorecard. Frankly, I’m not sure if this is a good scorecard, but I can’t be sure. The efficacy of this trial scorecard likely comes down to implementation and whether the value analysis committee is willing to seriously consider qualitative clinical feedback. If the quantitative score is largely used to make decisions, then the scorecard seems to value product labeling too highly. When your products are subjected to a trial at a facility (especially if your products are new to the facility) you’ll need to understand both the evaluation metrics and where there is nuance is the evaluation process.

Like the scorecard? There's more resources like it.

This scorecard comes from our Value Analysis Committee Resource Center, a collection of resources available to Health Care Industry Committee members to help them better align their go-to-market approach with provider decision-making, ensure messaging resonates with their target audience, and differentiate products and services.