Should uncertainty be incorporated in preference-sensitive
decision support? Invitation to debate
Mette Kjer Kaltoft12, Jesper Bo Nielsen2, Jack Dowie24
Conclusion: In one view setting up mean and uncertainty as separate criteria involves double counting, the calculation of the mean as Expected Value has already ‘synthesised’ the uncertainty in the distribution. In the alternative view, the mean calculation, while reflecting the uncertainty in the distribution, has left it as a separate output to be addressed by the decision maker. Their preference-based weighting of mean and uncertainty can therefore be entered into a new Expected Value calculation, leading to a preference-sensitive result. A method to implement this is shown to be feasible. Whether it should be followed is opened for debate.
1 Odense University Hospital - Svendborg, Denmark; 2 University of Southern Denmark, Odense, Denmark; 4 London School of Hygiene and Tropical Medicine, UK,
Contact: mkaltoft@health.sdu.dk jack.dowie@lshtm.ac.uk
Simple Hypothetical Example
Empirical examples: Bone health and HIV Antiretrovirals
End-of-life example: Will Tremain
Purpose. In the dominant paradigm, researchers establish the uncertainties, along with the expected average outcomes, and hand over the task of dealing with the separated outputs – e.g. Means and Credible Intervals - to the decision maker. Whether individual or group, the decision maker is expected to make the necessary trade-off as part of the final deliberative ‘making up of mind’. There is no attempt to synthesise, transparently and systematically, the joint means and uncertainties of the performance ratings for all options on all criteria. The essence of an MCDA-based person-centred decision support tool (DST) is that all required value judgments are elicited from the decision maker inside the tool, at or near the point of decision. Sensitivity analyses and stochastic displays merely re-present the same task in a different way.
�Method.
The method, proposed for debate, involves treating the means and Credible Intervals of all outcomes and process considerations as separate criteria in a Multi-Criteria Decision Analysis-based DST. The decision maker has their importance weights for mean and uncertainty elicited for each criterion separately, so the resulting trade-offs are criterion-specific and may differ between life expectancy and quality of life, for example. The Expected Value (EV) score for each option within the DST incorporates this trade-off. (N.B. The uncertainty reported in a study cannot be regarded as an indicator of its technical quality, which should be separately graded.)
�Results. Four proof-of-method examples are provided. Each can be explored interactively online (https://ale.rsyd.dk/) enter 1406 and 1424 as survey IDs or access by QR codes). They comprise a hypothetical example with two options/three considerations/six criteria; an empirical 6/2/4 example on medications for bone health; and a 6/4/8 example on antiretroviral regimes for HIV.
QR to access surveys for hypothetical and �empirical examples
QR to access survey �for End-of-life �example
References
Broekhuizen H, Ijzerman MJ, Hauber AB. Weighing clinical evidence using patient preferences : An application of probabilistic Multi-Criteria Decision Analysis. Pharmacoeconomics. 2017;35(3):259-69.
Freemantle N, Cooper C, Diez-Perez A, Gitlin M, Radcliffe H, Shepherd S, et al. Results of indirect and mixed treatment comparison of fracture efficacy for osteoporosis treatments: a meta-analysis. Osteoporosis International 24(1):209-17
Will Tremain is given 6 months to live on Palliative Care, with a Credible Interval around this average estimate of 3 to 9 months. An Operation offers an average of 9 months, with a Credible Interval of 0 to 18 months. In the first of the Annalisas below his initial weightings produce equipoise. In the centre Annalisa his final weights show Palliative Care as the opinion of the tool when length of life is the only consideration. In the third Annalisa on the right adding other criteria fails to change the opinion, given his ratings and weightings for the additional criteria.
The data are from a Network Meta-Analysis of medications for reducing fracture risk by Freemantle et al. (2013). We selected six of the medication options and two of the criteria: Vertebral Fracture and Non-Vertebral Fracture (i.e. Hip, Shoulder, or Wrist). The relative importance attached to avoiding each is elicited, along with the extent of concern with the uncertainty. The verdict in favour of Zolendronate is robust, but concern with Uncertainty can lead to Teraparatide.
This 4 considerations/8 criteria illustration is based on Broekhuizen et al (2017) MCDA for highly active antiretroviral treatment regimes for HIV. Weightings data are from their Table 1, Ratings data (reoriented to positive direction) are from their base case in Table 2. Average preference data are supplied in paper, but in the online interactive version they can be varied using the cursor, resulting in a change of opinion in the tool from left to right.
There are two considerations, Benefit and Avoided Harm. Hypothetical data are used for the mean effect of two options (Drug and Surgery) on these criteria, and for the parameter uncertainty measures as absolute Credible Intervals (Cis). The Annalisa presented to the decision maker has four criteria: the mean and CI of Benefit and the mean and CI of Avoided Harm. They have their importance weights for the means and uncertainties of both considerations elicited before the Annalisa decision screen is presented. This elicitation is done for each criterion separately, so the trade-offs are criterion-specific, permitting that for Benefit to be different from that for Avoided Harm. Ignoring uncertainty produces a ‘toss-up’ or equipoise (left). Alternative weights can flip the opinion of the DST either way (centre and right).
COI: Jack Dowie has a financial interest in Annalisa
This poster