Sensitivity auditing

Sensitivity auditing is an extension of sensitivity analysis for use in policy-relevant modelling studies. Its use is recommended when a sensitivity analysis (SA) of a model-based study is meant to demonstrate the robustness of the evidence provided by the model, but in a context where the inference feeds into a policy or decision-making process.

In these cases, the framing of the analysis itself, its institutional context, and the motivations of its author may become highly relevant, and a pure SA - with its focus on parametric (i.e. quantified) uncertainty - may be insufficient. The emphasis on the framing may, among other things, derive from the relevance of the policy study to different constituencies that are characterized by different norms and values, and hence by a different story about `what the problem is' and foremost about `who is telling the story'. Most often the framing includes implicit assumptions, which could be political (e.g. which group needs to be protected) all the way to technical (e.g. which variable can be treated as a constant).

In order to take these concerns into due consideration, sensitivity auditing extends the instruments of sensitivity analysis to provide an assessment of the entire knowledge- and model-generating process. It takes inspiration from NUSAP, a method used to qualify the worth (quality) of quantitative information with the generation of `Pedigrees' of numbers. Likewise, sensitivity auditing has been developed to provide pedigrees of models and model-based inferences. Sensitivity auditing is especially suitable in an adversarial context, where not only the nature of the evidence, but also the degree of certainty and uncertainty associated to the evidence, is the subject of partisan interests. These are the settings considered in Post-normal science or in Mode 2 science. Post-normal science (PNS) is a concept developed by Silvio Funtowicz and Jerome Ravetz, which proposes a methodology of inquiry that is appropriate when “facts are uncertain, values in dispute, stakes high and decisions urgent” (Funtowicz and Ravetz, 1992: 251–273). Mode 2 Science, coined in 1994 by Gibbons et al., refers to a mode of production of scientific knowledge that is context-driven, problem-focused and interdisciplinary. Carrozza (2015) offers a discussion of these concepts and approaches.

...
Wikipedia