A sensitivity analysis of the PAWN sensitivity index

Abstract

The PAWN index is gaining traction among the modelling community as a sensitivity measure. However, the robustness to its design parameters has not yet been scrutinized: the size (𝑁) and sampling (𝜀) of the model output, the number of conditioning intervals (𝑛) or the summary statistic (𝜃). Here we fill this gap by running a sensitivity analysis of a PAWN-based sensitivity analysis. We compare the results with the design uncertainties of the Sobol’ total-order index (𝑆 ∗ 𝑇 𝑖). Unlike in 𝑆 ∗ 𝑇 𝑖, the design uncertainties in PAWN create nonnegligible chances of producing biased results when ranking or screening inputs. The dependence of PAWN upon (𝑁, 𝑛, 𝜀, 𝜃) is difficult to tame, as these parameters interact with one another. Even in an ideal setting in which the optimum choice for (𝑁, 𝑛, 𝜀, 𝜃) is known in advance, PAWN might not allow to distinguish an influential, non-additive model input from a truly non-influential model input.