Quantifying plasticity in simulation models

Different methodologies for evaluating aspects of model performance going beyond the pure agreement between measured and simulated data have been recently proposed. These indicators and criteria for the evaluation of, e.g., complexity and robustness can be used in conjunction with well-known metrics for the evaluation of model accuracy - such as root mean square error and modelling efficiency - to get a deeper knowledge about models structure and behaviour. The aim of this paper is to propose an indicator of model plasticity, defined as the aptitude of a model to change the sensitivity to its parameters while changing the conditions of application. Sensitivity was here analyzed using the Sobol' method for sensitivity analysis (SA). Concordance among parameters relevance (total order effect) estimated under different conditions allowed to quantify changes in the way models react to different environments. The concordance among the different SA results was related to the variability of a normalized agrometeorological indicator used to characterize the explored conditions. The plasticity indicator was tested using three different crop models (WARM, CropSyst, WOFOST; rice was simulated), 10 European locations, and 10 years for each location, for a total of 5,939,200 simulations and 300 SA experiments. Results indicated WOFOST as the most plastic, both within location, year, and using all the combinations location × year, whereas WARM showed to be the less plastic across the conditions explored. Previous studies carried out on the same models in northern Italy seem to suggest a direct relationship between model complexity and plasticity, whereas model accuracy seems to be unrelated to these features. This consideration underlines that, in case of availability of different models with a similar degree of accuracy, different choices should be performed for different modelling studies, characterized by different aims and conditions of application.