Modern Tools for Agent-Based Model Sensitivity Analysis

Since the eighties of the last century, agent-based modeling has become both a complement and a substitute for more traditional economic-modeling methodologies. In macroeconomics, for example, agent-based models (ABMs) are nowadays considered as a valid and effective competitor of standard Dy- namic Stochastic General Equilibrium (DSGE) models [1]. Likewise, ABMs of financial markets are often considered better than traditional models based on the efficient-market hypothesis in explaining the statistical properties of buy- and-sell high-frequency dynamics [2].
Sensitivity analysis (SA) constitutes an open challenge for ABMs in eco- nomics and finance. SA is central to effective model design and intuitive evalua- tion, where one aims to rank parameter importance when explaining output vari- ance or ”sensitivity” to changes in the input parameters (or initial conditions) of the model. This sensitivity to input parameters represents a response for how the model (approximately) behaves when input parameters are changed. Un- derstanding the models response to (possibly joint) changes in parameter values forms the basis for assessing model robustness and drawing robust implications from policy exercises [3]. Unfortunately, as ABMs become more realistic, they require an increasing number of parameters. This results in highly prohibitive computational costs when assessing ABM sensitivities. Surrogate meta-models offer a solution to this computational burden [4].
We propose a wide sensitivity analysis of the Brock and Hommes asset- pricing model [5] and the Franke and Westerhoff limit-order book model [2] directly and on a variety of surrogate meta-models using two well-known ap- proaches to SA and a recently introduced game-theoretic algorithm for ”model explanation” from the machine learning literature.
Novel methods for assessing the importance of each parameter in explaining the variance of the output are performed [6; 7]; they are as well compared to other ones previously employed (in literature) with the same purposes [8]. In particular, it’s shown the output response of the two models employed varying one or more parameters through SA methods (e.g. first and second orders Sobol’s indices); morover, it’s highlighted how changing bounds can heavily impact the sensitivity analysis and how the modeler can use their intuition for how the sensitivity should rank to find bounds that reflect this. Furthermore, it’s debated how variation plots can be used to show how a user can work with a proper agreed calibration in exploring specific parameter variations.