Axiomatic properties of inconsistency indices for pairwise comparisons
Matteo Brunelli1* and Michele Fedrizzi2 1Aalto University, Espoo, Finland; and 2University of Trento, Trento, Italy
Pairwise comparisons are a well-known method for the representation of the subjective preferences of a decision maker. Evaluating their inconsistency has been a widely studied and discussed topic and several indices have been proposed in the literature to perform this task. As an acceptable level of consistency is closely related to the reliability of preferences, a suitable choice of an inconsistency index is a crucial phase in decision-making processes.
The use of different methods for measuring consistency must be carefully evaluated, as it can affect the decision outcome in practical applications. In this paper, we present five axioms aimed at characterizing inconsistency indices. In addition, we prove that some of the indices proposed in the literature satisfy these axioms, whereas others do not, and therefore, in our view, they may fail to correctly evaluate inconsistency.
Journal of the Operational Research Society (2015) 66(1), 1–15. doi:10.1057/jors.2013.135
Published online 4 December 2013
Keywords: pairwise comparisons; inconsistency indices; axiomatic properties; analytic hierarchy process 1. Introduction
Pairwise comparisons have been used in some operations research methods to represent the preferences of experts and decision makers over sets of alternatives, criteria, features and so on. For simplicity, in this paper we shall speak of alternatives only, bearing in mind that it is a reductive view. The main advantage in using pairwise comparisons is that they allow the decision maker to compare two alternatives at a time, thus reducing the complexity of a decision-making problem, especially when the set under consideration is large, and serve as a starting point to derive a priority vector that is the final rating of the alternatives. Pairwise comparisons have been used in well-known decision analysis methods, for instance, the Analytic Hierarchy Process (AHP) by Saaty (1977) (see Ishizaka and
Labib, 2011 for an updated discussion), and its generalizations, which have been proved effective in solving many decision problems (Ishizaka et al 2011).
In the literature, and in practice, it is assumed that the dependability of the decision maker is related to the consistency of his/her pairwise judgements. That is, the more rational the judgements are, the more likely it is that the decision maker is a good expert with a deep insight into the problem and pays due attention in eliciting his/her preferences. Similarly, if judgements are very intransitive and irrational, it is more plausible that the expert expressed them with scarce competence, as he/ she would lack the ability to rationally discriminate between different alternatives. This is summarized by Irwin’s (1958) thesis claiming that ‘… preference is exactly as fundamental as discrimination and that if the organism exhibits a discrimination, it must also exhibit a preference and conversely’. Following Saaty (1994), the approach to decision making based on pairwise comparisons, and the AHP in particular, is grounded in the relative measurement theory and it is in this framework that Saaty (1993) too claimed that pairwise comparisons should be ‘near consistent’ to ensure that they are a sufficiently good approximation of the decision makers’ real preferences. This seems to support the importance of having reliable tools capable of capturing the degree of inconsistency of pairwise comparisons. The importance of having reliable inconsistency indices becomes even more evident when one considers that their practical use has gone beyond the sole quantification of inconsistency. For instance, they have been employed by
Lamata and Peláez (2002) and Shiraishi et al (1999) to estimate missing comparisons, by Harker (1987) to derive ratings of alternatives from incomplete preferences, and by Xu and
Cuiping (1999) and Xu and Xia (2013) to improve the consistency of pairwise comparisons. Moreover, although in the following we shall focus on the by far most popular type of pairwise comparisons, consistency has been an important topic for other representations of preferences, for instance linguistic preference relations represented by 2-tuples (Dong et al, 2013).
On this fertile ground, researchers have proposed various inconsistency indices—functions associating pairwise comparisons with real numbers representing the degrees of inconsistency of the pairwise judgements. In this paper, we concern ourselves with the fact that inconsistency indices have been introduced heuristically and independently from each other, *Correspondence: Matteo Brunelli, Systems Analysis Laboratory, Department of Mathematics and Systems Analysis, Aalto University, PO Box 11100,
Espoo, Aalto, FIN-00076, Finland.
Journal of the Operational Research Society (2015) 66, 1–15 © 2015 Operational Research Society Ltd. All rights reserved. 0160-5682/15 www.palgrave-journals.com/jors/ referring neither to a general definition nor to a set of axiomatic properties. Hence, this paper introduces some axiomatic properties for inconsistency indices and shows that some indices proposed in the literature fail to satisfy these axioms. This paper is outlined as follows. In Section 2, we introduce preliminary notions and the notation. In Section 3, we briefly define the inconsistency indices that are studied in the paper. Next, in
Section 4 we introduce and interpret five axioms, and in Section 5 we present results regarding the inconsistency indices and prove that some of them satisfy the required axioms while four others do not. For a simpler description, some proofs are given in the Appendix. In Section 6, we conclude the discussion of the axioms and draw the conclusions.
Throughout this paper, we refer to ‘inconsistency indices’, as what they really measure is the amount of inconsistency in pairwise comparisons. Nevertheless, in the literature such indices are often referred to as ‘consistency indices’, while both expressions refer to an index that estimates the deviation from consistency. 2. Preliminaries