Introduction

The ultimate task of many image analysis problems is image interpretation. Commonly, this task involves the assignment of each object segmented out from the image into a semantic category. The assignment, or labelling, is based on a set of measurements characterising the object. The design issue is then to identify a decision rule which will accomplish this task with minimum classification error.

Recent studies reported in the literature suggest that the object labelling performance can be significantly improved by means of combining the opinions of multiple experts. The idea is to design several decision rules and to combine their outputs in order to reach a consensus decision about the object identity. Although the majority of the fusion strategies advocated in the literature are largely heuristic, recently a few attempts to underpin a subset of these strategies by a common theoretical framework have been reported [ 2 , 7 ].

From the point of view of their analysis, the approaches to multiple expert fusion can be divided into four categories. The first two groups include strategies applicable for fusing expert opinions based on identical measurements [ 7 , 6 ] and distinct measurements [ 2 ] respectively. The third category comprises multistage combination rules [ 9 ]. Finally, the last family of approaches encompasses data dependent fusion schemes [ 8 ]. In this paper we focus on the second category where the individual experts deploy distinct representations (distinct sets of measurements) and return soft decisions. We present a common theoretical framework for multiple expert fusion and use it as a basis for deriving several fusion strategies. In contrast to our earlier work [ 2 ], we take into account the confidence in the individual expert opinions. We show that this leads to combination strategies which incorporate weighting factors. The weighting factors can be supplied by the experts themselves or where this is impracticable, they can be determined by means of training.

The developed fusion strategies are applied to the problem of automatic detection of microcalcifications (MC) in digital mammograms. Four different experts using different sets of measurements are designed to differentiate between normal and abnormal (MC) bright blobs segmented out from the images by a preprocessing technique. We compare the effectiveness of fusing the multiple experts with and without weighting factors. We show that when weighting factors are ignored, the fused decision can be adversely affected by the worst expert. Although equal weighted fusion will, in general, ameliorate the performance, it will not necessarily outperform the best classifier. Thus weighted multiple expert fusion offers definite advantages over the less sophisticated combination technique.

The paper is organised as follows. In the next section, the theoretical foundations for multiple expert fusion are developed. We then focus on linear combination strategies. An analysis of the sensitivity of the strategies to estimation errors is performed to enhance the understanding of their properties. In Section  3 the fusion strategies are applied to the problem of mammographic image analysis. Two figures of merit are introduced to assess their effectiveness. The section presents the experimental results achieved with the different combination options. Finally, the last section summarises the main results of the paper and offers concluding remarks.