Speaker
Description
The aim of the work is to find important characterizations of mixture of experts which have
an impact on improvement of combined classifier performance over the averaged
performance of the base learners. The problem was examined for various high
dimensional genomic data sets.
Mixture of experts are useful for responses differentiating among base classifiers.
From this point of view diversity of architectures is key. Different models can respond
differently to limited amounts of data, which is crucial problem in genomic data, reducing
the risk of overfitting.
Diversity can be forced by merging learners of the following different ideas. Base
classifiers are among other: neural networks with different parameters as: decay
parameter, starting weight, number of neurons and layers. Additionally, the following
machine learning methods were applied: Support Vector Machines with different kernels
and regularization parameters diagonal and non-diagonal shrinkage discriminant analysis, naive Bayes and random forest. Those algorithms present different optimization strategies, which may help to avoid getting stuck in local minima.
Diversity of base classifiers is additionally forced by taking different defined sizes of
variables sets and different selection methods (single and combined). We are also
interested in the number of combined classifiers from aforementioned set, because too few
models in the expert mix may not provide an advantage while too many similar models
lead to redundancy.
Generalized estimating equations with auto-regressive correlation structure for increasing number of genes, identity link and variance to mean Gaussian relation were applied to model the improvement of mixture of experts over the mean of base learners performance. The explanation variables in the model were: the number of selected genes, diversity measure, the squared previous values and interaction between diversity and size of genes, also for squared
values. Clusters in generalized estimating equations were defined as the genes selection
method.
Various diversity and similarity measures based on results of constituent classifiers
are taken into account: diversities based on entropy, several types of mutual information
and various measures based on averaged diversities for all pairs of classifiers.
Examined predictors are important in different scenarios. Standardized coefficients in generalized estimating equations models with P-values indicate the most significant characterizations for improvement of performance of joined classifiers. The results maybe useful to in order to predict the best possible ensemble of classifiers.
References
Kamateri, E.; Salampasis, M. An Ensemble Framework for Text Classification. Information
2025, 16 (2), 85.
32144123319