Classification involves assigning observations to known classes based on some common features. Fisher’s discriminant analysis commonly known as Linear and Quadratic Discriminant Analysis (LDA and QDA, respectively) are famous methods used to solve classification problems. Both LDA and QDA assume the given classes follow a multivariate normal distribution. In the case of LDA, the classes share a common covariance matrix which could be too restrictive. In the case of QDA, each class has its own covariance matrix, which could lead to a problem while estimation.
Clustering methods search for natural groups of similar observations in data. Model-based clustering, particularly mixtures of Gaussian, remains one of the popular clustering methods where the search for classes relies on the assumption that samples within a cluster come from a specific normal distribution.
This work proposes a covariance-based clustering method to build a classifier to overcome the simplicity of LDA and the complexity of QDA. For this, a finite mixture of Wishart is developed for clustering the cross-product matrices with similar structures. The parameter estimates of the mixtures are obtained through the Expectation Maximization (EM) algorithm. Initialization of parameters for the EM algorithm is proposed. Having identified similar covariance structures in the data, we propose a method, cluster-based LDA, which uses the result of the clustering to build a classifier. The proposed method is studied through simulated data and application to glass dataset in the evclass R package.