Jointly optimized ensemble deep random vector functional link network for semi-supervised classification

Abstract

Randomized neural networks have become more and more attractive recently since they use closed-form solutions for parameter training instead of gradient-based approaches. Among them, the random vector functional link network (RVFL) and its deeper version ensemble deep random vector functional link network (edRVFL) show great performance on both classification and regression tasks. However, the previous research on these two models mainly focuses on the supervised learning area. Although there have been efforts to extend the RVFL network to solve semi-supervised learning problems, the potential of the edRVFL network has not been fully investigated. Therefore, we propose a jointly optimized learning strategy for the edRVFL network (JOSedRVFL) for semi-supervised learning tasks in this paper. The JOSedRVFL network uses an iterative procedure to compute the output weights and consequently predicts the class labels of the unlabeled training data during the training process. In addition, we propose another semi-supervised edRVFL network (SS-edRVFL) using manifold regularization in this work. We then do a brief comparison between these two methods to illustrate their similarities and differences. In the experimental part, we conduct the first set of experiments using the UCI datasets to compare the performance of our proposed semi-supervised algorithms against 11 other classifiers to demonstrate the superior performance of the SS-edRVFL and JOSedRVFL networks. JOSedRVFL achieves the highest accuracy on all 4 datasets while SS-edRVFL takes the second place 3 times which is only worse than JOSedRVFL. Moreover, we apply the proposed methods to real-world applications using the electroencephalography-based emotion recognition dataset to compare the performance of RVFL-based methods (RVFL, SS-RVFL, and JOSRVFL) and their edRVFL counterparts (edRVFL, SS-edRVFL, and JOSedRVFL). Results from this test revealed that the edRVFL-based models (edRVFL, SS-edRVFL, and JOSedRVFL) can obtain higher accuracy than the RVFL-based versions (RVFL, SS-RVFL, and JOSRVFL) with the same learning framework on 45 real-world semi-supervised benchmarks. We then perform the Wilcoxon signed-rank test to show that JOSedRVFL is significantly better than 5 other competitors, which supports our claim that JOSedRVFL can be treated as a superior classifier for semi-supervised classification on both benchmark datasets and real-world applications.