Self-organized Operational Neural Networks with Generative Neurons
View/ Open
Publisher version (Check access options)
Check access options
Date
2021Author
Kiranyaz, Mustafa SerkanMalik J.
Abdallah H.B.
Ince T.
Iosifidis A.
Gabbouj M.
...show more authors ...show less authors
Metadata
Show full item recordAbstract
Operational Neural Networks (ONNs) have recently been proposed to address the well-known limitations and drawbacks of conventional Convolutional Neural Networks (CNNs) such as network homogeneity with the sole linear neuron model. ONNs are heterogeneous networks with a generalized neuron model. However the operator search method in ONNs is not only computationally demanding, but the network heterogeneity is also limited since the same set of operators will then be used for all neurons in each layer. Moreover, the performance of ONNs directly depends on the operator set library used, which introduces a certain risk of performance degradation especially when the optimal operator set required for a particular task is missing from the library. In order to address these issues and achieve an ultimate heterogeneity level to boost the network diversity along with computational efficiency, in this study we propose Self-organized ONNs (Self-ONNs) with generative neurons that can adapt (optimize) the nodal operator of each connection during the training process. Moreover, this ability voids the need of having a fixed operator set library and the prior operator search within the library in order to find the best possible set of operators. We further formulate the training method to back-propagate the error through the operational layers of Self-ONNs. Experimental results over four challenging problems demonstrate the superior learning capability and computational efficiency of Self-ONNs over conventional ONNs and CNNs.
Collections
- Electrical Engineering [2647 items ]
Related items
Showing items related by title, author, creator and subject.
-
Real-Time Patient-Specific ECG Classification by 1D Self-Operational Neural Networks
Malik J.; Devecioglu O.C.; Kiranyaz, Mustafa Serkan; Ince T.; Gabbouj M. ( IEEE Computer Society , 2021 , Article)Despite the proliferation of numerous deep learning methods proposed for generic ECG classification and arrhythmia detection, compact systems with the real-time ability and high accuracy for classifying patient-specific ... -
Exploiting heterogeneity in operational neural networks by synaptic plasticity
Kiranyaz, Mustafa Serkan; Malik J.; Abdallah H.B.; Ince T.; Iosifidis A.; Gabbouj M.... more authors ... less authors ( Springer Science and Business Media Deutschland GmbH , 2021 , Article)The recently proposed network model, Operational Neural Networks (ONNs), can generalize the conventional Convolutional Neural Networks (CNNs) that are homogenous only with a linear neuron model. As a heterogenous network ... -
Operational neural networks
Kiranyaz, Mustafa Serkan; Ince T.; Iosifidis A.; Gabbouj M. ( Springer , 2020 , Article)Feed-forward, fully connected artificial neural networks or the so-called multi-layer perceptrons are well-known universal approximators. However, their learning performance varies significantly depending on the function ...