Deep learning-based EEG analysis: investigating P3 ERP components

¹ Department of Electrical, Electronic and Information Engineering “Guglielmo Marconi” (DEI), University of Bologna, Cesena Campus, 47522 Cesena, Italy

² Alma Mater Research Institute for Human-Centered Artificial Intelligence, University of Bologna, 40126 Bologna, Italy

³ Interdepartmental Center for Industrial Research on Health Sciences & Technologies, University of Bologna, 40126 Bologna, Italy

^*Correspondence: davide.borra2@unibo.it (Davide Borra)
^† These authors contributed equally.

J. Integr. Neurosci. 2021, 20(4), 791–811; https://doi.org/10.31083/j.jin2004083

Submitted: 28 May 2021 | Revised: 7 July 2021 | Accepted: 19 July 2021 | Published: 30 December 2021

This is an open access article under the CC BY 4.0 license (https://creativecommons.org/licenses/by/4.0/).

Abstract

The neural processing of incoming stimuli can be analysed from the electroencephalogram (EEG) through event-related potentials (ERPs). The P3 component is largely investigated as it represents an important psychophysiological marker of psychiatric disorders. This is composed by several subcomponents, such as P3a and P3b, reflecting distinct but interrelated sensory and cognitive processes of incoming stimuli. Due to the low EEG signal-to-noise-ratio, ERPs emerge only after an averaging procedure across trials and subjects. Thus, this canonical ERP analysis lacks in the ability to highlight EEG neural signatures at the level of single-subject and single-trial. In this study, a deep learning-based workflow is investigated to enhance EEG neural signatures related to P3 subcomponents already at single-subject and at single-trial level. This was based on the combination of a convolutional neural network (CNN) with an explanation technique (ET). The CNN was trained using two different strategies to produce saliency representations enhancing signatures shared across subjects or more specific for each subject and trial. Cross-subject saliency representations matched the signatures already emerging from ERPs, i.e., P3a and P3b-related activity within 350–400 ms (frontal sites) and 400–650 ms (parietal sites) post-stimulus, validating the CNN+ET respect to canonical ERP analysis. Single-subject and single-trial saliency representations enhanced P3 signatures already at the single-trial scale, while EEG-derived representations at single-subject and single-trial level provided no or only mildly evident signatures. Empowering the analysis of P3 modulations at single-subject and at single-trial level, CNN+ET could be useful to provide insights about neural processes linking sensory stimulation, cognition and behaviour.

Keywords

Electroencephalography

P3a

P3b

Convolutional neural networks

Decision explanation

Figures

Fig. 1.

Previous article in this issue

Next article in this issue

Fig. 1.

Proposed data-driven EEG analysis tool: workflow and scheme of the adopted CNN. (A) CNN+ET analysis framework. The input EEG trial ${X_{i}}^{(s)}$ is processed by the CNN-based parametric classifier $f\left({X_{i}}^{\left(s\right)};\theta{}\right)$ . Using $f\left({X_{i}}^{\left(s\right)};\theta{}\right)$ and the correct output label ${y_{i}}^{(s)}$ , the relevance representation $g\left({X_{i}}^{\left(s\right)}\right)$ is computed. (B) Schematic representation of EEGNet, implementing the parametric classifier $f$ . Only its main layers are represented, for a more detailed description see Section 2.3.1 and Supplementary Table 1; in particular note that non-linear activation layers have been omitted in this figure, but they are present in the implementation. The input ${X_{i}}^{(s)}$ is processed by the CNN through many layers (with the layer name reported on the left) organized in 3 main blocks (spatio-temporal block: yellow, temporal block: green, fully-connected block: grey) to obtain the output conditional probabilities $p\left(\left.l_{k}\right|{}{X_{i}}^{\left(s\right)}\right),\ k=0,1,2$ . The intermediate output of each layer is reported as a white box with the spatial and temporal dimensions along rows and columns, respectively. Coloured boxes represent the filters of convolutional (blue boxes) and pooling (red boxes) layers.

1 of 11

J. Integr. Neurosci. Print ISSN 0219-6352 Electronic ISSN 1757-448X