# Classification of EEG signal based on pattern recognition for orientation selectivity in attentional blink task

Tanvi Kathuria

National Institute of Technology, Kurukshetra 136119

Dr. Shridharan Devarajan

Center for Neuroscience, Indian Institute for sciences, Bangalore 560012

## Abstract

Attention is the cognitive process of selection of behaviorally relevant information, while ignoring other irrelevant information for better decision making and enhanced processing. But the Brain's attention capacity is limited, which is evident in a phenomenon called Attentional blink. Attentional blink is the transient lapse in attention when judgment is to be made between two temporally discrete target stimulus, within a short period of time, ~500ms. Previous theories based on working memory suggest that working memory load associated with the processing of the first target makes the selection of the second target difficult. Whereas, some distractor based theories suggest that the suppressensorysion or functional inhibition initiated by the distractors or non target stimulus in the visual stream along with the targets interferes with the selection of the second target. Till date, the neural mechanism of attentional blink is an active subject matter of investigation. Literature suggests that due to momentary lapse in attention, the neural processing of sensory information of the second target gets affected, due to which, subjects find it difficult to report the exact features of the second target. In the parent study, it is being investigated, if this loss of information occurs at sensory processing stage or decisional stage. We record EEG signals of subjects doing an RSVP task in a psychophysics experiment, which consists of both blink and non-blink trials, with stimuli differing in their orientations across trials. In this study, we analyze the EEG signals by classifying them based on the orientational information present in them to analyze at which stage the orientational information gets lost.

Keywords: logistic regression, SVM, machine learning, statistical learning

## Abbreviations

Abbreviations
 AB Attentional Blink SVM Support Vector Machine EEG Electroencephalogram PCA Principal Component Analysis RFE Recursive Feature Elimination T1 Target 1 T2 Target 2 RSVP Rapid Serial Visual Presentation cw clockwise ccw counter-clockwise fMRI Functional Magnetic Resonance Imaging

## Background

Every second, we are bombarded with lots of sensory signals present in our environment. We need to process this informations for a meaningful interaction with our surroundings. This includes auditory, visual, olfactory and other sensory data. But at any given point of time, the brain attends only to a small proprtion of sensory data that is relavant to the ongoing process. This cognitive phenomenon of selectively processing information based on its relevance is called Attention. It assists in better decision making and enhanced sensory processing of behaviourally relevant stimuli.

But the attentional resources of the brain are limited. In other words, it can only attend to a certain amount of information coming in through the senses. This is evident in a phenomenon called attentional blink, which reflects the temporal cost of allocating selective attention. Attentional blink refers to the transient loss in attention when judgment is to be made between two temporally discrete target stimulus, within a short period of time, ~500ms. Since brains' attentional resources are limited and the two targets are temporally very close, the brain's occupation with processing the first target causing the second target to be missed.

But the exact neural dynamics of attentional blink are not known yet. Studies suggest that impairments in attention during AB affect the neural processing of sensory information [1] . But the exact stage of processing i.e sensory or decisional (temporal, parietal, occipital or frontal lobe based on the EEG signals from different brain regions) where the information is lost is not known yet. The parent study [2] of this study tries to investigates the same. In that study, a psychophysics experiment is conducted to measure the attentional blink. Subjects perform an RSVP task where they have to focus on 2 target stimuli embedded in a stream of distractor gabbors. Correspondingly, EEG data is recorded.

EEG is a measure of electrical impulses of the brain, generated by the synchronous activity of thousands of neurons. Its excellent time-resolution enables activities even at sub-second timescales to be identified. So the EEG signals related to the second target (which undergoes blink) are analyzed with an aim to identify the effect of blink on the neural signatures of orientation selectivity evoked by this target stimuli. To do this, we build a classifier and train it to classify the EEG signals, based on the orientation of the second target (clockwise, anti-clockwise or no stimulus) in non-blink trials, where we expect the orientation information is intact in the neural signatures evoked by the target. We wish to test the performance of the classifier in blink trials at different stages of the target signal processing (like Frontal, Parietal, Temporal or Occipital area of the brain) to identify the stage of processing where the signal gets affected by blink.

This study focuses on the EEG analysis part of the main AB neural mechanism investigation.

## Problem Statement

To build a classification model, which can classify the EEG signals based on the neural signatures of orientation of target gabor T2 (clockwise or counterclockwise), present in them.

## Objective of Research

To classify the EEG signals based on the orientational information it contain. This is done in three steps :

1. Data augmentation

2. Feature extraction

3. Training and testing classification model

## Scope

The same study can be repeated on new, cleaner data which is in recording stage yet. In this study, dimensionality reduction was not used. So techniques like PCA and RFE can be used for the same.

## LITERATURE REVIEW

AB is widely studied, but still, a long standing debate in the literature. The sources of the limitations on the brain's attention capacity is persistently a challenge to discover and the neural mechanism of this information processing limits are still not fully understood. But it's a wide assumption that these limitations are adaptive as they provide a way for sensory activities to gain full control over motor systems to carry out goal-driven action. While AB is clearly an indicator of temporal limitation of visual selective attention, charecterizing the neural basis of it is still a challenge.

fMRI lacks the temporal resolution to characterize neural activities at 8-12 Hz RSVP rates, which is needed to elicit AB. EEG, even with a better temporal resolution, gives smeared responses to RSVP stream items.

Conventionally EEG signal analysis is done on event related potentials (ERPs) which gives an overall marker for attentional blink for the target related event processing, but fail to provide any insight into how neural responses provide featural information of the stimuli. In this study, we are trying to detect the effect of AB on the feature related information i.e orientational information.

## Logistic regression

Logistic regression is a statistical model that in it's basic form uses a logistic function to model a binary dependent variable. Logistic regression can be extended to multi-class classification using one-vs-all technique. Mathematically, simple logistic regression can be expressed as :

For output class as 0 or 1 :

Hypothesis => g = $\varnothing$*X [Where, X is a vestor of input features and $\varnothing$ is a vector of feature weights]​

Class-probability => h $\varnothing$(X) = sigmoid (g) [ Gives the probability of class 1 ]

Cost-function = > - log (h $\varnothing$(X) ), if y = 1

- log (1-h $\varnothing$(X)) , if y = 0

Sigmoid Function​[3]

## Support Vector Machine (SVM)

A Support Vector Machine (SVM) is a discriminative classifier formally defined by a separating hyperplane. In other words, given labeled training data (supervised learning), the algorithm outputs an optimal hyperplane which categorizes new examples. In two dimentional space this hyperplane is a line dividing a plane in two parts where in each class lay in either side.​[4]

Maximum-margin hyperplane and margins for an SVM trained with samples from two classes. Samples on the margin are called the support vectors. [5]

## Collection of EEG Data

EEG signals are recorded using a 128 channel EEG cap. The subjects are asked to perform a psychophysics task measures the attentional blink using the traditional RSVP stream. The stream has two target gabors (T1 and T2, which differ in their orientation across trials), embedded in a stream of several distractors. The time taken by the distractors between two targets is variable which determines whether the trial is going to be a blink or non-blink trial. We tested on (n =10) subjects and they were required to accurately report the orientation of the target gabors (clockwise/anticlockwise for T1 and clockwise/anticlockwise/No T2 for T2). Attentional blink is predominately observed in trials when the time taken by the distractors between two target stimuli is 180-450 ms. In such trials, the subject finds it difficult to report the orientation of second stimuli accurately. During these experiments, the responses of subjects and the EEG signal from the scalp are recorded.

## Pre-Processing of EEG data

The experiment consists of 504 trials with 228 short lag (time gap between T1 and T2 is < 300ms), 144 medium lag (300ms < time gap <= 600ms), and 72 long lag trials (time gap >= 600ms). Total channels/ electrodes were 128. And for classification perpose, we take EEG data from 300ms before T2 onset to 600ms after T2, with 100ms of T2 data. So this accounts for 1000ms ERP.

Therefore the Data matrix obtained as EEG recording is a 504*128*1000 (trials*electrodes*time-points) matrix. We select only occipital and parietal data (33channels). Since we are sure for existence of orientational information in long and medium trials, initially we choose only T2 present (with cw or ccw orientation) trials of these lags for classification.

We take 3/4th of data for training and 1/4th for testing and perform 4-fold cross validation on the same. So we get 4 matrices out of the Data matrix :

1. x_train_cw (54*33*1000) : 3/4th of data of cw - long and medium trials; occipital and parietal electrodes

2. x_train_ccw (54*33*1000) : 3/4th of data of ccw - long and medium trials; occipital and parietal electrodes

3. x_test_cw (18*33*1000) : 1/4th of data of cw - long and medium trials; occipital and parietal electrodes

4. x_test_ccw (18*33*1000) : 1/4th of data of ccw - long and medium trial; occipital and parietal electrodes

Structure of Data

To increase the number of trials, we perform data augmentation. For that, we randonly sample 5 trials out of all the available trials within a matrix and average it. We repeat this process 3500 times for training matrices and 1500 times for test matrices, generating 3500 training trials and 1500 test trials of each, cw and ccw type. The averaging operation is performed to reduce the noise in the data.

1. gen_x_train_cw (3500*33*1000)

2. gen_x_train_ccw (3500*33*1000)

3. gen_x_test_cw (1500*33*1000)

4. gen_x_test_ccw (1500*33*1000)

Data augmentation by random-sampling and averaging

As the input for one classification model, the differential potential of electrodes in a time window of 20 ms is taken. The (electrode*time-window) matrix serves as input data for a classification model, which is linearized to obtain an input vector of (electrode*time-window, 1) dimension and training being done across trials. This gives us a total of electrode*time-window (660) number of features.

Illustration to show  linearization

A seprate classification model is trained for each time window and the stride value is of 5ms. So intotal we get 197 classification models.

Reshaping the matrices after this linearization operation gives us :

1. X_train_cw (3500*660*197)

2. X_train_ccw (3500*660*197)

3. X_test_cw (1500*660*197)

4. X_test_ccw (1500*660*197)

Now, the two training matrices and 2 test matrices are concatenated along the rows to obtain the following matrices :

1. X_train (7000*660*197)

3. X_test (3000*660*197)

Concatenating test and training matrices respectively.

Correspondingly, Y (7000,1) and Y_test (3000,1) represent the label matrices for the training and test matrices.

## Logistic regression

Logistic regression was implemented on this data using matlab in-built function, mnrfit() and it's prediction (class probabilities) over test data (X_test) were obtained using another matlab in-built function, mnrval() as follows:

Logistic Regession Model implimentation

Probabilities of class 1 were rounded off to obtain class labels from pridicted class probabilities.

## SVM

SVM was implemented on this data using matlab in-built function, fitcsvm() and it's prediction (class labels) over test data (X_test) were obtained using another matlab in-built function, predict() as follows:

SVM model implementation

## Accuracy calculation

The percentage of accurate prediction made by the classifier of each time- window was obtained by comparing the predicted class with actual class and this accuracy was plotted across time-windows to obtain a accuracy v/s time-window plot.

We expected the prediction accuracy to show a hike in certain region (which is expected to have orientational information) as follows :

Expected Accuracy v/s time-window plot

## RESULTS AND DISCUSSION

The prediction by both logistic regression and SVM is at chance (~50% overall) and does not show the expected hike. One of the reasons for the same can be the the nature of distractors, which in itself has a lot of orientation rich information.

​​Distractor used in the experiment​ [2]

Now, the experiment is being redesigned with an orientational neutral distractor. The same EEG analysis can be repeated on the new data to be collected to test the classifiers accuracies.

## CONCLUSION AND RECOMMENDATIONS

The ability of the logistic regression and SVM to classify EEG data based on orientational neural signatures is not known yet. The test for accuracies for the classification model needs to be done with the data from redesigned experiment. Also, dimensionality reduction techniques like PCA and RFE can be used to reduce the feature size.

## ACKNOWLEDGEMENTS

I am really grateful to the science Academie's for giving me this opportunity to work in IISc and to experience how research proceeds. I would like to thank my guide, Dr. Sridharan Devarajan, for his constant guidance and support throughout the project. Also, I would like to thank my mentor, Ms. Swagata Halder, for helping me figure out the various intricacies involved.

#### References

• Matthew F. Tang, Lucy Ford, Ehsan Arabzadeh, James T. Enns, Troy A.W. Visser, Jason B. Mattingley, 2019, Neural dynamics of the attentional blink revealed by encoding orientation selectivity during rapid visual presentation

• Current PhD project, Swagata Halder, Principal Investigator: Dr. Sridharan Devarajan, Center for Neuroscience, IISc Banglore

• https://towardsdatascience.com/derivative-of-the-sigmoid-function-536880cf918e

• https://medium.com/machine-learning-101/chapter-2-svm-support-vector-machine-theory-f0812effc72

• https://en.wikipedia.org/wiki/Support-vector_machine#/media/File:SVM_margin.png

More
Written, reviewed, revised, proofed and published with