Документ взят из кэша поисковой машины. Адрес оригинального документа : http://brain.bio.msu.ru/papers/Shishkin_Ganin_Basul_Zhigalov_Kaplan_2009_JIN__N1_wave_in_the_P300_BCI.pdf Дата изменения: Thu Nov 12 20:12:44 2009 Дата индексирования: Mon Oct 1 20:22:49 2012 Кодировка:

Novemb er 9, 2009

14:31

WSPC/INSTRUCTION FILE

jinkaplan

Journal of Integrative Neuroscience c Imp erial College Press

N1 WAVE IN THE P300 BCI IS NOT SENSITIVE TO THE PHYSICAL CHARACTERISTICS OF STIMULI

SERGEY L. SHISHKIN Faculty of Biology, M.V. Lomonosov Moscow State University, 1/12, Leninskie Gory, Moscow, 119991, Russia sergshishkin@mail.ru http://brain.bio.msu.ru ILYA P. GANIN Faculty of Biology, M.V. Lomonosov Moscow State University, 1/12, Leninskie Gory, Moscow, 119991, Russia ipganin@mail.ru IVAN A. BASYUL Faculty of Biology, M.V. Lomonosov Moscow State University, 1/12, Leninskie Gory, Moscow, 119991, Russia Faculty of Biology, N.I.Lobachevsky State University of Nizhni Novgorod Nizhni Novgorod, Russia basul@inbox.ru ALEXANDER Y. ZHIGALOV Faculty of Biology, M.V. Lomonosov Moscow State University, 1/12, Leninskie Gory, Moscow, 119991, Russia a.zhigalov@mail.ru ALEXANDER Ya. KAPLAN Faculty of Biology, M.V. Lomonosov Moscow State University, 1/12, Leninskie Gory, Moscow, 119991, Russia akaplan@mail.ru Received 29 Octob er 2009 Revised Day Month Year One of the widely used paradigms for the brain-computer interface (BCI), the P300 BCI, was prop osed by Farwell and Donchin as a variation of the classical visual oddball paradigm, known to elicit the P300 comp onent of the brain event-related p otentials (ERP). We show that this paradigm, unlike the standard oddball paradigm, elicit not only the P300 wave but also a strong p osterior N1 wave. Moreover, we present evidence that the sensitivity of this ERP comp onent to targets cannot b e explained by the variations of the p erceived stimuli energy. This evidence is based on comparing the ERP obtained for usual P300 BCI stimuli and for the "inverted" stimulation scheme with low stimulus related variations of light energy (gray letters on the light gray background, "highlighted" by very light darkening). Despite the dramatic difference b etween the stimuli in the standard and "inverted" schemes, no difference b etween N1 amplitudes were found, supp orting the view 1


Novemb er 9, 2009

14:31

WSPC/INSTRUCTION FILE

jinkaplan

2

S.L. Shishkin, I.P. Ganin, I.A. Basyul, A.Y. Zhigalov, A.Y. Kaplan that this comp onent's sensitivity to targets cannot b e based simply on "foveating" the target, but may b e related to spatial attention mechanisms, whose involvement is natural for the P300 BCI. Efforts to optimize the P300 BCI should address b etter use of b oth P300 and N1 waves.

1. Introduction Brain-computer interface (BCI) is a technology providing the user's brain with new output pathways [12, 26], enabling sending messages or controlling various devices without using the muscles. A user is typically trained to change the activity of his/her brain in such a way that the related change in some physiological signal can b e detected by a computer and converted into commands. As such a physiological signal, the electroencephalogram (EEG) is used most often, b ecause it provides real time non-invasive estimation of the brain activity with high temp oral resolution using p ortable equipment, while not requires immobilizing the user's head and/or placing it into a scanner. The main application area of this technology is assisting patients with severe neurodegenerative diseases [3, 27]. In recent years, there has b een also a growing interest in the use of non-invasive BCIs for creating new typ es of computer games [18]. The demands from the enormously wide, diverse and active area of modern game technology may give rise to many new typ es of BCI, which, in turn, would op en new horizons for assistive BCIs, or even for efficiently introducing BCI into completely new areas of application. In addition, BCI may inherit the ability to improve attention regulation from its precursor, the neurofeedback training, and, p ossibly, even make it much more efficient [11]. Most of the existing BCIs, however, require a substantial p eriod of training (from tens of minutes to weeks) b efore the user may start to op erate the BCI more or less efficiently. This prevents using BCI in some emergency situations and also is a serious obstacle for its wide implementation in the world of computer games. On the other hand, the skill of using BCI is usually not b ecoming automatic even after a long training, due to the necessity to use conscious strategies (such as: imagine left or right hand movements; imagine an arrow b eing drawn on a b ow or an arrow shooting up from the b ow; etc.; see, e.g., [17]). When a skill is not automatic, it is associated with extensive and inefficient use of the brain/mind resources [19]. One solution for the automatization problem can b e provided by unconscious op erant conditioning of the brain activity patterns, which we observed in a form of learning to control the intensity of three colors simultaneously [12]; however, learning of this typ e of control was slow, and finding the ways to sp eed up it evidently requires more research. A different solution, surprisingly, has b een provided by one of the oldest typ es of BCI, the so called P300 BCI. Its name was given by the P300 wave, which will b e discussed b elow. This BCI was invited more than 20 years ago [4], but, surprisingly, for many years it attracted little attention. Recently, its efficiency has


Novemb er 9, 2009

14:31

WSPC/INSTRUCTION FILE

jinkaplan

N1 Wave in the P300 BCI Is Not Sensitive to the Physical Characteristics of Stimuli

3

b een recognized, and its p opularity is still growing: for example, the ma jority of BCI-related p osters at the 2009 SPR annual meeting [24] were related to this typ e of BCI; as another example, the two BCI pap ers published so far in Russian journals were b oth related to this paradigm [13, 16]. The p otential of this BCI to provide high sp eed of learning of BCI use and, at the same time, high information transfer rate have b een widely accepted. But it also p osses a feature less often discussed: unlike in most of the other BCIs, little conscious efforts of the user is needed to control this BCI. As complicated mental strategies are not needed to translate the intention of the user into the detectable changes in his/her brain activity, the goal of achieving automatic p erformance with this BCI is more realistic. In the P300 BCI paradigm the user usually watch a matrix (most typically 6x6 or slightly larger size) containing cells with letters, numb ers or other characters (in this case, it is usually used for typing and can b e also called the "P300 sp eller"), or with pictures which are typically related to some commands. The rows and columns of the matrix are highlighted (intensified, flashed) for a short time in a random order. The user attends a given cell and silently counts the numb er of times it flashed or at least attentively note each flash, irresp ectively of was it flashed as a part of a row or as a part of a column. The inventors of the paradigm suggested that the P300 wave will b e elicited in scalp EEG each time a column or row including the attended cell is flashing [4]. This wave is one of the b est known comp onents of the brain event-related p otentials (ERPs). It app ears as a p ositive deflection approximately 300 ms after the onset of the target flash, with the highest amplitude over the centroparietal area. By detecting it, the computer program may find the column and row containing the attended cell, then the cell is identified as their intersection, and the computer execute the command associated with this cell. In the case of P300 sp eller, for example, the letter from the identified target cell is typ ed. The target-related ERP should b e differentiated from the ERP elicited by nontarget flashes. Both typ es of resp onses, moreover, are mixed with the background EEG, acting as a strong "noise" preventing easy detection of the target-related resp onse. To make the detection p ossible, a numb er of resp onses should b e usually collected for each row and column, and a pattern classification algorithm should b e applied. P300 is an automatic resp onse and no training is needed to make it b eing elicited when relevant stimuli are detected. However, some training is imp ortant for the classification algorithm, to adapt it to individual features of the user's brain activity. Thus, b efore starting controlling a computer with BCI, the user should participate in a "calibration" session, where he/she is asked to attend flashes of known cells. In most of the published studies related to this BCI the "calibration" typically took tens of minutes, but in a recent pap er it was shown that this p eriod can b e shorten to 5 minutes [5]. Though mental counting or noting the flashes is a very simple procedure comparing to imaginary manipulations required by the most of other BCIs, the need to


Novemb er 9, 2009

14:31

WSPC/INSTRUCTION FILE

jinkaplan

4

S.L. Shishkin, I.P. Ganin, I.A. Basyul, A.Y. Zhigalov, A.Y. Kaplan

do it for quite a numb er of times b efore the desired letter is typ ed or command is executed (e.g., 15 times for rows plus 15 times for columns in [5]) still may b e an obstacle for fast automatization of the skill of using this BCI. How to reduce significantly the numb er of the brain resp onses needed to b e collected b efore accurate classification b ecomes p ossible? Interestingly, a way to do this seems to b e already suggested in the seminal pap er by Farwell and Donchin: "It may ... b e p ossible to enhance the sp eed of the system by incorp orating additional comp onents of the ERP" ([4], p.591). But so far, to the b est of our knowledge, this advise have b een followed in none of many pap ers of their followers. The only exception is mentioning, by Sellers et al. [23], Hoffmann et al. [8] and Krusienski et al. [14] the fact that the accuracy of the P300 BCI in their work was increased due to a negative comp onent app earing in the p osterior areas prior to P300. This comp onent, similarly to P300, had a much higher amplitude in resp onse to targets comparing to non-targets. However, no systematic study of this comp onent has b een undertaken. A negative comp onent with a similar latency range was analyzed in the P300 BCI paradigm by Allison and Pineda [1], but in their study the electrodes were located only at Fz, Cz, and Pz sites, where only small difference could b e found. Some authors used the p osterior locations without explaining for what reasons they do this (e.g., [5, 25]). Understanding of the nature of the p osterior negative comp onent in the P300 BCI paradigm could not only justify the use of the p osterior locations, but, much more imp ortantly, guide the development of the paradigm in the ways which would help to increase and/or stabilize this comp onent's sensitivity to targets, and also would lead to implementation of signal processing techniques p ossibly able to dramatically increase the p erformance of the P300 BCI by fine tuning to b oth P300 and this comp onent, and by combined use of their features. Sellers et al. [23] and Krusienski et al. [14], as the generators for the negative comp onent they observed seemed to b e located in visual areas, discussed the p ossibility that b oth it and the P300 wave in the P300 BCI may b e modulated not only by attention but, to much extent, also by "foveating the target". If this were the case, the ERP dep endence on targets were resulting from the difference b etween the light energy coming to the fovea from target and non-target flashes. Such an explanation could b e an argument against the P300 BCI in general, as it makes the BCI dep endent on gaze control provided by muscles, and, therefore, it could not help severely paralyzed individuals. Moreover, the dep endence on light energy variations could lead to the dep endency of physical features of the cell content (e.g., it would b e difficult to use pictures with highly different brightness in the same matrix). Although Sellers et al. [23] and Krusienski et al. [14] provided some arguments against "foveating the target" explanation, they have not undertaken any exp erimental test of the p ossible ERP dep endence on the stimulus physical characteristics in the P300 BCI paradigm. It should b e noted that the dep endence of attention-related p osterior negative comp onents (N1) on stimuli luminance has b een rep orted [10], thus, there is a need to check exp erimentally if such dep endence exists sp ecifically in the


Novemb er 9, 2009

14:31

WSPC/INSTRUCTION FILE

jinkaplan

N1 Wave in the P300 BCI Is Not Sensitive to the Physical Characteristics of Stimuli

5

settings of the P300 BCI paradigm. Finding the contribution of the p osterior negative comp onent to P300 BCI [8, 14, 23] seemed to b e an unexp ected event in the history of the P300 BCI development. This is not surprising, as little research has b een so far devoted to the understanding and improvement this BCI paradigm from the psychophysiological p oint of view, probably due to the common b elief that this paradigm is just a simple variation of the "oddball paradigm", very much studied in psychophysiology. This view was prop osed as early as by the inventors of the P300 BCI [4]. More recently, Sellers et al. [22] found the dep endence of the P300 waveforms at Pz electrode location on the stimulus probability (manipulated by changing the matrix size) to b e consistent with the understanding of this BCI as a true oddball paradigm. No attempts have b een made so far to check if there are substantial differences b etween the P300 BCI and the oddball paradigm outside the P300 wave. As the result, in many tens of journal pap ers further development and application of the P300 BCI methodology is based solely on optimizing elicitation and detection of the P300 wave. The oddball paradigm is the most common exp erimental paradigm for eliciting P300. In this paradigm, the sub ject is asked to resp ond to some rare visual or auditory event, called "oddball" or target, or count them mentally, while "standard" or non-target events are presented more often and should not b e attended. P300 is associated with the targets stimuli. All of this corresp ond well to the temp oral organization of stimuli in the P300 BCI paradigm. However, the temp orally organized events in the P300 BCI are also organized in space domain. Non-target events cannot app ear at the same location where a target event occurs, thus, for counting the target flashes it is sufficient only to detect flashes at the target location. Once the target cell is located, which happ ens normally b efore the flashing starts, there is no need to further analyze the content of the attended cell, as no change happ ens there other than flashes going on/off. Thus, to detect target flashes one should use selective visual spatial attention, but not employ the mechanisms for the analysis of semantic or even physical features of the stimuli. In contrast, in a typical visual oddball the targets and non-targets cannot b e differentiated by their locations, as presented at the same p osition, but instead the visual system must analyze details of each stimulus. These differences should lead to several imp ortant consequences. First, as the task of the visual system in this paradigm is relatively simple, one may exp ect good p erformance even with quite a high rate of stimulation, not typical for visual oddball. Indeed, this has b een demonstrated in many studies related to P300 BCI. For instance, Sellers et al. [22] presented flashes each 175 ms or each 350 ms and found even b etter information transfer rate in the case of the shortest intrestimulus interval. Secondly, the visual attention should play imp ortant role in the paradigm, and it is worth to consider the ERP comp onents related to it [7, 15] as a p ossible additional source of information for differentiating targets and non-targets. The


Novemb er 9, 2009

14:31

WSPC/INSTRUCTION FILE

jinkaplan

6

S.L. Shishkin, I.P. Ganin, I.A. Basyul, A.Y. Zhigalov, A.Y. Kaplan

p osterior negative comp onent mentioned ab ove is a candidate for such a role. Finally, as there is no need to analyze any details during stimulation, the processing of the visual information can b e simplified, and, due to this, little dep endence of the ERP on the stimulus physical characteristics will occur. No evidence, however, has b een published so far to supp ort this latter hyp othesis. In this pap er, we test the critical issue of the p ossible dep endence of the early negative comp onent (N1, i.e., the first considerable negative deflection) in the P300 BCI on the physical characteristics of the stimuli. Sp ecifically, we compared the standard stimulation color scheme with a new typ e of the color scheme of the stimuli matrix. This color scheme, in general, was similar to one recently used by Salvaris and Sepulveda [?] but had much lower stimuli contrast. Unlike in the study of these authors, we aimed not on the improvement of the BCI p erformance by finding the color scheme enabling the highest p erformance, but on clarifying one of the imp ortant asp ects in the P300 BCI paradigm's basic mechanism. Therefore, instead analyzing the BCI accuracy we focused on the ERP, mainly on the N1 comp onent. 2. Methods 2.1. Participants Ten healthy volunteers (age 19-23, five males) with normal or corrected to normal vision and without previous exp erience of BCI use participated in the study. They were motivated for participation by the opp ortunity to get an unusual exp erience of free sp elling with the BCI in the end of the exp eriment. The participants were told that the b etter they p erform the tasks, the b etter they and the computer program will b e trained and the more successful "mind sp elling" they would exp erience. Each of them was informed ab out the purp ose and procedure of the study and signed informed consent. 2.2. Procedure The participants sat in upright p osition in an armchair, viewing a standard 17-inch CRT monitor at approximately 90 cm distance from their eyes. Stimuli presentation and EEG recording were organized using BCI2000 system (Schalk et al.,[21]). Several conditions were used. In Standard condition, one block of stimulating/recording was presented. The participants viewed a 6x6 matrix typical for P300 BCI (Fig. 1, left). The matrix consisted of the letters of Russian alphab et, as all the participants were native Russian sp eakers, plus several punctuation marks (not used in the exp eriment as target characters). On the top of the screen, a five letter word was shown, selected randomly from the most frequent nouns of Russian language, on the condition that each letter was presented only once. The letters of this word app eared consequently in the parenthesis after it, one at time. App earing of a new letter in the parenthesis started a new run. Five seconds after the letter app eared, columns and rows started


Novemb er 9, 2009

14:31

WSPC/INSTRUCTION FILE

jinkaplan

N1 Wave in the P300 BCI Is Not Sensitive to the Physical Characteristics of Stimuli

7

flashing in a random order. The participant's task was character matrix, and to count silently how many times was its a part of flashing a row or a column. After flash additional pause of 2.5 sec length, and then the next run

to find this letter in the it flashed, irresp ective of ing finished, there was an started.

Fig. 1. P300 BCI stimuli matrix used in the exp eriment. Left, matrix for the Standard condition. Right, matrix for the Inverted condition. During the stimulation, the rows and columns flashed in a random order. In the left matrix, flashing of the 2nd column is shown. In the right matrix, flashing (darkening, for this typ e of matrix) of the 4th row is shown.

The visual angle at which the matrix was viewed was approximately at 10.5 x 8 , while the size of each character was 1 x 0.9 . The background color was black, or, more exactly, (0,0,0) in the 8-bit RGB model, the default character color was (85,85,85) (dark gray), and flashing was made by changing the character color to (255,255,255) (white). These colors followed those used in the original Farwell and Donchin [4] protocol and were common for the currently typical P300 BCI studies. The duration of this highlighting was 125 ms, while the SOA (stimuli onset asynchrony, i.e., the distance b etween the b eginnings of the stimuli) was 188 ms. Flashing was organized as presenting "flashing cycles" (often referred to in the BCI literature as "sequences of flashes") without pauses b etween them. Each of the flashing cycles consisted of flashing once each row and each column. For each character, the numb er of flashing cycles was set to five. The latter parameter may deserve some notes explaining its relatively low value. Though we optimized most of the parameters of the P300 BCI stimuli, electrode locations and data analysis algorithms based on the results of recently published comparisons b etween various parameter sets [8, 14, 22], and, in general, our BCI configuration was similar to those used in recent studies achieving the highest efficiency (e.g., Guger et al.,[5]), we differently approached the problem of setting the numb er of flashing cycles. Arb el and Donchin [2] found differences b etween the accuracy data obtained in real time and those estimated by an offline and emphasized the insensitivity of the offline analysis to such factors as the difficulty to sustain attention within a long trial. Based on this rep ort and on our exp erience, we prefer to use smaller numb er of flashing cycles than is typically used (e.g., 15 cycles in [5], and some other pap ers). We hop e that, though this may lead to slightly deteriorated accuracy, the negative feedback app earing in online sp elling in the form of errors can b e, due to shorter time interval to which this feedback is related, easier associ-


Novemb er 9, 2009

14:31

WSPC/INSTRUCTION FILE

jinkaplan

8

S.L. Shishkin, I.P. Ganin, I.A. Basyul, A.Y. Zhigalov, A.Y. Kaplan

ated by the user with variations in his/her state negatively influencing the accuracy. Though this idea still is waiting for sufficient supp ort from exp erimental data, there is no reasons not to use the small numb er of flashing cycles at least in the basic studies of the ERP features in the P300 BCI paradigms, as it helps to reduce not only fluctuations of attention which may app ear in long trials, but also the numb er of blinking and other artifacts. The use of five flashing cycles meant presenting 10 target stimuli (intensifying (illuminating) the rows or columns including the attended letter) and 50 non-target stimuli (illuminating other rows or columns) p er run (i.e., p er one letter), i.e., 50 targets and 250 non-targets for the whole block (for one word), i.e., for the whole condition. In Inverted condition, all the parameters were the same except for the colors, which were changed in the same direction as in Salvaris and Sepulveda [20] but with much lower difference b etween the colors (Fig. 1, right). Sp ecifically, we used in this condition (153,153,153) (light gray) for the background, (116,116,116) (gray) for the default character color, and (85,85,85) (dark gray) for the "flashing". Imp ortantly, the flashing not only was made in this condition by darkening instead of the usual highlighting (illuminating), but the intensity of the this color change was very low, only slightly higher than the threshold of detection of the change for the given fast rate of stimuli presentation (the temp oral parameters of the stimuli were the same as in Standard condition). Such low contrast stimuli were used to provide much lower stimulation related variations of the light energy comparing to the Standard condition. The order in which Standard and Inverted conditions were presented was random and counterbalanced across the participants. Each condition was preceded by a practice block of runs with the same setting as the condition, but comprising attending letters from a three letter word instead of a five letter word in the main exp erimental block. Oddball and Slow BCI conditions. The usual P300 BCI stimuli presentation rate (i.e., 5.3 flashes p er second in the Standard and Inverted conditions in our study) is too fast for the visual oddball, where it would b e associated with a too heavy load for the sub ject's vision and may b e associated with an unacceptably low target detection p erformance. The stimuli presentation rate was, therefore, set to 1 stimulus p er second in oddball, sufficient for a stable p erformance, and a Slow P300 BCI condition was introduced with the same stimuli presentation rate, to make the comparison of the ERP waveforms b etween oddball and BCI conditions more justified. The Slow BCI condition could b e also useful for b etter characterizing the comp onents of the ERP, as the non-overlapp ed waveforms can b e obtained. In the Oddball condition, the stimuli were the first six letters of Russian alphab et presented in the center of the black screen. One run consisted of presenting 20 sequences of 6 letters. Within a sequence, each letter app eared only once, and the order was random. No pauses were made b etween the sequences. Each letter was


Novemb er 9, 2009

14:31

WSPC/INSTRUCTION FILE

jinkaplan

N1 Wave in the P300 BCI Is Not Sensitive to the Physical Characteristics of Stimuli

9

shown for 375 ms, with SOA of 1 sec. Before the run, the participant was told which letter (randomly chosen) should b e attended. His/her task was to count silently the numb er of times this letter was presented. Four runs (two in the b eginning of the exp eriment and two in the end) was made for each participant; thus, ERPs to 1 target x 20 sequences x 4 runs = 80 targets and to (6-1) non-targets x 20 sequences x 4 runs = 400 non-targets were collected p er participant. In the Slow BCI condition, most of the parameters were the same as in the Standard BCI condition, except the SOA and the stimulus duration, which were the same as in the Oddball condition, the numb er of flashing cycles set to 2, and the numb er of blocks of runs (i.e., the numb er of words) set to 4 (as for oddball, two in the b eginning of the exp eriment and two in the end). This constituted 2 targets x 2 flashing cycles x 5 runs (letters) x 4 blocks = 80 targets and 10 targets x 2 flashing cycles x 5 runs (letters) x 4 blocks = 400 targets. Therefore, not only the temp oral parameters were equal in Oddball and Slow BCI conditions, but the target/non-target ratio as well (1:5 in b oth conditions), which was imp ortant b ecause the probability of target stimuli is the main factor determining the P300 amplitude in the oddball paradigm. The order in which Oddball and Slow BCI conditions were presented was random. The order was counterbalanced across the participants. Each condition was preceded by a shorter practice run (in Oddball) or block of runs (in Slow BCI), organized in a way similar to Standard and Inverted conditions. In addition, several other BCI related conditions were used for each participant. The description of these conditions and the results obtained with them will b e published elsewhere. 2.3. EEG acquisition EEG was obtained from 14 Ag/AgCl electrodes p ositioned at C3, Cz, C4, TP7, TP8, P7, P3, Pz, P4, P8, PO7, PO8, O1, O2 locations, with electrically joined reference at earlob es and ground at Fpz. These electrodes were fixed with a ribb on net adjusted for each participants individually in such a way that no pressure from it was felled during the whole exp eriment. Vertical EOG was recorded with bip olar electrodes placed ab ove and b elow the left eye, and horizontal EOG was recorded with electrodes on the outer canthi. The EEG and EOG signals were bandpass filtered in the range 0.1-30 Hz with a notch filter at 50 Hz, amplified and sampled at 128 Hz by CONAN hardware. 2.4. ERP analysis Signal processing and data analysis was made using MATLAB 7.1 The EEG and EOG signal was converted into ep ochs, one ep och p er Using EEGLAB, the ep ochs where ±50µV threshold was exceeded in the range -0.275..+0.525 sec relative to the flash onsets where marked (MathWorks). each stimulus. any channel in automatically,


Novemb er 9, 2009

14:31

WSPC/INSTRUCTION FILE

jinkaplan

10

S.L. Shishkin, I.P. Ganin, I.A. Basyul, A.Y. Zhigalov, A.Y. Kaplan

and with subsequent visual screening they were rejected in the case of presence of artifacts; other ep ochs were also screened and were also rejected manually if contained strong EOG or EMG artifacts. Then, the non-ep oched EEG was filtered with 2nd order Butterworth filter in forward and backward direction (thus providing zero phase shift) in 0.5-20 Hz range, ep ochs were extracted again (-0.275..+0.525 s relative to the flash onsets) and those containing no significant artifacts (according to the ab ove describ ed screening of the nonfiltered ep ochs) were averaged for each sub ject and condition, separately for target and non-target flashes. The amplitude of N1 p eak in p osterior locations (P7, P8, PO7, PO8, O1 and O2) was estimated as the minimum value in 125..275 ms time interval relative to the stimulus onset for the difference ERPs (target minus non-target) computed from averaged filtered waveforms. The difference curves were screened visually to check if the N1 p eak indeed corresp onded to this criterion. In three participants, the lower threshold was changed (no more than by 50 ms) to adapt to the individual ERP features while comparing Oddball vs. Slow BCI conditions; no changes were needed while comparing Standard vs. Inverted conditions. Wilcoxon matched pairs test was used for testing statistical hyp otheses. 3. Results As exp ected, ERPs in Oddball and Slow BCI conditions at p osterior locations were very similar in the range of P300 wave (starting approximately from 250 ms), but highly different b efore it (Fig. 2). The detailed analysis of the difference b etween these conditions will b e published elsewhere, but it is worth to mention here that the ERPs in Slow BCI conditions were, in general, similar to ERPs recorded in Standard condition discussed b elow, with the main difference related to app earing a kind of steady state resp onse in the Standard condition, where the high stimulus presentation rate (ca. 5.3 Hz) lead to substantial overlap in the resp onses. The grand averaged difference ERP are shown in Fig. 3. Only PO7 and O1 locations are shown to save space, however, other locations also demonstrated very little or almost no difference b etween the waveforms. Despite the huge difference in contrast and brightness of the stimuli, the size and the direction of brightness change, the shap e and the amplitude of waveforms in Standard and Inverted conditions were very similar. Statistical comparisons were made for the amplitude of N1 comp onent in the difference waveforms (targets minus non-targets) at occipito-temp oral and occipital locations P7, P8, PO7, PO8, O1, O2, in the following pairs of conditions: Oddball vs. Slow BCI, Oddball vs. Standard, Slow BCI vs. Standard, and Standard vs. Inverted. Significant differences were found only when Oddball was compared with the P300 BCI conditions, either Slow BCI or Standard (Fig. 4). In general, N1 amplitude in Oddball condition was much lower than in the P300 BCI conditions, while no difference was found b etween different BCI conditions; among them, Standard and


Novemb er 9, 2009

14:31

WSPC/INSTRUCTION FILE

jinkaplan

N1 Wave in the P300 BCI Is Not Sensitive to the Physical Characteristics of Stimuli

11

Fig. 2. Grand average (over the group of participants, n=10) ERP to targets for two electrode locations. Solid line, Slow BCI condition. Dotted line, Oddball condition. Vertical lines show the onset of stimuli.

Fig. 3. Grand average (over the group of participants, n=10) difference ERP (targets minus nontargets) for two electrode locations. Solid line, Standard condition. Dotted line, 'Inverted' condition. Vertical lines show the onset of stimuli.

Inverted conditions were also not different.

4. Discussion The main finding of this study was the luck of ERP change followed dramatically changed physical characteristics of stimuli in P300 BCI matrix. Not only P300 but also N1 comp onent in the p osterior visual areas was almost identical in grand averaged ERP for two conditions with completely different brightness and contrast, and with different direction and size of change of the brightness in the target area under the process of stimulation. While in Standard condition the "foveated" target


Novemb er 9, 2009

14:31

WSPC/INSTRUCTION FILE

jinkaplan

12

S.L. Shishkin, I.P. Ganin, I.A. Basyul, A.Y. Zhigalov, A.Y. Kaplan

Fig. 4. N1 comp onent group mean amplitudes and SEM (n=10) for difference ERP (targets minus non-targets) in four conditions at p osterior electrode locations P7, P8, PO7, PO8, O1, O2. Wilcoxon mathed pairs test: * p< 0.05, ** p< 0.005.

provided much higher variations of the light energy received by the fovea comparing to non-target p ositions in the matrix, only small variations of the light energy evidently were p ossible in the Inverted condition. Thus, the high similarity b etween the difference (target minus non-targets) waveforms in these conditions excludes the explanation of N1 sensitivity to targets is based on "foveating" the targets. The difference of N1 amplitude, and, more generally, of all the ERP interval b efore P300 b etween the Oddball and BCI conditions were, in contrast, very pronounced. As the "physical" nature of N1 variations has b een ruled out by the "inverted" color scheme test, the most p ossible explanation of this comp onent variations is its relation to spatial attention. This explanation should b e checked more directly in further studies. Our findings highlight the role of spatial factors in the P300 BCI paradigm, which probably should b e b est understood as a "p osition based BCI". A unique


Novemb er 9, 2009

14:31

WSPC/INSTRUCTION FILE

jinkaplan

N1 Wave in the P300 BCI Is Not Sensitive to the Physical Characteristics of Stimuli

13

feature of the P300 BCI is that the user can choose, in one step, from a rather high numb er of commands, e.g., typ e a character selected from a set comprising the whole alphab et plus punctuation marks and digits. This is made p ossible due to attaching the commands to the simultaneously visible cells, whose p ositions are fixed in 2D space relative to each other. The spatial p ositions of the cells are mapp ed into time by flashing known rows and columns at known p oints in time, thus enabling deciphering the attended cell by analyzing temp orally organized brain resp onses to the flashes. Both the high resolution of visuospatial attention and the ERP's high temp oral resolution are nicely used together in the machinery of this BCI. Imp ortantly, during the stimulation the ma jor task of user's visual system is only to maintain the fixation of attention and (though p ossibly less precisely) of the gaze on the selected location, and to detect the flashes at this location. There is no need to analyze in detail the content of the attended cell, as no change happ ens there other than flashes going on/off. Due to this, stable p erformance is p ossible even with a relatively high rate of flashing, not typical for visual oddball. For instance, Sellers et al. [22] presented flashes each 175 ms or each 350 ms and found even b etter information transfer rate in the case of shortest interstimulus intervals. One may hyp othesize that the simplified task for the visual system in the P300 BCI paradigm and the very simple course of ERPs in the time interval b efore P300 (without clearly defined comp onents other than N1, which could b e associated with extensive processing of the details of visual images, as in the visual oddball paradigm) may b e related to each other, but such a hyp othesis clearly need more studies for justification. Though the critical role of spatial attention in the Farwell and Donchin BCI paradigm have b een little addressed in the course of the further development of this paradigm within the last 20 years, several attempts has b een made recently to manipulate the features of stimuli, to ensure b oth well recognizable resp onses to targets and low fatigue and discomfort from the stimulation, at the same time. As we already mentioned, Salvaris and Sepulveda [20] compared BCI sp elling in the "inverted" color scheme, similar to those used in our study but with much higher contrast and brightness variations, to the standard P300 BCI color scheme. They also tested some other stimuli variations. Takano et al. [25] studied green/blue stimulation. Hill et al. [6] prop osed the use of flipping rectangles on which the letters in the matrix were sup erimp osed, so that the flips are used instead of flashing. In all these studies, the new stimulation procedures were found to b e sup erior to the standard stimulation. The fact found in our study, that even a very strong change of stimulation may not lead to changes in ERP, suggest that the BCI develop ers should not b e constrained to relatively small modifications of the original Farwell and Donchin paradigm, but, instead, feel free to try to introduce any reasonable changes. In a search for b etter organization of stimulation in the P300 BCI, Hong et al. [9] prop osed a "motion-onset based" modification of this paradigm in a form of sophisticated paradigm including app earing colored bars b elow each character and


Novemb er 9, 2009

14:31

WSPC/INSTRUCTION FILE

jinkaplan

14

S.L. Shishkin, I.P. Ganin, I.A. Basyul, A.Y. Zhigalov, A.Y. Kaplan

their movement; the participant were asked to mentally name the color of the bar b elow the target character. Interestingly, they found an ERP comp onent they called N200 with location and latency similar to N1 discussed in the current pap er, which also well differentiated targets and non-targets. Though Hong et al. [9] considered this comp onent as motion-sp ecific one, it remains to b e proven, from our p oint of view, that it is not related to the app earing of the stimuli bar, in which case it could b e a manifestation of the same or similar mechanism as what is indexed by the N1 in usual P300 BCI paradigm. In any case, it makes sense to compare these comp onents in the future studies. The high dep endence of the p osterior N1 in the P300 BCI paradigm to targets and its indep endence of the stimuli physical characteristics means that it should b e sp ecifically addressed in BCI research and BCI development. First, efforts should b e made to ensure large and stable target-related N1 variations. Further clarification of the mechanisms underlying this comp onent in the P300 BCI paradigm is imp ortant for this. Secondly, BCI signal processing and pattern recognition techniques should b e adjusted to employ b oth the P300 and N1 waves, to make p ossible for them to b enefit from the combined use of these comp onents which are rather different in latency, locations and other characteristics.

Acknowledgments This study was partly supp orted by the grant 09-04-12094-ofi m of the Russian Foundation of Basic Research.

References
[1] Allison BZ, Pineda JA, ERPs evoked by different matrix sizes: implications for a brain computer interface (BCI) system, IEEE Trans Neural Syst Rehabil Eng 11:110-113, 2003. [2] Arbel Y, Donchin E, Speed accuracy tradeoff of the P300 BCI speller measured in real time, Psychophysiol 45:S107, 2008. [3] Birbaumer N, Cohen LG, Brain-computer interfaces: communication and restoration of movement in paralysis, J Physiol 579:621-636, 2007. [4] Farwell LA, Donchin E, Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials, EEG Clin Neurophysiol 70:510-523, 1988. [5] Guger C, Daban S, Sellers E, Holzner C, Krausz G, Carabalona R, Gramatica F, Edlinger G, How many people are able to control a P300-based brain-computer interface (BCI)? Neurosci Lett 462:94-98, 2009. [6] Hill J, Farquhar J, Martens S, Biessmann F, Schoelkopf B, Effects of stimulus type and of error-correcting code design on BCI speller performance, in: Kol ler D et al. (eds.), Advances in Neural Information Processing Systems 21: Proc. of the 2008 Conference, MIT Press, Cambridge, MA, USA, 2009. [7] Hillyard SA, Vogel EK, Luck SJ, Sensory gain control (amplification) as a mechanism


Novemb er 9, 2009

14:31

WSPC/INSTRUCTION FILE

jinkaplan

N1 Wave in the P300 BCI Is Not Sensitive to the Physical Characteristics of Stimuli

15

[8] [9] [10] [11] [12]

[13]

[14] [15] [16]

[17]

[18]

[19] [20] [21]

[22]

[23]

[24]

of selective attention: electrophysiological and neuroimaging evidence, Philos Trans R Soc Lond B Biol Sci 353:1257-1270, 1998. Hoffmann U, Vesin JM, Ebrahimi T, Diserens K, An efficient P300-based braincomputer interface for disabled sub jects, J Neurosci Meth 167:115-125, 2008. Hong B, Guo F, Liu T, Gao X, Gao S, N200-speller using motion-onset visual response, Clin Neurophysiol 120:1658-1666, 2009. Johannes S, Miinte TF, Heinze HJ, Mangun GR, Luminance and spatial attention effects on early visual processing, Cog Br Res 2:189-205, 1995. Kaplan A, Kildani P, Minikes L, Bandler R, Combining neurofeedback and brain computer interface: New paradigm in psychophysiology, Int J Psychophysiol 69:166, 2008. Kaplan AY, Lim JJ, Jin KS, Park BW, Byeon JG, Tarasova SU, Unconscious operant conditioning in the paradigm of brain-computer interface based on color perception, Int J Neurosci 115:781-802, 2005. Konyshev VA, Karlovskii DV, Mikhailova ES, Slavutskaia AV, Avdeichik VG, Shmelev AS, Shevelev IA, Study of the letter and word recognition by the brain-computerinterface with P300 wave of human visual evoked potential, Ross Fiziol Zh Im I M Sechenova, 93:141-149, 2007. (In Russian) Krusienski DJ, Sellers EW, McFarland DJ, Vaughan TM, Wolpaw JR, Toward enhanced P300 speller performance, J Neurosci Meth 167:15-21, 2008. Mangun GR, Neural mechanisms of visual selective attention, Psychophysiol 32:4-18, 1995. Mikhailova ES, Chicherov VA, Ptushenko EA, Shevelev IA, Spatial gradient of P300 area of the brain visual evoked potential in the brain-computer interface paradigm, Zh Vyssh Nerv Deiat Im I P Pavlova 58:302-308, 2008. (in Russian) Neumann N, Kubler A, Kaiser J, Hinterberger T, Birbaumer N, Conscious perception Ё of brain states: mental strategies for brain-computer communication, Neuropsychologia 41:1028-1036, 2003. (in Russian) Nijholt A, Reuderink B, Bos DO, Turning shortcomings into challenges: brain-computer interfaces for games, in: Nijholt A, Reidsma D, Hondorp H (eds.), INTETAIN 2009, LNICST 9, 2009. Saling LL, Phillips JG, Automatic behaviour: efficient not mindless, Brain Res Bul l 73:1-20, 2007. Salvaris M, Sepulveda F, Visual modifications on the P300 speller BCI paradigm, J Neural Eng 6:046011, 2009. Schalk G, McFarland DJ, Hinterberger T, Birbaumer N, Wolpaw JR, BCI2000: a general-purpose brain-computer interface (BCI) system, IEEE Trans Biomed Eng 51:1034-1043, 2004. Sellers EW, Krusienski DJ, McFarland DJ, Vaughan TM, Wolpaw JR, A P300 eventrelated potential brain-computer interface (BCI): the effects of matrix size and inter stimulus interval on performance, Biol Psychol 73:242-252, 2006. Sellers EW, Krusienski DJ, McFarland DJ, Wolpaw JR, Non-invasive brain-computer interface research at the Wadsworth Center, in: Dornhege et al. (eds.), Towards braincomputer interfacing, MIT Press, 2007. Society for Psychophysiological Research, Abstracts for the 49th Annual Meeting. October 21-24, 2009.


Novemb er 9, 2009

14:31

WSPC/INSTRUCTION FILE

jinkaplan

16

S.L. Shishkin, I.P. Ganin, I.A. Basyul, A.Y. Zhigalov, A.Y. Kaplan

[25] Takano K, Komatsu T, Hata N, Naka jima Y, Kansaku K, Visual stimuli for the P300 brain-computer interface: a comparison of white/gray and green/blue flicker matrices, Clin Neurophysiol 120:1562-1566, 2009. [26] Wolpaw JR, Brain-computer interfaces as new brain output pathways, J Physiol 579:613-619, 2007. [27] Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM, Braincomputer interfaces for communication and control, Clin Neurophysiol 113:767-791, 2002.