Sage Journals: Discover world-class research

Abstract

Our brains rapidly respond to human faces and can differentiate between many identities, retrieving rich semantic emotional-knowledge information. Studies provide a mixed picture of how such information affects event-related potentials (ERPs). We systematically examined the effect of feature-based attention on ERP modulations to briefly presented faces of individuals associated with a crime. The tasks required participants (N = 40 adults) to discriminate the orientation of lines overlaid onto the face, the age of the face, or emotional information associated with the face. Negative faces amplified the N170 ERP component during all tasks, whereas the early posterior negativity (EPN) and late positive potential (LPP) components were increased only when the emotional information was attended to. These findings suggest that during early configural analyses (N170), evaluative information potentiates face processing regardless of feature-based attention. During intermediate, only partially resource-dependent, processing stages (EPN) and late stages of elaborate stimulus processing (LPP), attention to the acquired emotional information is necessary for amplified processing of negatively evaluated faces.

Keywords

affective person knowledge attention task attention to emotional information feature-based attention EEG ERP open data preregistered

Faces constitute a significant part of human communication because they transfer nonverbal signals to other people and transmit rich information about a person, including unique identity information. A growing body of research shows that contextual effects influence how we interpret faces (Wieser & Brosch, 2012), including during instructed threat, in which by merely informing participants that a face will be associated with a shock, persistent threat-related effects are found (Bublatzky et al., 2014). These effects modulate specific event-related potentials (ERPs; see Schellhaas et al., 2020). Such threat signals can likewise be elicited by associating a neutral face with danger because of highly negative biographical knowledge, typically conveyed in various media outlets. Although research has shown that evaluative knowledge modulates ERPs, findings regarding the time course of such effects are conflicting, leading to an ongoing debate on whether early (rather automatic) or late (controlled) stages of processing are modulated (e.g., see Abdel Rahman, 2011; Baum et al., 2020; McCrackin & Itier, 2018).

Early and late ERP components represent distinct stages of face processing (for a review, see Schweinberger & Neumann, 2016). The occipitally scored P1 is known to reflect early stages of stimulus detection and discrimination (e.g., Luck & Hillyard, 1994). The following N170 is viewed as a structural encoding component, and larger amplitudes are reported for faces compared with objects at this component (Eimer, 2011). The subsequent early posterior negativity (EPN) is observed as a differential negativity when contrasting emotional and neutral expressions and has been related to early attentional selection processes (Schupp et al., 2004). The EPN indexes sensitivity to salient emotional information (Junghöfer et al., 2001) and is modulated by emotional expressions partially independent of attentional resources (e.g., see Frühholz et al., 2011; Hammerschmidt et al., 2018; Itier & Neath-Tavares, 2017; Rellecke et al., 2012; Schacht & Sommer, 2009). Finally, the late positive potential (LPP) indicates elaborated stimulus evaluation and controlled attention processes, particularly when the appraisal of affective meaning is involved (e.g., see Hajcak et al., 2009; Schupp et al., 2006).

Evaluative person knowledge has been shown to modulate early and late ERPs. However, experimental tasks and examined ERPs vary considerably, and the resulting findings are conflicting. These experiments included tasks that required responding to an oddball stimulus (Xu et al., 2016), discriminating gender (Luo et al., 2016), recollecting nationality information (Suess et al., 2014), discriminating semantic features (well-known faces vs. new faces, nationality, and names of the individuals pictured; Abdel Rahman, 2011), or responding to evaluative emotional information (Baum et al., 2020; Kissler & Strehlow, 2017). Only one study examined the P1, and it showed no evaluative effects (Luo et al., 2016), whereas there are two conflicting findings regarding the N170 (Luo et al., 2016; Xu et al., 2016). One study used a passive-viewing task and observed no effects (Luo et al., 2016; Xu et al., 2016), whereas the other study required participant to explicitly attend to the gender of faces and found an increased N170 (Luo et al., 2016). It remains an open question whether conflicting findings depend on differences in feature-based attention to faces and emotional information of faces. For later components, larger EPN amplitudes were found for faces paired with negative compared with neutral biographical information (Abdel Rahman, 2011; Luo et al., 2016; Suess et al., 2014; Xu et al., 2016). However, other studies observed no effect of negative evaluative information at the EPN level (Baum et al., 2020; Kissler & Strehlow, 2017; for unfamiliar faces, see Abdel Rahman, 2011). For the LPP, a majority of the studies show larger amplitudes for faces with negative associations (Abdel Rahman, 2011; Baum et al., 2020; Kissler & Strehlow, 2017; Xu et al., 2016; but see Luo et al., 2016). In summary, studies show that evaluative information modulates ERPs but provide a mixed picture of which information-processing stages are affected. Research on emotional expressions showed that feature-based attention to the emotional information is crucial for late modulations, whereas early components are less vulnerable to a competing task (Rellecke et al., 2012; Schindler, Bruchmann, et al., 2020). Whether this holds true for faces associated with evaluative information is unknown.

Statement of Relevance

Humans are highly sensitive to individual faces that have been associated with negative information. In this research, we asked how our brain differentiates individual human faces on the basis of relevant biographic information, such as that provided by newspaper headlines. We gave adult participants the task of responding to faces, some of which we associated with a brutal crime. While they viewed the faces, we recorded electroencephalograms (EEGs). Very early in the viewing period, we found an attention-independent potentiation of brain responses as participants processed the faces of supposed criminals. When participants’ attention was directed to the evaluative background information, subsequent processing of the faces also showed an additional increase in neural activity. These findings suggest a mandatory early prioritization of the processing of negatively evaluated faces, whereas attention to relevant facial features evokes a second stage of potentiated brain responses to faces of individuals charged with a crime. Our findings indicate that negative biographical information that resembles typical newspaper headlines strongly affects the neuronal processing of individual faces.

Because ERPs index distinct stages of processing and these stages depend on available resources, studies in which task conditions are systematically varied might resolve the inconsistent findings during the processing of faces with acquired emotional meaning. In this preregistered study (https://osf.io/erkdt), we examined how feature-based attention modulates evaluative person knowledge for early (P1, N170), midlatency (EPN), and late (LPP) ERPs. Participants were required to (a) perform a perceptual line-discrimination task, in which faces served as distractor stimuli; (b) discriminate the age of faces, in which attention was directed to the face but not to the evaluative information; or (c) decide about the person’s evaluative backstory, thus directing participants’ attention to the emotional information. We expected no P1 effects of evaluative emotional information. For the N170 and EPN, on the basis of findings of partially resource-independent increases for emotional information, we expected increased differentiation between negative and neutral faces when attention was directed to the face and even more when attention was directed to the emotional information. Finally, larger LPP amplitudes for negative compared with neutral faces were expected only in the emotion task.

Method

Participants

Following our preregistered data-sampling plan, we recruited 40 participants. Calculations using G*Power (Version 3.1.7; Faul et al., 2009) showed that this sample size would yield a power of greater than 90% to detect medium-sized effects of evaluative information and interactions of evaluative information and attention tasks (η_p² = .06). In total, 41 participants were recruited at the University of Münster, but one participant had to be excluded because of a neurological disorder. The remaining 40 participants (30 women) were 23.50 years old (SD = 3.31) on average, gave written informed consent, and received €10 per hour for participation. All participants had normal or corrected-to-normal vision, were right-handed, and reported no history of neurological or psychiatric disorders.

Stimuli

The facial stimuli for the experiment were taken from the FACES database with permission for use in the current experiment (Ebner et al., 2010). Specifically, eight identities (four young men, four middle-age men)¹ with neutral expressions were chosen. Half of the faces were associated with highly negative evaluative knowledge and the other half with neutral information (see below). Each half contained two young and two middle-age men. Identities were counterbalanced when they were assigned to the conditions, thus they were equally often associated with negative and neutral information across participants. Moreover, the grouping of identities was balanced across participants; thus, all combinations of identities were equally distributed across experimental conditions. To ensure face validity when coupling the faces with different pieces of narrative information, we displayed only headshots of each face, keeping the facial hair but removing the shirts. The faces were always presented with an overlay of horizontal or vertical thin lines (creating using Presentation software; Version 21.1; Neurobehavioral Systems, 2019). Five horizontal or five vertical lines appeared within the boundaries of each face (horizontal lines: 3.3 degrees of visual angle [DVA]; vertical lines: 4.5 DVA; thickness: 0.03 DVA; centered x = 0, y = 0).

Procedure

Participants first responded to a demographic questionnaire. Then they were seated 60 cm in front of a gamma-corrected display (Iiyama G-Master GB2488HSU) running at 60 Hz with a Michelson contrast of .9979 (minimum L = 0.35 cd/m²; maximum L = 327.43 cd/m²). The background was set to medium gray (red, green, blue [RGB] value = 108, 108, 108). Before participants started the first task, we gave them negative and then neutral background information for each of the eight face identities by presenting newspaper headlines and detailed background explanations. Two young and two middle-age men were portrayed as having committed a brutal crime—raping, mutilating, and killing two young women. The other two young and two middle-age men were described as firefighter trainees. After each newspaper article, all four individual faces were presented on the screen, and participants were requested to attentively look at all faces and memorize them. This information was presented each time before a task started (see Fig. 1).

Fig. 1.

Schematic overview of (a) the experimental flow and (b) trial presentation in all tasks. After electroencephalogram (EEG) preparation was complete (a), the experiment started with one of the three attention tasks, counterbalanced across participants. Each task required them to perform a binary forced-choice decision, in which they had to discriminate line orientation, age, or emotional information. During each trial (b), the backstory was presented in the form of newspaper articles, after which the faces of all four group members were presented. The story and members of the criminal group were presented first, followed by the story and members of the firefighters. Next, participants were informed about the discriminative feature. During each task, the trial structure was identical. Note that stimulus displays in the illustration are not drawn to scale, and the depicted face identity was not used in the experiment but is shown only for display purposes.

Participants were instructed to avoid eye movements and blinks during the stimulus presentation and started with the perceptual task, the age task, or the emotion task (see Fig. 1). Task order and response keys (X and M) were counterbalanced across participants. In each trial, participants had to make a two-alternative forced choice, deciding (a) whether the overlaid line orientation was horizontal or vertical, (b) whether the age of the face was old or young, or (c) whether the associated evaluative emotional information was negative or neutral (i.e., whether the face belonged to the criminal group or the firefighter-training group). Before each task started, the background stories were repeated to ensure that participants would remember the evaluative information. The trial structure and mode of presentation were kept constant across all tasks. Each trial started with the display of a fixation cross for 800 to 1,000 ms, after which a face was presented for 100 ms. This short presentation time of the faces was done to avoid attention shifts to other facial features. The face was followed by another fixation cross presented for 1,500 ms, during which responses were recorded. Each face was repeated 16 times during each task, yielding 64 trials presenting faces associated with neutral information and 64 trials with faces associated with negative information, totaling 384 trials across the three tasks. After testing, participants rated each face on valence, arousal, and perceived threat. Finally, they responded to the Beck Depression Inventory-II and the State-Trait Anxiety Inventory (Hautzinger et al., 2009; Spielberger et al., 1999) as well as to a short version of the NEO Five-Factor Inventory (Körner et al., 2008); these measures were not relevant for the current study but for the analyses of individual differences in effects of evaluative person knowledge in larger samples.

Electroencephalogram (EEG) recording and preprocessing

EEG signals were recorded from 64 active electrodes using Biosemi’s ActiView software (Version 8.12; https://www.biosemi.com/download.htm). Four additional electrodes measured horizontal and vertical eye movements. The recording sampling rate was 512 Hz. Off-line data were re-referenced to the average reference and a low-cutoff filter of 0.01 and a high-cutoff filter of 40 Hz were applied. Recorded eye movements were corrected using the automatic eye-artifact correction method implemented in BESA Research (Version 6.0; BESA, 2014). A trigger delay of 15 ms was measured using a photodiode and corrected during the process of extracting epochs from the EEG time window. Filtered data were segmented from 100 ms before stimulus onset to 1,000 ms after stimulus presentation. The 100 ms before stimulus onset was used for baseline correction.

Data analyses

Data were analyzed using 2 (emotion: negative, neutral) × 3 (task: perceptual, age, emotion) repeated measures analyses of variance. For ERP analyses of the P1, N170, and EPN, channel-group laterality (left, right) was included as a factor. Effect sizes were calculated as η_p²s (Cohen, 1988). When Mauchly’s test of sphericity indicated a violation of sphericity, degrees of freedom were corrected following the Greenhouse-Geisser method. For secondary analyses, hit rates and reaction times were examined, excluding trials with reaction times of less than 100 ms and greater than 1,500 ms.

EEG scalp data were statistically analyzed using Electromagnetic Encephalography Software (EMEGS; Version 2.8; Peyk et al., 2011). The main effects of emotion, task, and their interaction were examined. Time windows were segmented into intervals from 80 to 100 ms for the P1, 120 to 170 ms for the N170, 250 to 350 ms for the EPN, and 400 to 600 ms for the LPP. We measured the P1, N170, and EPN over two symmetrical occipital clusters (P1 and N170: left P9, P7, PO7; right P10, P8, PO8; EPN: left P9, P7, PO7, O1; right P10, P8, PO8, O2). Additionally, we measured the LPP component over a centroparietal cluster (C1, Cz, C2, CP1, CPz, CP2, P1, Pz, P2). We slightly deviated from our preregistration in time (preregistered N170: 130–170 ms; preregistered EPN: 200–350 ms) and space (preregistered EPN: left P9, P7, PO7; right P10, P8, PO8; preregistered LPP: C1, Cz, C2, CP1, CPz, CP2). This was because of the preregistered approach to validate ERP windows for the P1 and N170 by collapsing ERPs across all conditions (Luck & Gaspelin, 2017). For the EPN and LPP, which are typically scored as differences between negative and neutral stimuli, we collapsed waveforms to negative faces and neutral faces across all attention tasks to identify differential effects. Finally, analyses of covariance (ANCOVAs) with reaction time as a covariate were calculated to account for possible influences of reaction time differences on ERP modulations. All data (https://osf.io/rzevn) and a copy of the preregistration (https://osf.io/erkdt) and are available on OSF.

Results

Manipulation check

Face ratings

After the experiment, participants rated the valence, arousal, and perceived threat of faces. Rating showed effects of emotional evaluative information for valence, F(1, 39) = 33.92, p < .001, η_p² = .465 (see Fig. 2); arousal, F(1, 39) = 15.00, p < .001, η_p² = .278; and perceived threat, F(1, 39) = 37.73, p < .001, η_p² = .492. Faces associated with negative evaluative information were rated as more negative (criminal faces: M = 2.88, SD = 0.78; neutral faces: M = 3.99, SD = 0.71), more arousing (criminal faces: M = 3.84, SD = 1.35; neutral faces: M = 2.84, SD = 1.03), and more threatening (criminal faces: M = 4.26, SD = 1.25; neutral faces: M = 2.59, SD = 0.88) than faces paired with neutral information. Finally, participants’ ratings of the credibility of the cover story were rather high (Likert-type scale: 1 = not credible, 4 = highly credible; M = 3.1, SD = 0.55).

Fig. 2.

Ratings and behavioral results in the three attention tasks. Average ratings of the (a) valence, (b) arousal, and (c) perceived threat of the stimuli are shown for each emotion condition. Average (d) reaction time (RT) and (e) hit rate (proportion of correct responses) for each emotion condition are shown separately for the three different tasks. Data bars show group means, and error bars depict 95% confidence intervals. Semitransparent dots depict individual data.

Hit rate

With regard to hit rate, there was no main effect of emotion, F(1, 38) = 1.48, p = .231, η_p² = .037 (see Fig. 2 and Table 1), but a significant effect of task, F(1.45, 55.13) = 27.59, p < .001, η_p² = .421. The percentage of correct choices was lowest in the emotion task compared with both the age and the perceptual tasks (ps < .001). Further, more hits were made in the perceptual task compared with the age task (p = .003). There was no significant interaction between emotion and task, F(2, 76) = 0.81, p = .449, η_p² = .021.

Table 1.

Means for Behavioral Measures Across the Three Attention Tasks

Measure	Perceptual task		Age task		Emotion task
Measure	Criminal faces	Neutral faces	Criminal faces	Neutral faces	Criminal faces	Neutral faces
Number of hits	61.95(1.87)	62.18(1.97)	60.90(3.61)	59.93(5.29)	54.93(11.01)	55.28(8.20)
Reaction time (ms)	581.73(91.86)	581.67(91.65)	634.20(93.18)	631.80(91.50)	722.00(142.96)	786.55(96.95)

Note: Values in parentheses are standard deviations. For four participants, behavioral responses were recoded in at least one condition (emotion task in four participants, age task in one participant) because they reported having confused the response keys. No responses were recorded from one participant in one subcondition of the emotion task, presumably because the participant pressed a wrong key.

Reaction time

Regarding reaction time, main effects of emotion, F(1, 38) = 18.02, p < .001, η_p² = .322 (see Fig. 2 and Table 1), and task, F(2, 76) = 127.86, p < .001, η_p² = .771, were found. These main effects show shorter reaction times for faces associated with negative compared with neutral information and the shortest reaction times during the perceptual task, the age task ranking second and the emotion task eliciting the longest reaction times (all ps < .001). Moreover, there was a significant interaction between emotion and task, F(2, 76) = 16.38, p < .001, η_p² = .301. When we resolved this interaction, it became evident that there were no differences between negative and neutral faces for the comparison of the perceptual and the age tasks (p = .916). However, larger reaction times were found for the emotion task compared with both the age task (p < .001) and perceptual task (p < .001). In the emotion task, faster responses were found toward faces paired with negative information. For descriptive information, see Table 1.

ERP results

P1

With respect to the P1, there was no main effect of emotion, F(1, 39) = 0.16, p = .693, η_p² = .004, but a main effect of task, F(2, 78) = 4.82, p = .011, η_p² = .110. Furthermore, the effect of channel-group laterality did not reach significance, F(1, 39) = 1.85, p = .182, η_p² = .045. Regarding the main effect of task, smaller P1 amplitudes were found in the emotion task compared with the perceptual task (p = .038) and the age task (p = .002); the latter two did not differ significantly in amplitude from each other (p = .674). There was no interaction between emotion and task, F(2, 78) = 1.18, p = .312, η_p² = .029, and further interactions did not reach significance either (Fs < 3.79, ps > .059).

N170

For the N170, there was a large main effect of emotion, F(1, 39) = 16.12, p < .001, η_p² = .292 (see Fig. 3), but no main effect of task, F(2, 78) = 0.81, p = .448, η_p² = .020. Moreover, the main effect of channel-group laterality reached significance, F(1, 39) = 8.50, p = .006, η_p² = .179. With respect to the main effects, faces paired with negative information elicited larger N170 amplitudes than those coupled with neutral information, and larger N170 amplitudes were recorded over the right compared with the left electrode cluster. There was no significant interaction between emotion and task, F(2, 78) = 0.24, p = .784, η_p² = .006, nor did any further interactions reach significance (Fs < 0.54, ps > .584).

Fig. 3.

Main effects of emotion on the P1 and N170, respectively, in each of the three tasks (perceptual, age, and emotion). Scalp topographies depict the differences between event-related potentials (ERPs) in response to faces associated with criminal (negative) and neutral backstories. Black dots indicate positions of electrodes. The top row of ERP waveforms shows the mean time course for highlighted sensors in the negative and neutral conditions. The bottom row of ERP waveforms shows differences between means for the two emotion conditions; error bands show 95% bootstrapped confidence intervals. The gray shaded areas in all waveform graphs highlight the range of the P1 and N170, respectively. The bar plots show average amplitude across selected sensors in each combination of task and emotion condition. Data bars show group means, and error bars depict 95% confidence intervals. Semitransparent dots depict individual data.

EPN

Concerning the EPN, there was no main effect of emotion, F(1, 39) = 2.60, p = .115, η_p² = .063, but the main effect of task reached significance, F(2, 78) = 8.15, p = .001, η_p² = .173. Moreover, no effect of channel-group laterality was identified, F(1, 39) < 0.01, p = .948, η_p² < .001. Smaller EPN amplitudes were recorded in the age task compared with the perceptual task (p = .002) and emotion task (p < .001). The perceptual and emotion tasks did not differ significantly in amplitude (p = .801).

We observed a significant interaction between emotion and task, F(2, 78) = 3.45, p = .037, η_p² = .081 (see Fig. 4). When we resolved this interaction, results showed no significant differences between the perceptual and age tasks for differences between faces paired with negative compared with neutral information (p = .564). In contrast, larger differentiation was found for the emotion task compared with both the age task (p = .027) and perceptual task (p = .039). Moreover, a significant interaction of task and channel-group laterality was found, F(2, 78) = 6.70, p = .002, η_p² = .147. Here, analyses showed that the task differences were stronger over the left electrode cluster, F(2, 78) = 10.50, p < .001, η_p² = .212, than over the right cluster, F(2, 78) = 5.51, p = .006, η_p² = .124. All further interactions remained insignificant (Fs < 3.29, ps > .077).

Fig. 4.

Main effects of emotion on the early posterior negativity (EPN) in each of the three tasks (perceptual, age, and emotion). Scalp topographies depict the differences between event-related potentials (ERPs) in response to faces associated with criminal (negative) and neutral backstories. Black dots indicate positions of electrodes. The top row of ERP waveforms shows the mean time course for highlighted sensors in the negative and neutral conditions. The bottom row of ERP waveforms shows differences between means for the two emotion conditions; error bands show 95% bootstrapped confidence intervals. The gray shaded areas in all waveform graphs highlight the range of the EPN. The bar plot shows average amplitude across selected sensors in each combination of task and emotion condition. Data bars show group means, and error bars depict 95% confidence intervals. Semitransparent dots depict individual data.

LPP

Regarding the LPP, the main effects of emotion, F(1, 39) = 20.34, p < .001, η_p² = .343 (see Fig. 5), and of task reached significance, F(1.70, 66.28) = 6.52, p = .004, η_p² = .143. Faces associated with negative information elicited larger positivities than faces paired with neutral information. Furthermore, smaller LPP amplitudes were found for the perceptual task compared with the age task (p = .048) and emotion task (p = .004), the latter two not differing significantly in amplitude (p = .051). Importantly, we observed the predicted interaction between emotion and task, F(2, 78) = 6.29, p = .003, η_p² = .139 (see Fig. 5). When we resolved this interaction, results showed no significant differences for faces paired with negative and neutral information between the perceptual task and the age task (p = .831). In contrast, a larger differentiation was found for the emotion task compared with both the age task (p = .008) and perceptual task (p = .001).

Fig. 5.

Main effects of evaluative information on the late positive potential (LPP) in each of the three tasks (perceptual, age, and emotion). Scalp topographies depict the differences between event-related potentials (ERPs) in response to faces associated with criminal (negative) and neutral backstories. Black dots indicate positions of electrodes. The top row of ERP waveforms shows the mean time course for highlighted sensors in the negative and neutral conditions. The bottom row of ERP waveforms shows differences between means for the two emotion conditions; error bands show 95% bootstrapped confidence intervals. The gray shaded areas in all waveform graphs highlight the range of the LPP. The bar plot shows average amplitude across selected sensors in each combination of task and emotion condition. Data bars show group means, and error bars depict 95% confidence intervals. Semitransparent dots depict individual data.

Control analyses: ANCOVAs with reaction time as a covariate

Because we observed reaction time differences between the tasks as well as interactions of emotion and task, we calculated ANCOVAs with reaction time as a covariate (see Table 2). For the P1, the main effect of task disappeared when the analysis accounted for reaction time differences; however, N170 and LPP main effects of emotion, EPN and LPP main effects of task, and most importantly, EPN and LPP interaction effects of emotion and task remained significant.

Table 2.

Results From Repeated Measures Analyses on the Four Event-Related Potential Components, Both With and Without Reaction Time (RT) as a Covariate

Component and effect	dfs	With RT		Without RT
Component and effect	dfs	F	p	F	p
P1
Emotion	(1, 39)	0.158	.693	0.513	.478
Task	(2, 78)	4.816	.011	1.050	.337
Emotion × Task	(2, 78)	1.184	.312	1.878	.160
N170
Emotion	(1, 39)	16.120	< .001	15.298	< .001
Task	(2, 78)	0.810	.448	2.090	.150
Emotion × Task	(2, 78)	0.244	.784	0.707	.496
Early posterior negativity
Emotion	(1, 39)	2.601	.115	3.736	.061
Task	(2, 78)	8.152	.001	10.114	.001
Emotion × Task	(2, 78)	3.448	.037	4.455	.015
Late positive potential
Emotion	(1, 39)	20.335	< .001	15.846	< .001
Task	(2, 78)	6.520	< .001	4.050	.029
Emotion × Task	(2, 78)	6.287	< .001	5.690	.008

Note: Significant main and interaction effects are highlighted in boldface. All p values were Greenhouse-Geisser corrected whenever Mauchly’s test indicated a violation of the sphericity assumption.

Discussion

We tested how feature-based attention to perceptual, face, or emotional information differentially influences evaluative-knowledge effects on early (P1, N170), midlatency (EPN), and late (LPP) ERP components. Faces of supposed criminals were rated more negatively, arousing, and threatening than neutral faces. During all tasks, accuracy was at ceiling, whereas for reaction times, faster responses toward faces of criminals were found only during the evaluative task. Interestingly, we found a task-independent effect of negative evaluative knowledge on the N170 as well as interactions of emotion and task for the EPN and LPP, showing differentiation between faces with negative and neutral associations only when participants attended to the associated emotional information.

The P1 is hypothesized to be related to early stimulus detection and discrimination (e.g., Luck & Hillyard, 1994). We expected no significant effects of evaluative emotional information, in line with the few studies examining P1 effects for different kinds of evaluative or context manipulations (e.g., Luo et al., 2016; Wieser et al., 2014). This indicates that the mere evaluative information is not sufficient to increase P1 amplitudes for neutral expressions, in contrast to findings of studies that examined associated monetary gains (Hammerschmidt et al., 2017; but see Hammerschmidt et al., 2018). C1 modulations have been observed even in studies associating monetary information with visual stimuli (Rossi et al., 2017) and studies in which participants were conditioned to associate faces with an electric shock (Rehbein et al., 2014). Finally, we found a smaller P1 during the emotion task, which could indicate differences in tonic vigilance.

The N170, which follows the P1, is viewed as a structural encoding component related to configural processing, and its amplitude is increased for faces compared with objects (Eimer, 2011). Here, we found a task-independent evaluative-information effect with negative biographical information increasing amplitudes for the presented faces. Interestingly, N170 modulations are rarely examined in studies on evaluative person knowledge, but results are conflicting (Luo et al., 2016; Xu et al., 2016). A critical factor in our results could be the small number of face identities. Because we used two groups and presented all group members together, it might have been easy to distinguish between subsequently presented faces and associate all face identities with their evaluative background. This possibly enabled participants to discriminate configural features between putative criminal and neutral identities. The task-independent N170 modulation indicates that this discriminatory process did not need attention to face or emotional features. Building on this empirical finding, we found that the processing of configural information can be influenced by instructed evaluative knowledge, enabling an early sensory differentiation. Adding to this idea, a recent study has shown increased N170 responses for neutral faces for which participants exhibit more negative stereotypes (Giménez-Fernández et al., 2020).

For the following EPN component, which has been examined more frequently, studies have also provided a mixed picture so far (e.g., Baum et al., 2020; Kissler & Strehlow, 2017; Luo et al., 2016; Suess et al., 2014). We expected increasing EPN differences across tasks. However, we observed an effect of negative biographical information only when attention was directed to the biographical background of a given face. Thus, our findings conflict with those of studies showing increased EPN amplitudes for faces associated with negative person knowledge in tasks that do not require attention to the emotional information (Suess et al., 2014; Xu et al., 2016). An explanation for this finding could be related to the short presentation time of the faces, which was done to avoid attention shifts to other facial features. Further explanations for conflicting EPN findings could be related to the specific learning task used to acquire the evaluative biographical information (see the Supplemental Material at https://osf.io/abx27/), the limited number of identities, or the specific attention tasks. It might be that in tasks, which require attending to a semantic face feature (e.g., see Suess et al., 2014), the associated emotional biographical information is more readily retrieved than in other tasks. We believe that during the EPN stage, attentional selection mechanisms depend on both the emotional saliency of the given stimulus and available processing capacities (see Junghöfer et al., 2001; Schindler, Caldarone, et al., 2020). Previous studies using emotional facial expressions (e.g., Rellecke et al., 2012; Schindler, Bruchmann, et al., 2020) have clearly shown that the EPN can be dissociated from both earlier stages (N170) and later stages (LPP; see below). For example, in a similar paradigm, we showed increasing EPN amplitudes with increasing attention to relevant features of fearful faces, showing also an increased EPN during a gender-discrimination task (Schindler, Bruchmann, et al., 2020). In line with this, other studies showed significant EPN modulations by emotional expressions when participants discriminated face identity (Hammerschmidt et al., 2018) and even when participants performed perceptual-discrimination tasks (Itier & Neath-Tavares, 2017; Rellecke et al., 2012). Thus, even though experimental effects on the EPN and LPP were highly similar in the present study, this does not imply that both components have similar functional roles.

In contrast to the intermediate stage associated with the EPN, the LPP indicates stimulus evaluation and controlled attention processes (e.g., see Hajcak et al., 2009; Schupp et al., 2006), and LPP effects are based on various sources, including evaluative, episodic, personal, and biographical information (see Schweinberger & Neumann, 2016). The majority of studies that used evaluative-knowledge designs found late differential effects (Abdel Rahman, 2011; Baum et al., 2020; Kissler & Strehlow, 2017; Klein et al., 2015; Xu et al., 2016). Regarding those studies with no significant LPP modulation (e.g., Luo et al., 2016), our data suggest that the attended feature matters much for a late differentiation, and elaborate differentiation does not occur inevitably but requires attention to the emotional information. These findings are similar to those of studies that manipulated feature-based attention to threatening emotional expressions (Rellecke et al., 2012; Schindler, Bruchmann, et al., 2020) and suggest that at these late processing stages, top-down processes interact with the associated emotional significance of a face more generally.

Constraints on generality and future directions

To the best of our knowledge, this is the first EEG study that systematically manipulated attention to evaluative emotional person knowledge. However, the stimulus presentation was very short, which possibly added to the clarity of our findings, and we used specific tasks to manipulate attention to perceptual, face, or emotional features. Future studies are needed, using other tasks to generalize our findings (e.g., attention to gender, face, or nationality information). Secondly, for cover-story purposes, the faces we used were all male, whereas the sample was predominantly women (n = 30 vs. n = 10 men), and effects could be more pronounced for female participants. In this regard, future studies should also resolve how depicted face identity interacts with evaluative information (e.g., age, gender, ethnic background). Third, other operationalizations of emotional face learning should be examined, including instructed threat or classic conditioning (e.g., Bublatzky et al., 2014; Rehbein et al., 2014; Schellhaas et al., 2020). Lastly, we focused only on negative evaluative information because it is reported to be highly effective (see Bublatzky et al., 2020; Suess et al., 2014). For future studies, it seems promising to include positive evaluative manipulations, for which studies have shown modulations starting with the P1 component (see Hammerschmidt et al., 2017; Schacht & Vrticˇka, 2018).

Conclusion

In summary, early (N170) effects of evaluative emotional knowledge are task independent and increased for arguably criminal identities. In contrast, differential EPN and LPP effects depended on attention to the emotional background information. These findings are vitally important for researchers who conduct ERP studies using evaluative information because they reveal a systematic pattern of emotional sensitivity for competing attention tasks.

Footnotes

We thank Nele Johanna Bögemann for her corrections and all participants who contributed to this study.

Transparency

Action Editor: Karen Rodrigue

Editor: Patricia J. Bauer

Author Contributions

S. Schindler and M. Bruchmann contributed equally to this study. All the authors contributed to the study design. R. Moeck programmed the experiment. S. Schindler,C. Krasowski,and M. Bruchmann pilot-tested the experiment. C. Krasowski tested participants. S. Schindler and M. Bruchmann analyzed the data. S. Schindler and M. Bruchmann wrote the manuscript under the supervision of T. Straube. All the authors approved the final manuscript for submission.

ORCID iD

Sebastian Schindler

References

Abdel Rahman

(2011). Facing good and evil: Early brain signatures of affective biographical knowledge in face recognition. Emotion, 11(6), 1397–1405. https://doi.org/10.1037/a0024717

Baum

Rabovsky

Rose

S. B.

Abdel Rahman

(2020). Clear judgments based on unclear evidence: Person evaluation is strongly influenced by untrustworthy gossip. Emotion, 20(2), 248–260. https://doi.org/10.1037/emo0000545

BESA. (2014). BESA Research (Version 6.0) [Computer software]. https://www.besa.de/products/besa-research/besa-research-overview/

Bublatzky

Gerdes

A. B. M.

Alpers

G. W.

(2014). The persistence of socially instructed threat: Two threat-of-shock studies. Psychophysiology, 51(10), 1005–1014. https://doi.org/10.1111/psyp.12251

Bublatzky

Guerra

Alpers

G. W.

(2020). Watch out, he’s dangerous! Electrocortical indicators of selective visual attention to allegedly threatening persons. Cortex, 131, 164–178. https://doi.org/10.1016/j.cortex.2020.07.009

Cohen

(1988). Statistical power analysis for the behavioral sciences. Routledge.

Ebner

N. C.

Riediger

Lindenberger

(2010). FACES—A database of facial expressions in young, middle-aged, and older women and men: Development and validation. Behavior Research Methods, 42(1), 351–362. https://doi.org/10.3758/BRM.42.1.351

Eimer

(2011). The face-sensitive N170 component of the event-related brain potential. In Rhodes

Calder

Johnson

Haxby

J. V.

(Eds.), The Oxford handbook of face perception (pp. 329–344). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780199559053.013.0017

Faul

Erdfelder

Buchner

Lang

A.-G.

(2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160.

10.

Frühholz

Jellinghaus

Herrmann

(2011). Time course of implicit processing and explicit processing of emotional faces and emotional words. Biological Psychology, 87(2), 265–274. https://doi.org/10.1016/j.biopsycho.2011.03.008

11.

Giménez-Fernández

Kessel

Fernández-Folgueiras

Fondevila

Méndez-Bértolo

Aceves

García-Rubio

M. J.

Carretié

(2020). Prejudice drives exogenous attention to outgroups. Social Cognitive and Affective Neuroscience, 15(6), 615–624. https://doi.org/10.1093/scan/nsaa087

12.

Hajcak

Dunning

J. P.

Foti

(2009). Motivated and controlled attention to emotion: Time-course of the late positive potential. Clinical Neurophysiology, 120(3), 505–510. https://doi.org/10.1016/j.clinph.2008.11.028

13.

Hammerschmidt

Kagan

Kulke

Schacht

(2018). Implicit reward associations impact face processing: Time-resolved evidence from event-related brain potentials and pupil dilations. NeuroImage, 179, 557–569. https://doi.org/10.1016/j.neuroimage.2018.06.055

14.

Hammerschmidt

Sennhenn-Reulen

Schacht

(2017). Associated motivational salience impacts early sensory processing of human faces. NeuroImage, 156, 466–474. https://doi.org/10.1016/j.neuroimage.2017.04.032

15.

Hautzinger

Keller

Kühner

(2009). BDI-II. Beck-Depressions-Inventar. Revision: Vol. 2. Auflage [BDI-II. Beck Depression Inventory Version 2]. Pearson Assessment.

16.

Itier

R. J.

Neath-Tavares

K. N.

(2017). Effects of task demands on the early neural processing of fearful and happy facial expressions. Brain Research, 1663, 38–50. https://doi.org/10.1016/j.brainres.2017.03.013

17.

Junghöfer

Bradley

M. M.

Elbert

T. R.

Lang

P. J.

(2001). Fleeting images: A new look at early emotion discrimination. Psychophysiology, 38, 175–178.

18.

Kissler

Strehlow

(2017). Something always sticks? How emotional language modulates neural processes involved in face encoding and recognition memory. Poznan Studies in Contemporary Linguistics, 53(1), 63–93. https://doi.org/10.1515/psicl-2017-0004

19.

Klein

Iffland

Schindler

Wabnitz

Neuner

(2015). This person is saying bad things about you: The influence of physically and socially threatening context information on the processing of inherently neutral faces. Cognitive, Affective, & Behavioral Neuroscience, 15(4), 736–748. https://doi.org/10.3758/s13415-015-0361-8

20.

Körner

Geyer

Roth

Drapeau

Schmutzer

Albani

Schumann

Brähler

(2008). Persönlichkeitsdiagnostik mit dem NEO-Fünf-Faktoren-Inventar: Die 30-Item-Kurzversion (NEO-FFI-30) [Personality assessment with the NEO-Five-Factor Inventory: The 30-Item-Short-Version (NEO-FFI-30)]. PPmP-Psychotherapie Psychosomatik Medizinische Psychologie, 58(6), 238–245.

21.

Luck

S. J.

Gaspelin

(2017). How to get statistically significant effects in any ERP experiment (and why you shouldn’t). Psychophysiology, 54(1), 146–157.

22.

Luck

S. J.

Hillyard

S. A.

(1994). Electrophysiological correlates of feature analysis during visual search. Psychophysiology, 31(3), 291–308. https://doi.org/10.1111/j.1469-8986.1994.tb02218.x

23.

Luo

Q. L.

Wang

H. L.

Dzhelyova

Huang

(2016). Effect of affective personality information on face processing: Evidence from ERPs. Frontiers in Psychology, 7, Article 810. https://doi.org/10.3389/fpsyg.2016.00810

24.

McCrackin

S. D.

Itier

R. J.

(2018). Is it about me? Time-course of self-relevance and valence effects on the perception of neutral faces with direct and averted gaze. Biological Psychology, 135, 47–64. https://doi.org/10.1016/j.biopsycho.2018.03.003

25.

Neurobehavioral Systems. (2019). Presentation (Version 21.1) [Computer software]. www.neurobs.com

26.

Peyk

De Cesarei

Junghöfer

(2011). ElectroMagnetoEncephalography software: Overview and integration with other EEG/MEG toolboxes. Computational Intelligence and Neuroscience, 2011, Article 861705. https://doi.org/10.1155/2011/861705

27.

Rehbein

M. A.

Steinberg

Wessing

Pastor

M. C.

Zwitserlood

Keuper

Junghöfer

(2014). Rapid plasticity in the prefrontal cortex during affective associative learning. PLOS ONE, 9(10), Article e110720. https://doi.org/10.1371/journal.pone.0110720

28.

Rellecke

Sommer

Schacht

(2012). Does processing of emotional facial expressions depend on intention? Time-resolved evidence from event-related brain potentials. Biological Psychology, 90(1), 23–32.

29.

Rossi

Vanlessen

Bayer

Grass

Pourtois

Schacht

(2017). Motivational salience modulates early visual cortex responses across task sets. Journal of Cognitive Neuroscience, 29(6), 968–979.

30.

Schacht

Sommer

(2009). Emotions in word and face processing: Early and late cortical responses. Brain and Cognition, 69, 538–550.

31.

Schacht

Vrticˇka

(2018). Spatiotemporal pattern of appraising social and emotional relevance: Evidence from event-related brain potentials. Cognitive, Affective, & Behavioral Neuroscience, 18(6), 1172–1187. https://doi.org/10.3758/s13415-018-0629-x

32.

Schellhaas

Arnold

Schmahl

Bublatzky

(2020). Contextual source information modulates neural face processing in the absence of conscious recognition: A threat-of-shock study. Neurobiology of Learning and Memory, 174, Article 107280. https://doi.org/10.1016/j.nlm.2020.107280

33.

Schindler

Bruchmann

Steinweg

A.-L.

Moeck

Straube

(2020). Attentional conditions differentially affect early, intermediate and late neural responses to fearful and neutral faces. Social Cognitive and Affective Neuroscience, 15(7), 765–774. https://doi.org/10.1093/scan/nsaa098

34.

Schindler

Caldarone

Bruchmann

Moeck

Straube

(2020). Time-dependent effects of perceptual load on processing fearful and neutral faces. Neuropsychologia, 146, Article 107529. https://doi.org/10.1016/j.neuropsychologia.2020.107529

35.

Schupp

H. T.

Flaisch

Stockburger

Junghöfer

(2006). Emotion and attention: Event-related brain potential studies. Progress in Brain Research, 156, 31–51. https://doi.org/10.1016/S0079-6123(06)56002-9

36.

Schupp

H. T.

Junghöfer

Weike

A. I.

Hamm

A. O.

(2004). The selective processing of briefly presented affective pictures: An ERP analysis. Psychophysiology, 41, 441–449. https://doi.org/10.1111/j.1469-8986.2004.00174.x

37.

Schweinberger

S. R.

Neumann

M. F.

(2016). Repetition effects in human ERPs to faces. Cortex, 80(Suppl. C), 141–153. https://doi.org/10.1016/j.cortex.2015.11.001

38.

Spielberger

C. D.

Sydeman

S. J.

Owen

A. E.

Marsh

B. J.

(1999). Measuring anxiety and anger with the State-Trait Anxiety Inventory (STAI) and the State-Trait Anger Expression Inventory (STAXI). In Maruish

M. E.

(Ed.), The use of psychological testing for treatment planning and outcomes assessment (2nd ed., pp. 993–1021). Erlbaum.

39.

Suess

Rabovsky

Abdel Rahman

(2014). Perceiving emotions in neutral faces: Expression processing is biased by affective person knowledge. Social Cognitive and Affective Neuroscience, 10(4), 531–536.

40.

Wieser

M. J.

Brosch

(2012). Faces in context: A review and systematization of contextual influences on affective face processing. Frontiers in Psychology, 3, Article 471. https://doi.org/10.3389/fpsyg.2012.00471

41.

Wieser

M. J.

Gerdes

A. B. M.

Büngel

Schwarz

K. A.

Mühlberger

Pauli

(2014). Not so harmless anymore: How context impacts the perception and electrocortical processing of neutral faces. NeuroImage, 92, 74–82. https://doi.org/10.1016/j.neuroimage.2014.01.022

42.

Diao

Fan

Yang

(2016). Contextual valence and sociality jointly influence the early and later stages of neutral face processing. Frontiers in Psychology, 7, Article 1258. https://doi.org/10.3389/fpsyg.2016.01258