Borderline personality disorder (BPD) is a severe psychiatric disorder characterized by a pervasive pattern of affective and behavioral instability (American Psychiatric Association, 2013). However, tracking this instability over time has proven difficult. Researchers have found discrepancies between retrospective self-reports and real-time assessments of mood, symptoms, traits, and behaviors (Fahrenberg et al., 2007; Shiffman et al., 2008). People with BPD more accurately recall their experiences on days without significant mood shifts and struggle to retrospectively recreate days with such shifts (Solhan et al., 2009). Additionally, people with BPD tend to overestimate the intensity of negative moods and underestimate the intensity of positive moods in the context of post-hoc self-reports (Santangelo et al., 2014). People with untreated BPD may also be less accurate at identifying and naming their emotions compared with healthy controls and people with BPD being treated with Dialectical Behavior Therapy (EbnerPriemer et al., 2007; Ebner-Priemer et al., 2008). These difficulties may be particularly exacerbated during periods of heightened psychological distress. Taken together, these findings suggest that assessing mood fluctuations in people with BPD using retrospective self-report may provide different information than ratings captured in vivo.
One sampling technique researchers have used to address these difficulties is ecological momentary assessment (EMA). In EMA designs, participants use a personal device (e.g., cell phone) to report phenomena of interest as they occur in real time. EMA reduces the recall bias associated with retrospective self-report, strengthens ecological validity, and is better equipped to capture affective variability and instability in those with BPD (Shiffman et al., 2008). Ratings of emotional intensity in particular may translate readily to EMA batteries (Santangelo et al., 2014). For example, existing validated questionnaires of emotional experiences (e.g., positive and negative affective schedule (PANAS; Watson et al., 1988) have been abbreviated for frequent real-time administration (e.g., Jacobson et al., 2020). There is also evidence that using single items representing discrete emotions (i.e., indicating the extent to which a person is experiencing sadness or anxiety) is a valid approach to capture affective experiences (Harmon-Jones et al., 2016; Jacobson et al., 2020).
Two studies have assessed affective instability characteristic of BPD using the PANAS in samples of individuals with BPD (Hepp et al., 2020) and individuals with a range of psychopathology (Ringwald et al., 2020). Importantly, these studies also assessed behavioral responses to emotions and interpersonal stressors at each EMA beep and found that more variability in the types of behavioral responses reflected maladaptive emotion dysregulation (Ringwald et al., 2020) and that interpersonal stressors were associated with greater negative affect (Hepp et al., 2020). Although these studies contribute to the relatively sparse literature characterizing affective instability using EMA methods, neither study collected qualitative data regarding how participants responded behaviorally to interpersonal stressors. Furthermore, the studies that have assessed behavioral responses to stressful stimuli using EMA methods among participants with BPD typically only assess engagement in a single maladaptive behavior, such as rumination or suppression (e.g., Chapman et al., 2009; Yaroslavsky et al., 2019).
Thus, it is unclear whether brief measures can accurately capture the full range of coping behaviors used by people with BPD. When measuring emotion regulation, researchers have typically applied one of two approaches: (1) using multi-item scales to assess the use of relatively fewer regulation strategies (e.g., Brockman et al., 2017; Medland et al., 2020) or (2) using single items to capture a broader range of regulatory behaviors (e.g., Heiy & Cheavens, 2014; Southward & Cheavens, 2020; Southward et al., 2019). Unfortunately, multi-item scales cannot account for the full nature of a person's emotion regulation repertoires and single-item scales often demonstrate relatively lower content validity and reliability. In two systematic reviews of studies utilizing EMA methods, many researchers approached measurement development by simply selecting items from cross-sectional measures and adapting the wording to fit the study's timeframe (Griffin et al., 2020; Trull & Ebner-Priemer, 2020). Furthermore, although certain EMA measures of emotion regulation behaviors may demonstrate internal validity and reliability, one assumption underlying all these measures is that participants are interpreting the EMA response choices in line with the researchers' intentions. However, without direct participant feedback describing their behaviors, it is unclear whether the behavioral response participants select from an EMA item to describe their regulatory behavior aligns with that intended by the researchers.
Current Study
In a recent trial examining the impact of a brief treatment on the frequency of maladaptive behavioral coping in people with BPD, Sauer-Zavala et al. (2020) asked participants to categorize their daily emotion regulation behaviors using five relatively broad categories (i.e., emotional avoidance, emotional savoring, engage in impulsive behavior, problem solving, acceptance). Participants also provided qualitative descriptions of each behavior. In the current study, we conducted a secondary data analysis of the agreement between participants and researchers regarding participants' reports and descriptions of their emotion regulation behaviors from Sauer-Zavala et al. (2020), Southward et al. (2020) and Cardona et al. (2020). Specifically, our research team independently categorized participants' qualitative responses as one of five behavioral response options presented to participants during the parent study and examined agreement with participants' own categorizations of their behaviors. We hypothesized that researchers and participants would demonstrate acceptable agreement regarding these classifications. We then explored several potential moderators (i.e., type of emotion experienced, emotion intensity, researcher confidence, severity of BPD symptoms, treatment responder status) that may affect the observed level of agreement. Given the exploratory nature of these moderators, no specific hypotheses were made.
Method
Participants
Participants included eight individuals with BPD (Mage = 21.57, SD = 3.05). The majority of the sample was female (63%), Asian American (63%), non-Hispanic (88%), and not taking psychotropic medication (88%). Individuals were included if they met the following criteria: (1) Diagnostic and statistical manual of mental disorders–5th edition (DSM-5; American Psychiatric Association, 2013) diagnosis of BPD, (2) willingness to maintain their current dose of psychotropic medication throughout the duration of the study, (3) willingness to abstain from additional psychosocial treatment for the duration of the study, (4) fluency in English, and (5) access to a personal smartphone. Participants were excluded if they endorsed conditions that necessitated prioritization of alternative treatment. Specifically, exclusion criteria consisted of the following: (1) current manic episode, schizophrenia, schizoaffective disorder, or other organic mental disorder, (2) current acute suicidal risk, and (3) current or recent (within 3 months) history of substance dependence.
Participants were recruited through local treatment sites, online postings, and direct contact with individuals with BPD who had previously participated in a nontreatment study conducted by the research team. Potential participants completed a phone screen and those likely to be eligible were invited to the research clinic to complete a semistructured, diagnostic assessment to confirm eligibility. Of the 10 individuals that completed this baseline assessment, eight were eligible to participate in this study. All enrolled participants (N = 8) completed all study procedures. Details about study flow and full participant demographics can be viewed in Sauer-Zavala et al. (2020).
Study Design
The primary study from which these data are derived (Sauer-Zavala et al., 2020) utilized a single-case experimental design (Barlow et al., 2009). The study consisted of three phases: baseline, intervention, and follow-up. Specifically, participants were randomly assigned to a baseline period of either 2 or 4 weeks followed by an intervention phase consisting of 4 weekly sessions of the Countering Emotional Behaviors module of the Unified Protocol (Barlow et al., 2018). Finally, participants completed a 4-week assessment-only follow-up phase. Participants completed daily diary entries throughout all study phases, as well as hourly entries on two randomly chosen days (one during the baseline phase and one during the last week of the intervention phase). Notifications were sent via text message or email (based on participant preference) to participants' smartphones to gather information regarding their current emotional experiences, the triggers that prompted these emotions, and their behavioral responses to them. All study procedures were approved by the local university institutional review board and participants provided their informed consent prior to engaging in study-related activities.
Measures
Diagnostic Assessment. The Structured Clinical Interview for DSM-IV Axis II Disorders–Borderline Personality Disorder Module (First et al., 1997) was used to identify the presence/absence of BPD according to DSM-IV-TR criteria, which is identical to DSM-5 criteria, in the current study. The Structured Clinical Interview for DSM-IV Axis II Disorders–Borderline Personality Disorder Module has demonstrated good psychometric properties (Ryder et al., 2007) and strong interrater reliability in prior research (κ = .91; Lobbestael et al., 2011). Additionally, specific modules from the Anxiety Disorders Interview Schedule for DSM-5 (Brown & Barlow, 2014) were used to assess study exclusion criteria. Advanced doctoral students administered these instruments to participants at their initial study visit and demonstrated excellent agreement regarding study eligibility (κ = 1.00; Sauer-Zavala et al., 2020).
BPD Symptom Severity. Participants completed the self-report version of the Zanarini Rating Scale for Borderline Personality Disorder (ZAN-BPD; Zanarini et al., 2015) weekly throughout all study phases. The ZAN-BPD is a nine-item measure designed to assess the severity of each of the DSM-IV-TR (American Psychiatric Association, 2000) criteria for BPD. Participants rate the degree to which they have experienced each symptom in the prior week using a 0 to 4 scale with unique anchors for each item. ZAN-BPD items have demonstrated good internal consistency (Cronbach's α = .84) and 7- to 10-day test–retest reliability (r = .66) in previous research (Zanarini et al., 2015). In the current study, ZAN-BPD items demonstrated acceptable average internal consistency across all weeks (Cronbach's α = .78; range: .56-.91)1 and good 1-week test–retest reliability (r = .67).
Daily Diary and EMA Assessments of Emotional Experiences Daily diary. Each day of the study, participants were prompted to report on the characteristics of their emotional experiences. Participants were first asked to identify which emotion(s), if any, they had experienced since their last diary report from a list of anger, anxiety, guilt/shame, and sadness. For each emotion identified, participants rated its intensity from 1 (no intensity at all) to 5 (greatest possible intensity). Finally, they were asked to select which of five behavioral responses they used in response to each emotion. The response options presented were chosen to broadly capture the range of behavioral responses individuals with BPD may engage in to regulate their emotions. Although each behavioral response was presented in lay English to participants in a way tailored to the emotion after which it was presented, responses represented five general domains: problem solving, acceptance, impulsive responding, emotional avoidance, and emotional savoring. For example, if a participant reported feeling anxious, emotional savoring was described as "dug in to the feeling (e.g., repeated checking, extra preparation for an event, cleaned, sought reassurance)," whereas if they reported feeling sad, emotional savoring was described as "dug in to the feeling (e.g., isolated myself, cried, listened to sad music, watched a sad movie)." Full texts of these response options are available in Southward et al. (2020).
Between each rating, participants were prompted to pro-vide qualitative descriptions of their emotional experiences and responses. After identifying an emotion experienced, participants were asked to describe in their own words what triggered that emotion. Similarly, after rating the intensity of the emotion, but before identifying a behavioral response, participants were asked to describe what they did in response to the emotion. Participants were not limited in how much, or how little, they could write in response to each prompt.
EMA. On two randomly selected days, one during the baseline phase and one during the last week of the intervention phase, participants completed nearly identical reports of their emotional experiences every hour for 12 consecutive hours. Participants were first asked to rate the intensity of their current emotional experience from 0 (no intensity at all) to 100 (greatest intensity). If participants selected 0, the survey ended. If participants indicated an intensity of 1 or higher, they were asked to select which of five emotions best described their current experience: anger, sadness, anxiety, guilt/shame, or joy/happiness. They were then asked to provide qualitative free response descriptions of what triggered this emotion and what they were doing in response to the emotion. Finally, participants were asked to select which of five behavioral categories best characterized their free response description of their behavior, using the same categories tailored to the emotion experienced provided in the daily diaries.