6 Personality and Broadband Self-Report Scales

Key takeaways for this chapter…

  • To complement parent- and teacher-completed rating scales, several popular instruments (e.g., Behavior Assessment System for Children-3; Conners Comprehensive Behavior Rating Scale) offer student-completed (self-report) options
  • It is argued that self-report instruments sometimes detect personal and internalizing problems (e.g., anxiety) missed by informant-completed scales
  • Broadband self-report instruments have historical roots in the Minnesota Multiphasic Personality Inventory (MMPI), a monumentally important psychological assessment tool
  • Some, but not all, self-report scales match the structure of their informant completed counterparts (i.e., scales completed by teachers and parents), which can strengthen inference making
  • Scores and item-level responses on self-report scales can springboard effective follow-up student interviews

Cases/vignettes in this chapter include…

  • Beth Kazan, adding self-report scales to her assessment battery
  • Jennifer, competing explanations
  • Jamal, what to make of his BASC-3 scores
  • Juanita, possible depression
  • Franklin, revisited from Chapter 2
  • Chance, BASC-3 scores
  • Kaspar, ASEBA YSR in Farsi

Perhaps there is head scratching at the sight of a separate chapter devoted entirely to self-report broadband scales. After all, the titles of many self-report scales are identical to the titles of teacher- and parent-completed scales seen in the preceding chapter (e.g., BASC-3, Conners CBRS). Despite familiar titles, self-report scales possess different capabilities and perspectives than their informant-completed counterparts. Some differences derive from an entirely self-evident fact. Thinking is a private experience. The same holds true regarding the texture of emotional life, perceptions of the social world, the nature and flow of thoughts, personal preferences, and predispositions that may never appear in one’s overt actions. Consider for a moment, ratings of anxiety. It’s true that anxiety often includes overt aspects (e.g., reluctance in the face of novelty). But covert aspects are also overwhelmingly common. For example, many adolescents suffer troubling private worries as they confront age-related biological, cognitive and identity-related changes (Dugas, Laugesen & Bukowski, 2012). And such worries may be fully appreciated only by each teen herself. What’s more, informants may miss important aspects of anxiety not only because of its covert nature but also because it is not readily witnessed in  certain  settings. For example, a teen, uniquely, may register strong anxiety felt during peer interactions that transpire with no parent-informants around to see it (Keeley et al., 2018). Equally important, a high school teacher can easily overlook strong overt expressions of anxiety because her class contains too many students to permit close vigilance of each one. Well-devised self-report scales might fill the gaps inherent in “other-completed” scales.

That said, in at least a few practice venues, with a least a few practitioners, the subjective world is apt to get short shrift. Self-report scales (as well as student interviews) may be downplayed or skipped. This is arguably true because school psychology’s dominant theoretical orientation during the last 30 years has been behavioral (see Bergen, 1990), characterized by a preference for behavioral techniques in both assessment (e.g., Kilgus, Kazmerski, Taylor & von der Embse, 2017) and intervention (e.g., Steege, Pratt, Wickerd, Guare & Watson, 2019). As further evidence of a preference for overt behavior, informant-completed rating scales top the list of the most-used assessment instruments, whereas self-completed instruments are far down that same list (Benson, Floyd, Kranzler, Ekert, Fefer & Morgan, 2019). In fact, the top-ranked parent-completed and teacher-completed scales were reportedly used between two and five times more often than self-report scales. In a contemporary world where behaviorism predominates many professional practices, self-report and the internal world may need advocacy. In this chapter it is argued that self-report scales help tap that world and prevent school psychologists from missing important aspects of students’ personality and adjustment.

There is an additional reason for a separate self-report chapter. That is because we need to talk briefly about one monumentally important self-report scale, the Minnesota Multiphasic Personality Inventory (MMPI). You will hear about it twice. Once concerning  self-report scales generally; this is because of its role as a trailblazer. You will hear about it again (as the MMPI’s adolescent version) because of its occasional use in middle school and high schools.

Things to Keep in Mind for All (Most) Self-report Scales and Personality Inventories

There are some general considerations when using self-report scales in schools. These considerations are covered before we turn to individual self-report scales.

Some Youth, Especially the Very Young, Are Poor Self-informants

Research suggests that by age five years children typically demonstrate an understanding of their own psychological characteristics and possess a formulated conception of self (Jia, Lang & Schoppe-Sullivan, 2016). This might imply, on its surface, that any child with a five-year-old’s developmental level would prove a good candidate to rate herself. Not so. In fact, the BASC-3 “Self-report of Personality” and Conners CCBRS self-report version are suited to those only 8-years and older. There seem to be two considerations that might constrain valid self-reporting: general developmental maturity and literacy level. Consider development first. We know as psychologists that there is certainly more to responding to self-report questions than a recognition of one’s own emotions. For example, consider all of the psychological processes needed to respond to the simple true/false question: “I feel sad.”

  1. After this item is read, working memory sufficient to hold the item (and its meaning) online is needed while long-term memory is accessed.
  2. Memory search strategies are invoked to scan prior experiences for occurrences of sadness, perhaps considering their frequency, intensity, and recentness.
  3. Analytical capabilities are tapped to permit a summary judgment about “sad feelings.”
  4. Judgment is applied to consider any off-setting feelings of happiness that might change a sadness rating.
  5. Judgment is used to convert the just-reviewed level of sadness against response options (true/false).
Figure 6.1 Young children may be incapable of producing valid self-ratings

These steps, for just a single item, require cognitive maturity. What’s more, all of these considerations are magnified when many items, not just one, are confronted. No wonder that self-report scales rarely reach down to children younger than age eight years. An immature brain may fail to approach these requirements with the ease of a parent or a teacher. Although a manual-specified age level might be listed as eight years, it seems plausible that children with developmental delays (e.g., low full-scale IQ) might fail to respond validly despite their age. The same is true for delayed language development. Indeed, delayed language development, is known to constrain young children’s emotional understanding and their ability to talk about first-hand emotional experiences (Rieffe & Wiefferink, 2017).

Next consider the issue of literacy. A child has to read fluently enough to complete a 100+ item scale. She also has to possess sufficient reading comprehension to grasp the meaning of each item. Of course, school psychologists routinely encounter students with poor literacy. What’s more, poor literacy is rife among students placed in emotional disorders programs (Hurry, Flouri, & Sylva, 2018), with average reading scores < 25th percentile in one study (Lane, Barton-Arwood, Nelson & Webby, 2008). This would imply that despite their reliance on items with simplified reading requirements, that self-report scales, at least on some occasions, will represent a challenge to read. It’s tempting to opt for simply reading the items for such a student. But for many self-report scales, each student reading items for herself was the method used during standardization. Consequently, the published norms are not applicable when standard administration procedures are violated.

Self-ratings: Advantageous for Internalizing Problems?

It’s intuitive that self-report scales hold certain advantages, even if selective ones, over scales completed by a teacher or parent. The most obvious of these relate to internalizing symptoms. After all, might not a teenager judge the degree of his subjective worry or the intensity of his own unhappiness better than a parent or a teacher? Consequently, shouldn’t a parent or teacher rating prove less valid? Accordingly, shouldn’t self-ratings miss fewer problems (produce fewer false negatives) than ratings coming from a teacher or a parent? The answer appears to be perhaps yes or perhaps no. There are hints from the published literature. For example, Youngstrom, Findling and Calabrese (2004) compared adolescents’ own ratings of symptoms of elevated mood (mania) and depressed mood compared to their parents. “Youth self-report of mania symptoms showed lower correlations with clinician ratings than did parent ratings” (Youngstrom et al, 2004, p. S5). The same was true regarding depressive symptoms. What about tools that school psychologists might use and more common concerns? Let’s look at the of Conners CBRS and anxiety. Table 6.1 indicates the classification accuracy of four Conners CBRS scales concerning their ability to predict membership in an anxiety clinical group compared to control youth. If correct classifications arising from self-report exceed those derived from informant reports, then the first column should have the highest values. This is does not seem to be the case. In fact, overall accuracy arising from self-reports rarely exceeds accuracy produced via informant reports. This is just one summary from one aspect of internalizing problems using just one scale, but it gives rise for caution. The adage that self-reports are uniformly best for internalizing problems may not always prove correct.

Table 6.1 Overall Correct Classification Rate of Clinical Sample with Anxiety Disorder Using Conners CBRS DSM Clinical Scales

Conners CBRS scale (clinical group) Self-report (p. 227) Teacher report (p. 223) Parent report (p. 218)
Generalized anxiety disorder (anxiety) .75 .82 .79
Separation anxiety disorder (anxiety) .67 .74 .71
Social phobia (anxiety) .76 .81 .73
Obsessive-compulsive disorder (anxiety) .72 .86 .74
Adapted from Conners (2010)

Same Title, Different Details

Beth Kazan is an experienced school psychologist just assigned to a high school after three years of elementary-school practice. At her elementary site, Beth was a consistent and confident user of both the BASC-3 Teacher Rating Scale (TRS) and Parent Rating Scale (PRS). She had, however, never touched a BASC-3 Self-Report of Personality (SRP). Beth recently conducted her first high school evaluation. Cognizant of the ability for self-appraisal among high school students lacking among her former elementary students, Beth opted to include in her assessment battery the BASC-3 SRP. But examining her first BASC-3 SPR printout proved shocking. Although she certainly found some well-known composite and clinical scale names (e.g., Internalizing Problems, Hyperactivity), other familiar scores were nowhere to be found (e.g., BSI, Externalizing Problems). More bewildering was what to make of the printout’s group of completely unfamiliar SPR scales. For example, the SRP reports a dimension entitled Sense of Adequacy and another entitled Ego Strength. Perhaps one can intuit the meaning of “Sense of Adequacy” and of “Ego Strength.” And, perhaps, those intuitions would prove to be correct. But guessing at the nature of a score’s meaning represents a poor way to practice. This means that novel users of self-report scales, like Beth, need to do due diligence and pre-acquaint themselves with their tools. You are advised against assuming that a BASC-3, is a BASC-3, is a BASC-3. Chagrined, Beth eventually grasp this reality as she opened and commenced  reading the BASC-3 manual.

Personality Inventories Same as, and Different from, Self-report Behavior Rating Scales

Self-report measures of personality are sometimes called personality inventories, which are common in adult practice settings. This title is applied because these tools can take an inventory of various aspects of personality via informants’ self-rating. These inventories, in turn, are applied to diverse aspects of practice. This is exemplified by adult-specific instruments such as the Revised NEO Personality Inventory (Costa & McCrae, 1992). This tool is designed to measure five factor-analytically-derived personality traits, the so-called “Big Five.” These are as follows:

  1. Openness to experience
  2. Conscientiousness
  3. Extraversion
  4. Agreeableness
  5. Neuroticism

The five traits then can be used in research and practice. For example, marriage satisfaction across life stages can be examined by looking at marriage partners’ similar versus complementary personality structures as characterized by the Big Five (Shiota & Levenson, 2017). Similarly, the Big Five personality traits are found to be differentially related to vocational interests, such as in science, technology, engineering and math–STEM (Perera & McIlveen, 2018). Closer to the topic of this book, the Big Five also predicts adult psychopathology. For example, in a study of 4,927 adults (college students) using the Revised NEO Personality Inventory it was found that “some predicted associations between self-evaluation and psychopathology were not found, and unanticipated associations emerged” (Durrett & Trull, 2005, p. 359). Although marriage and family therapist may employ them, and the same for guidance counselors aiding their clients in making career choices, the Big Five are generally of secondary importance for clinical diagnostic purposes. It turns out that personality inventories tapping the Big Five (including the 16 PF, see Cattell & Schuerger, 2003) look a lot like self-report behavioral rating scales that you will soon hear about (e.g., they have multiple dimensions, are norm-referenced). But they are primarily interested in generic personality, not clinical diagnosis, such as might be used in career and vocational planning. Thus, these instruments use nomothetic elements (norm referencing) but they are also idiographic. They depart from many purposes of self-report rating scales, such as assistance in reaching categorical, not dimensional, conclusions. Once again, assessment tools need to match the purpose at hand, as is often reflected in the referral question.

Contextualizing Self-report Tools: Start with the MMPI

Many of the features of self-report scales can be traced to a singular innovative endeavor. This was the creation and refinement of the original Minnesota Multiphasic Personality Inventory (MMPI). Note the use of the term personality inventory, even though the MMPI seems nearly identical to now-common self-report rating scales. A portion of the MMPI’s importance is foundational and trail blazing. Another portion is exemplified by revealing just what self-report scales can do with sufficient refinement. Yet another portion concerns watershed research conducted on the MMPI that exposed clinicians’ limitations of human judgment. In light of these reasons, a brief digression into the general version (adult) MMPI is in order before we turn to consideration of individual tests suited for school use, including the adolescent version(s) of the MMPI.

Way back in 1937, a psychologist, Starke Hathaway, and a psychiatrist, Charnley McKinley, at the University of Minnesota, formulated an instrument that became the MMPI. They described the first MMPI as “an objective aid in the routine psychiatric case work-up of adult patients and as a method of determining the severity of the conditions” (Dahlstrom, 1972, p. 4). The authors devised a vast array of one-line questions that adults read and confirmed or disconfirmed (i.e. answered “yes” or “no”). You can already see similarities to contemporary instruments. The MMPI’s items covered multiple dimensions of potential psychopathology, in someways paralleling a detailed diagnostic interview. But item wording was often subtle, avoiding the transparency that might allow respondents to discern the authors’ purpose and thus outsmart the test. The MMPI soon achieved widespread adoption during World War II (Hathaway & McKinley, 1942), a period short of trained psychologists and psychiatrists. As an alternative to time-consuming and professionally-taxing diagnostic interviews, it helped to usher in the era of objective assessment techniques comprised of many individual items (i.e., personality inventories). The early MMPI foreshadowed the objectivity of today’s tools: every administration consisted of identical items, all respondents were provided identical directions, raw scores were always generated in the same manner, plus derived scores (in this case T-scores) were invariably calculated and plotted. Standardization, in turn, facilitated quantification, soon followed by vast empirical efforts at scale interpretation.

Professional judgment, rather than psychometric tools and algorithms, predominated before the routine use of tools like the MMPI. Photo by Josh Rocklage on Unsplash.

Seminal MMPI research, conducted during an era when clinicians’ judgment was often viewed as sacrosanct, included one particularly eye-opening finding. It concerned the radical notion that mere scores (treated in algorithmic manner) are more accurate than clinicians’ non-numeric, expert judgments. The classic work of University of Minnesota professor Paul Meehl (1954), Clinical versus statistical prediction: A theoretical analysis and review of the evidence, proved to be a pivotal publication. This is because it documented this very finding—cold, hard numbers typically outperform the conclusions of trained mental health specialists. In other words, for diagnostic purposes, nomothetic procedures nearly universally outperform idiographic procedures. (Incidentally, an meta-analysis conducted 50 years after Meehl’s original work largely confirmed his original conclusion; Ægisdottir et al., 2006).

But what might an interpretative algorithm look like and why should we care about it today? In 1956, Paul Meehl published yet another important (and provocative) article. This one was entitled, oddly, “Wanted-A good cookbook.” Echoing his earlier finding that clinicians’ descriptions of clients’ characteristics routinely over-valued their own insights and conclusions, Meehl proposed his radical alternative, which was dubbed the “cookbook method.” This method was envisioned to assume various forms depending on the availability of research associated with an instrument’s cookbook. “Whatever its form, the essential point is that the transition from psychometric pattern to personality description is an automatic, mechanical, ‘clerical’ kind of task, proceeding by the use of explicit rules set forth in the cookbook.”

For the MMPI, the cookbook would revolve around a fixed interpretative process using a “two-point code.” Thus, various MMPI profiles would not be interpreted subjectively based on overall configuration or magnitude of several scale elevations. Rather, the basic 10 MMPI scales would always be examined and if two scales were elevated, then those two elevations (regardless of the order and magnitude) would be listed to anchor interpretation. For example, if an MMPI resulted in the high scales being scale 1 (Hypochondriasis, with a T-score of 74) and scale 3 (Depression, with a T-score of 77), then a 1-3 code would be documented.

According to his approach, the clinicians need not surmise what the admixture of hypochondriacal characteristics and depressive symptoms might portend for their case. Instead—and this is the key point—they next consult a code book that list empirically-established characteristics associated with a 1-3 code. Practitioners skip (or minimize) clinical judgment; they merely consult the empirical correlates of each case’s objective code type. The upshot of all of these factors helped to make the MMPI wildly popular among psychologists. In fact, it has now been translated into more than 50 languages. Over the past 75 years no topic in psychology has been the subject of more publications than the MMPI. For example, in 2017 alone the number of peer-reviewed, English language publications devoted to the MMPI was at least 88! For context, this one-year total is far more than can be found  for the BASC-3 (and BASC-3) over an entire decade.

You can probably envision the importance of a methodology like a user-friendly and quickly-determined two-point code. You can also probably envision push back from clinicians. Meehl, more than 60 years ago, saw this coming: “I am quite aware that the mere prospect of such a method will horrify some of you [speaking to his clinical psychology colleagues]; in my weaker moments it horrifies me” (Meehl, 1956, p. 264). Because Meehl’s “statistical prediction” and his cookbook proposal is all about rules and numbers, not clinical wisdom. You can sense that he was arguing for something akin to the Bayesian approach (probability nomogram) you saw in Chapter 2. And, of course, he was a champion of the nomothetic approach and harsh detractor of the idiographic approach (at least for most referral concerns).

There is more to the historical legacy of the MMPI; this concerns validity scales. Even for school psychologists who never touch an MMPI-A, its validity scales are still worth contemplating. This is the genesis of most validity scales you see in your current practice (e.g., as found on the BASC-3 or Conners CBRS). In the 1940s, validity scales were a novel feature and an insightful one. Adults seen in clinical settings sometimes wanted to make themselves look favorably. Perhaps a family member was advocating for unwanted treatment or (especially in the old days) institutionalization. Consequently, a set of MMPI items that could flag flagrant problem minimization might alert clinicians to a potentially invalid profile. This effort produced the L (Lie) scale. A related validity scale, the K scale, also concerned defensiveness. The K scale, however, was used to make statistical corrections to certain clinical scales to account for defensiveness. The K scale’s application allowed for fewer false negative profile interpretations (i.e., instances in which a genuine problem was missed). Research shows, however, that K-corrections do not enhance classification decisions with youth on the MMPI-A (Alperin, Archer & Coates, 1995), even though they may work with adults.

Of course, respondents may not only wish to present themselves positively, sometimes they sought the opposite. In other words, they attempted to feign severity. This might concern, for example, someone seeking a Workmen’s Compensation claim after a trauma who would like to appear as impaired as possible. It might also be someone attempting to avoid conscription in the Armed Forces (a real consideration for baby boomers during the Vietnam War). Accordingly, a scale comprised of items that are rarely confirmed by individuals was created (F scale). Critically, these items are infrequently endorsed even among adults with true psychopathology (cleverly, however, some of the F scale items possessed face validity suggesting psychopathology). Table 6.2 provides more information.

Table 6.2 MMPI-A Validity Scales

Scale Abbreviation Scale Name How derived Interpretation
? (CNS) Cannot Say
VRIN Variable Response Inconsistency Inconsistent responding to items with similar content Inconsistent responding
TRIN True Response Inconsistency
F Infrequency Endorsement of infrequently-endorsed items during entireinventory Random responding
F1 Infrequency 1 Endorsement of infrequently-endorsed items during 1st half of inventory Random responding
F2 Infrequency 2 Endorsement of infrequently-endorsed items during 2nd half of inventory Random responding
L Lie Endorsement of items with overly positive content Purposeful attempt to deny common human shortcomings and to present oneself in an overly positive way
K Correction Endorsement of items with overly positive content Statistical correction to certain clinical scales to account for defensiveness

Some Self-report Scales and Personality Inventories

Minnesota Multiphasic Personality Inventory-Adolescent (MMPI-A)

All of the popularity enjoyed by the MMPI in the adult clinical world notwithstanding, it is a much more modest player in the adolescent clinical world, and apparently even less when the venue is a school campus. School psychologists’ place it far down a list of potentially-used self-report scales (Benson et al., 2019). The MMPI-A was created using items from the original MMPI and the updated MMPI-2 in 1992. It was devised expressly for use with adolescents. The MMPI-A comprises 478 true/false items (Butcher et al., 1992) with an updated manual (Butcher et al., 2006) appearing some years ago. A new MMPI-RF (revised form) has been published based on the MMPI-A (Archer, Handel, Ben-Porath & Tellegen, 2016), although the prior MMPI-A remains in use and is indeed marketed concurrently with the newer MMPI-A-RF. The MMPI-A-RF is discussed a few paragraphs below. The MMPI-A is suited for those 14 to 18 years of age with reading levels of mid-to-high fourth grade; it is supported by norms derived from a national sample of approximately 1,600 teens.

MMPI-A Organization and Characteristics

The MMPI-A uses the 10 traditional clinical scales and associated single-digit numbers to indicate them. These are:

  • 1 Hs – Hypochondriasis
  • 2 D – Depression
  • 3 Hy – Hysteria
  • 4 Pd – Psychopathic Deviate
  • 5 Mf – Masculinity/Femininity
  • 6 Pa – Paranoia
  • 7 Pt  – Psychasthenia
  • 8 Sc – Schizophrenia
  • 9 Ma – Hypomania
  • 0 Si  – Social Introversion

Many of these terms sound archaic to our 21st century ears. Words like hysteria and psychasthenia have largely vanished from today’s lexicon (although terms like depression and paranoia persist). Scale names, however, may matter relatively little in MMPI interpretation. This is because there is a MMPI-A-specific system used to aid interpretation that often frees diagnosticians from the need to make inferences based on individual scales, as you saw with use of the two-point code earlier.

Table 6.3 MMPI-A Scales

Type of scale Number of dimensions Examples
Validity indicators 8 Variable Response Inconsistency,

Lie

Clinical scales 10 Depression, psychopathic deviate
Clinical subscales 30 Brooding, authority problems
Content scales 25 Anger, school problems
Content component scales 31 Suicide ideation, low motivation
Supplementary scales 6 Alcohol/drug proneness, immaturity
Personality psychopathology scales 5 Aggression, psychoticism

MMPI-A: Exemplifying Specialized Interpretation

Fortunately, use of the MMPI-A is supported by a text for psychologists, which comprises 459 pages (Archer, 2017). The usefulness of this vast MMPI information storehouse is exemplified by the case of 15-year-old Jennifer, a student referred for plummeting report card grades and waning school interest. Indeed, review of her cumulative file suggested two semesters of falling marks as well as mounting conflict with siblings. More troubling was a report of two episodes of loud disagreement coupled with agitation during second period English class. There was no evidence of health problems, and an interview with her parents contraindicated the presence of changes in family/living circumstances or noteworthy environmental stressors. With background information like this, the school psychologist’s preliminary hypotheses concerned (1.) a mood problem, perhaps involving rapid mood swings, (2.) emerging anti-social tendencies or (3.) perhaps the occurrence of trauma not detected by simply interviewing Jennifer’s parents. The school psychologist estimated that the base rates for the first and third hypotheses were fairly high in her practice setting (a middle-class high school where school psychologists embrace a mental health role). In contrast, she estimated the base rate for the second hypothesis as rare, but not unheard of, among adolescents seen at her site. Before conducting an interview, the school psychologist had Jennifer complete an MMPI-A. A 2-0 code type resulted. Relying on the two-point-code rubric, she need not surmise what scale 2 (depression) actually tapped or just what was implied by an elevation corresponding to the one Jennifer was expressing on this scale. Similarly, she did not need to conjecture about her scale 0 (social introversion) elevation and try to amalgamate elevations on scales 2 and 0 into a portrait of Jennifer’s social-emotional status. Even more important, there was no need to attend to and synthesize several other clinical elevations found in Jennifer’s MMPI-A profile. Instead, she wrote down the 2-0 code and turned to a published codebook. Consider the bullet points associated with this code:

  • quite uncommon in clinic settings (occurs among 1.8% of males; 3.8% of females)
  • often referred for shyness, lethargy, interpersonal sensitivity, symptoms of depression, anxiety, withdrawal, social introversion and feelings of inferiority
  • conforming, passive
  • very unlikely to engage in antisocial behavior
  • unlikely to use drugs or drink alcohol
  • most are compliant
  • there may be an elevated risk of bulimia or anorexia
  • assertiveness training and cognitive behavior therapy may help to decrease depressive thoughts

Although stand-alone MMPI use is ill advised, it nonetheless remains clear that much might have been learned about Jennifer by simply considering her two-point code. Speculation about a mood problem seems to have been substantially strengthened, whereas hypotheses about anti-social behavior and trauma seem to have been weakened. There is obviously more assessment work to be done, but Jennifer is arguably much better understood by virtue of the bullet point information listed above.

Reliance on two-point codes presupposes that adolescents designated by each two-point code share important characteristics (i.e., they are at least somewhat homogeneous regarding personality characteristics and mental health problems). Crucially, it also depends on research correlating code types with clinicians’ descriptions. Use of two-point codes also fuels applied research on success during treatment, gold standard diagnoses, common comorbidities, and the risk of various life events. Research suggests that relying on codes and concomitantly diminishing the importance of clinicians’ judgment fosters accuracy. This is Paul Meehl’s famous argument from nearly 70 years ago.

In part enhanced accuracy occurs because clinicians, like all humans, can be swayed by  transient factors, as well as longstanding biases, unavailable to conscious awareness (Kahnemann, 2011). And associated reliance on the Automatic System can stifle good judgment. Influences as trivial as time of day, level of comfort in the office where one is working, and even insignificant aspects of a client’s presentation (whether he selected a blue shirt, which the clinician likes, rather than a gray shirt, which the clinician detests). For clinicians working in mental health settings, where caseloads may consist of one adolescent after another, it’s easy to envision frequent use of the MMPI-A. In a practice like this, unique aspects of the MMPI-A and familiarity with various code types are possible. But for the infrequent user (e.g., a school psychologist who partitions her time among elementary and high school venues or one for whom there is limited mental health involvement), effective use of the MMPI-A may prove quite difficult.

MMPI-A Supplemental Scales

The adult MMPI offers many research and practice-relevant supplemental scales, an important advent. The same is true of the MMPI-A. The formulation of validity scales is just one of the MMPI’s historically important contributions. Introduction of content and supplemental scales is another. Here’s the logic. The initial MMPI item pool might be reconfigured to formulate alternative, novel scales whose items tapped considerations of interest to the clinicians. Indeed, this very proposal has garnered progressive popularity. Table 6.3, for example, indicates that at least 87 such scales exist for diagnosticians who work with teens. Many of these, obviously, might be useful for school psychologists seeking to confirm or disconfirm hypotheses (or even for formulating hypotheses when no obvious ones are yet apparent). For example, a scale called Alcohol/drug Problem Proneness (PRO) and another one called Low Achievement Orientation (A-las-1) might help clarify students’ problems. The same is true for two other scales, Immaturity (IMM) and Adolescent-low self-esteem (A-lse), which turn out to afford value in determination of high school students with emotional disturbance (as seen a few paragraphs below). Incidentally, as a review of the BASC-3 and Conners CBRS manuals will confirm, these instruments also use this methodology (i.e., creating novel supplemental scales via reconfiguring existing items).

MMPI-A and Classification Validity

It may now be obvious that complex algorithms are possible because of the MMPI-A’s objective scores and its many clinical, content, and supplemental scales. In fact, inventive researchers have demonstrated the capability of MMPI-A scores to predict an interesting array of outcomes. In one example, researchers concerned with predicting delinquency (Morton, Farris & Brenowitz, 2002) examined several scales working in concert. Theory were blended with straightforward empiricism. Excitatory tendencies (as measured by the psychopathic deviation and schizophrenia scales) contrasted with inhibitory tendencies (as measured by masculinity/femininity and social introversion) were envisioned to predict tendencies toward delinquency. This was confirmed. The score best combinations produced a DLRpositive of 3.5 and a DLRnegative of .28. Moreover, the rate of correct prediction remained as strong with cross-validation. In a second study of delinquency, researchers (Edner, Glaser, Calhoun, Dukes-Murray, Khan & Donaldson, 2020) used a specially crafted MMPI-A scale entitled the Trauma Scale for Juvenile Offenders (TSJO). The goal was to distinguish adjudicated youth with a history of trauma from those lacking such a history. High levels of TSJO endorsement resulted in a  DLRpositive of 1.83; low levels with a DLRnegative of .59. It’s not these values per se that matter but what else the researchers included in their article. Perhaps foreshadowing a happy future were diagnostic utility statistics abound, the authors proved vast information. This included: sensitivity, specificity, total accuracy, positive predictive value, negative predictive value, positive DLR, negative DLR, indication of statistical significance, effect size, all calibrated relative to changing cut-points.

Turning to more practical examples of classification validity, consider the research conducted in Texas (Miller, Li & Kabell, 2015) a state where school psychologists’ gatekeeping duties concern emotional disturbance (ED) but not specific learning disability (an exceptionality managed largely by educational diagnosticians). These researchers addressed, in part, the MMPI-A’s ability to distinguish students with emotional disturbance from controls as well as another group of children who were experiencing a current psychiatric hospitalization or treatment in a residential treatment center. Favorably, the researchers calculated diagnostic utility statistics—just the kind of analysis needed to inform classification decisions. As seen in Table 6.4, MMPI-A scores were predictive of ED status (e.g., three scales are associated with increased risk of ED), although effect sizes were unimpressive. Perhaps there will be more practice-friendly studies like this in the future.

Table 6.4 Effect Sizes and DLR Values for MMPI-A Scales Concerning Emotional Disturbance Identification

Scale Effect size associated with positive score Diagnostic Likelihood Ratio Positive  Effect size associated with negative score Diagnostic Likelihood Ratio Negative 
Hypochondriasis (Hs) Minimal 1.58 Minimal .75
Schizophrenia (Sc) Small 2.17 Minimal .58
Hysteria (Hy) Small 2.17 Minimal .78
Adolescent-health concerns (A-hea) No effect 0.91 No effect 1.05
Adolescent-low self-esteem (A-lse) Minimal 1.54 Small .28
Adolescent-low aspirations (A-las) Minimal 1.76 Minimal .53
Adolescent-school problems (A-sch) Minimal 1.30 Minimal .53
Immaturity (IMM) Small 2.22 Minimal .52
Source of data: Miller, Li and Kabell (2015)

MMPI-Adolescent-Restructured Form (MMPI-A-RF)

The MMPI-A keeps evolving, included are some fairly recent changes now marketed as the MMPI-A-RF (Archer, Handel, Ben-Porath & Tellegen, 2016). The MMPI-A-RF is briefer than its MMPI-A counterpart, comprised of 241 true/false items, all of which are shared with the MMPI-A. Interestingly, the standard MMPI-A remains available at the same time and from the same publisher (i.e., Pearson). Revisions were hoped to generate better accounting for variance associated with demoralization, an unwanted influence on traditional MMPI scales among adolescents (Stokes, Pogge & Archer, 2018). One obvious effect of the restructuring, however, was the advent of a hierarchical organization of score reporting (i.e., Higher-Order Scales, Restructured Clinical Scales and Specific Problem Scales), as reported by Handel 2016 and seen in Table 6.5. There were also updates in terminology. As seen in Table 6.5, the MMPI-A-RF’s terms seem modern (e.g., somatic complaints and dysfunction scale resonate to the contemporary practitioner more than archaic and arcane terms like hypochondriasis and psychasthenia as found in the MMPI-A). Regarding school-based use, scores for several comparison groups (e.g., youth in residential treatment, those in outpatient psychiatric settings, those in school settings) can be accessed from a test manual.

Regarding applications of both the MMPI-A and MMPI-A-RF, per-use scoring online (i.e., the “score report” option) may discourage some cash strapped school districts because of relatively high cost. Some school psychologists, however, may choose to use a hand score option. The complexity of the MMPI-A as well as the MMPI-A-RF is such that occasional use is probably contraindicated. MMPI-A dabbling risks interpretative errors and poor use of professional time. For these same reasons, some school psychologists are frustrated to find MMPI-A scores accompanying a teen who returns from a residential treatment stay or one who has undergone a clinic-based assessment. The effort to make sense of scores, especially when only a brief narrative report exists, can be dismaying. The concise volume prepared by Archer and Krishnamurthy (2002), although now dated, may still help in situations like this.

Table 6.5 MMPI-A-RF

Validity scales Variable Response Consistency

True Response Inconsistency

Combined Response Inconsistency

Infrequent Responses

Uncommon Virtues

Adjustment Validity

Higher-order scales Emotional/Internalizing Dysfunction

Thought Dysfunctional

Behavioral/Externalizing Dysfunction

Restructured clinical scales Demoralization

Somatic Complaints

Low Positive Emotions

Cynicism

Antisocial Behavior

Ideas of Persecution

Dysfunction

Aberrant Experiences

Hypomanic Activation

Specific problem scales Somatic-cognitive scales Malaise

Gastrointestinal Problems

Head Pain Complaints

(and 2 others)

Internalizing scales Helplessness/Hopelessness

Self-doubt

Inefficacy

(and 6 others)

Externalizing scales Negative School Attitude

Antisocial Attitude

Conduct Problems

(and 3 others)

Interpersonal scales Family Problems

Interpersonal Passivity

Social Avoidance

(and 2 others)

Personality Psychopathology Five Aggressiveness-Revised

Psychoticism-Revised

Disconstraint-Revised

Negative Emotion/Neuroticism-Revised

Introversion/Low Positive Emotion-Revised

 

Behavior Assessment System for Children-3 (BASC-3) Self-report of Personality

Now familiar with the MMPI-A and informant versions of the BASC-3 (teacher and parent option from the last chapter), you might ask a logical question about where the BASC-3 Self-Report of Personality (SRP) falls. More like the MMPI-A or BASC-3 TRS and PRS?  For starters, BASC-3 authors help distinguish self-report measures from informant completed scales. “In contrast to rating scales, self-report scales lend themselves well to recording what goes on in the inner world of the child; they are best suited to report thoughts, feelings, attitudes, and internal reactions to people and events, which give information to the respondent’s position on various aspects of personality” (Reynolds & Kamphaus, 2015, p. 6). But what about the SRP’s actual organization?

BASC-3 Organization and Characteristics

Here we look at features of the BASC-3 self-report versions for children (8-11 years) and adolescents (12-21 years). Just a quick explanatory note is in order—coverage of the BASC-3 self-report for young children (6-7 years) is postponed until near the end of this section because it combines elements of students’ self-report and interview, making it unique. Also, the BASC-3 self-report version for college students is not covered in this book because school psychologists rarely work at colleges. A good starting place for considering the BASC-3 SRP’s structure is how it arranges itself around its key score types: validity, composite, clinical, adaptive, and content, as depicted in Table 6.6. Besides the diversity of score types and their sheer number, another noteworthy consideration is revealed in Table 6.6, the number of items comprising the various BASC-3’s SRP sub-elements. If you scan down the third column of Table 6.6, you will notice that for each clinical, adaptive, and content scale that the BASC-3 SRP deploys only about 10 items. This is a relevant consideration for BASC-3 users because these numbers contrast with the MMPI-A’s longer scales, which you just read about. For example, the MMPI-A’s scale 2 (Depression) boasts 60 items. Sixty items to measure depression represents a five-fold increase over the number of items found on the BASC-3 SRP (adolescent version) Depression (clinical) scale. What’s more, as you will see in Chapter 7, the MMPI-A’s 60 depression items are twice as many as are found among some popular narrow band depression scales. As another example, consider the BASC-3 SRP’s 10 mania items (for adolescents) compared to the MMPI-A’s 46 items scale 9 (Hypomania).

Facts like this may prove important in the final analysis. This is so, because, as you saw in earlier chapters, both adequate construct representation and satisfactory reliability hinge on each scale’s use of a sufficient number of items to accomplish its intended purpose. It is for reasons like this that some school psychology graduate students come to grudgingly think of BASC-3s as more like screening instruments and as less like full-fledged clinical tools (e.g., instruments well suited to help nail down diagnoses). As such, it is reasonable to question if broadband scales, including self-report scales, should be expected to serve as cornerstones for social-emotional assessments. In parallel, it is fair to ask if results from broadband scales ought to dominate decision making about students (especially when there exist so many other sources of complementary social-emotional information).

Table 6.6 Various BASC-3 Self-report Scales (Child and Adolescent versions only)

Scale name

Type

Items in child/adolescent versions

F Index Validity 15/15
L Index Validity 13/15
V Index Validity 4/4
Inconsistency with Other Results Validity
Omitted items Validity
Patterned Responding Validity
Consistency Index Validity
Emotional Symptoms Composite
Inattention/Hyperactivity Composite
Internalizing Problems Composite
Personal Adjustment Composite
School Problems Composite
Anxiety Clinical Scale 11/13
Attention Problems Clinical Scale 9/8
Attitude to School Clinical Scale 8/8
Attitude to Teachers Clinical Scale 7/9
Atypicality Clinical Scale 10/10
Depression Clinical Scale 10/12
Hyperactivity Clinical Scale 8/8
Interpersonal Relations Clinical Scale 8/9
Locus of Control Clinical Scale 8/8
Relations with Parents Adaptive Scale 10/11
Self-esteem Adaptive Scale 7/7
Self-reliance Adaptive Scale 7/9
Sensation Seeking Adaptive Scale –/9
Sense of Inadequacy Adaptive Scale 8/12
Social Stress Adaptive Scale 9/11
Somatization Adaptive Scale –/7
Anger Control Content Scale –/10
Ego Strength Content Scale –/8
Mania Content Scale –/10
Test Anxiety Content Scale –/8
Functional Impairment Index Clinical Index 20/28

BASC-3 SRP and Questions about Interpretation

Next consider score interpretation. Once again juxtapose the MMPI-A and BASC-3 SRP. The BASC-3 has validity scales akin to the MMPI-A. Thus, validity scales are examined first in the interpretive process by diagnosticians using either the MMPI-A or the BASC-3. From here, things start to look different. Critically, code types, which you just learned are a long-valued aspect of MMPI interpretation, do not exist for the BASC-3. Instead, school psychologists using the BASC-3’s SRP’s interpretative process employ a different approach, much like what you heard about in Chapter 5. According to its manual, the BASC-3 “provides an overview of each scale, including a description of its content and interpretation and a discussion of the scale’s relationship to diagnostic, prevention, intervention planning, and evaluation processes” (Reynolds & Kamphaus, 2015, p. 73). But from here, the detailed step-by-step interpretative guide provided TRF and PRF users seems missing for SRP users. Instead, diagnosticians ready to make score interpretations are provided approximately 12 pages of supporting text in the BASC-3 manual. Importantly, BASC-3 SRP users will want to bookmark pages 73 to 84 in the manual (Reynolds & Kamphaus, 2015) and repeatedly refer to those pages during practice.

You may have noticed something interesting in Table 6.6—the SRP includes a few scales nonexistent in the BASC-3 TRS and PRS. Equally important, several SRP scales seem to lack self-evident meaning. School psychologists who fail to use the manual may be ignorant (or mistaken) about the constructs of Locus of Control, Ego Strength, or Social Stress as measured by the BASC-3. For this reason, BASC-3 users are forewarned against simply guessing at a scale’s content or surmising what to make of a student’s score elevations. Using either the manual or a student’s printout can help you understand each scale’s nature; so can using the standard descriptions found in the manual.

Some school psychologists may find the BASC-3 manual’s descriptions too brief for their practical needs. For example, interpretation of the Relations with Parents scale is summarized in eight sentences. What’s more, it proves difficult for readers to know the genesis of the BASC-3 manual’s interpretative suggestions. To this point, the manual states that “conduct disorder may be common among very low scorers” on the Relations with Parent scale (Reynolds & Kamphaus, 2015 p. 79). But, what is the source of this assertion? The manual leaves it unclear whether it springs from practitioners’ observations  conveyed to the test’s authors or, perhaps, field-based insights gleaned by the test’s authors themselves. Alternatively, it is uncertain if this statement derives from empirical studies (e.g., a sample of youth with low score vs. a sample with normal-range scores studied for their contrasting rates of conduct disorders) or from some other source (e.g., logic such as that the items comprising the Relations with Parents scale correspond closely to DSM-5 criteria for conduct disorder). For reasons like these, caution seems advisable.

Another interpretative challenge, rendered more salient when one thinks about the MMPI-A’s use of tidy two-point codes, arises in the face of several simultaneously elevated scales.  For example, let’s assume that Jamal is a teen whose SRP produces three “clinical range” scores (i.e., T-scores ≥ 70). Favorably, the manual provides statements to facilitate scale interpretation. Concerning these three scales, however, the manual’s advice about may prove puzzling. A portion the relevant statements follow.

Atypicality: “unusual thoughts and perceptions (seeing and hearing things that are not there, feeling that one is being watched or targeted) that are commonly associated with severe forms of psychopathology such as schizophrenia” (Reynolds & Kamphaus, 2015, p. 76).

Depression: “traditional symptoms of depression, including feelings of loneliness, sadness, and inability to enjoy life…broad problems with adjustment that may be overlooked by others because such children are notably unobtrusive” (Reynolds & Kamphaus, 2015, p. 76).

Hyperactivity: “…excessive activity level associated with ADHD….such behavior is pervasive and at a level that will result in significant behavior problems” (Reynolds & Kamphaus, 2015, p. 76).

How might a diagnostician meaningfully synthesize such diverse verbiage (and the underlying implications of each description)? In part, the conundrum appears to be that Jamal’s various scale elevations prompt potentially contradictory interferences. Is Jamal expressing overt behavior  associated with hyperactivity? Signs of schizophrenia? Signs of depression? How should the various descriptions above be reconciled to formulate a coherent picture? Should they actually be reconciled or simply reported perfunctorily, one after another? The latter option is, of course, antithetical to the HR approach you learned about in Chapter 2. It is also inconsistent an integrated, user-friendly report (more on this in Chapter 10). As you already saw in Jennifer’s case, a two-point code on the MMPI-A might circumvent such problems.

As you already saw in Chapter 5, the challenge to make sense out of several elevated scores on the BASC-3 SPR is sometimes circumvented by relying on interpretation of just one (grand) composite score. For example, you saw that both the CBCL and the Devereux Scales of Mental Disorders provide singular composite scores. In fact, both the Devereux composite and CBCL Total Problems scores correctly classified students with ED without resorting to profile analysis (Reddy, Pfeiffer & Files-Hall, 2007). In the example above, perhaps Jamal has a general mental health condition best conceptualized as an omnibus problem with various indistinct features. Maybe trying to blend descriptions from three (or more) elevations is futile. By the way, interpreting just one composite score is compatible with the idea of a “p” (e.g., the global indicator of psychopathology that you read about in Chapter 1; Caspi et al., 2014). At any rate, BASC-3 interpretation must proceed without a singular global composite score.

BASC-3 and Classification Validity

Practically speaking, can users trust SPR scores to aid mental health classifications? Are there applied statistics available to confirm this specific capability? Although the BASC-3 manual includes substantial validity evidence (e.g., factor analysis and concurrent validity studies), there is no information on classification validity per se. To be quite clear, as is true for BASC-3 TRF and PRF, for the BASC-3 SPR the manual includes no diagnostic utility information (e.g., sensitivity and specificity). Furthermore, a literature search failed to locate post-publication studies reporting sensitivity and specificity for the self-report versions of BASC, BASC-2 or BASC-3.

You were already warned in Chapter 1 about the fallacy of counting on group data to confirm classification validity, but that appears to be all that is available in the manual. What’s more, statistical significance between clinical groups and controls are not reported. Instead, mean values are reported for clinical groups. Readers are encouraged to examine the manual (Reynolds & Kamphaus, 2015, p. 266). Here one finds that mean values for various clinical groups (e.g., youth with ADHD, youth with ASD) that are generally slightly elevated. No group means, however, fall in either the “at risk” or “clinical range.” Thus, one might assume that the distributions for control and clinical groups, were we consumers able to see them, would evidence considerable overlap (think back to the example of men’s and women’s heights in Chapter 1). Caution about making categorical inference from SPR scores seems the only prudent course of action.

BASC-3 Self-report for Young Students

For young children (age 6 and 7 years) there is a BASC-3 self-report variant entitled Self-Report Interview (SRP-I). Although routine BASC-3 users are likely to find the SRP at least somewhat similar to TRS and PRS, not so for the SPR-I. The SRP-I is comprised of just 14 true/false questions that are read aloud to the child. Only 11 of the 14 items contribute to a raw score (one uncounted item is merely a transition and two more inquire about potential hearing and vision problems). There is a single score, the “Total Score,” which permits use of one of the following summary descriptions:

  • “low” (one or more standard deviations below the mean)
  • “average” (undefined in the manual)
  • “high” (one or more standard deviations above the mean)
  •  “very high” (two or more standard deviations above the mean).

This, of course, is a nomothetic characterization. Interestingly, however, any endorsed item obligates follow-up questioning. For example, a student who confirms disinterest in school, is followed-up with an open-ended inquiry on the same topic. As you might have already recognized, determining the particular aspects of a student’s school disinterest is an idiographic pursuit. It is conceptually akin to conducting elements of a clinical  interview, as you will see in Chapter 9. All these considerations mean that users of the BASC-3’s SPR-I are obligated to read the manual to assure correct administration and to guarantee proper interpretation. Don’t just wing it.

Conners Comprehensive Behavior Rating Scale (Self-report)

If you read about the Conners CBRS Teacher and Parent report options in Chapter 5, then you already know a lot about the Conners CBRS Self-report (Conners CBRS-SR). Thus, users of the teacher and parent versions of the Conner CBRS might simply opt for use of the self-report version. That said, surveys indicate that the Conners CBRS-SR lacks the popularity enjoyed by the BASC-3 SRP (Benson, Floyd, Kranzler, Ekert, Fefer & Morgan, 2019).

Conners CBRS-SR Organization and Characteristics

The Conners CBRS-SR comes in two options, the Conners Clinical Index made up of 24 items designed to screen for Disruptive Behavior Disorders, Learning and Language Disorders, Mood Disorders, Anxiety Disorders and ADHD (see Conners 2010, p. 19 for details). The Conners Clinical Index’s screening capability is covered in Chapter 15. More popular is the standard Conners CBRS-SR comprising 179 items for students 8 through 18 years, which is covered here.

Like the Conners CBRS options for teachers and parents, it is organized with the following features:

  • Content Scales
  • DSM Symptom Scales
  • Conners Clinical Scale
  • Validity Scales
  • Critical Items
  • Other Clinical Indicators
  • Impairment Items

Additional scales, such as those reporting a composite index or supporting DSM diagnoses, also exist (see Table 5.9 for details). In some ways the common organizational structure among Conners CCRS teacher, parent, and self-report versions keeps things simple. That is, frequent users of the former two tools can presumably easily use the latter tool.

Conners CCRS-SR and Interpretation

As with the BASC-3 SRP, let’s look at the Conners CBRS-SR interpretation vis-à-vis the MMPI-A. As you already know, the Conners CBRS-SR is one element of three integrated tools (teacher, parent, self). But there are no MMPI-A teacher or parent options to provide objective triangulation with students’ self-report scores. That said, unlike with the MMPI-A, Conners CCRS users will find relatively little guidance in the manual about interpretation. In fact, there is no separate set of interpretative guidelines devoted solely to self-report.

A sketch of a case may help us understand. Assume that an 18-year-old student sought assistance for recent and potentially severe depression (and declined the school psychologist’s suggestion for parents and teachers to complete rating scales). Accordingly, the school psychologist opted for a Conners CBRS-SR coupled with a clinical interview. He was particularly interested in scores on the Conners CBRS-SR’s “major depressive episode” DSM Symptom Scale, made up of 15 items. This permits us to scrutinize the possibilities for detecting depression. In contrast to the Conners CBRS-SR, the MMPI-A’s scale 2 (depression) is comprised of 60 items (Archer 2017). Moreover, a supplemental scale entitled A-dep is made up of 25 items. Archer’s MMPI-A-accompanying volume cites approximately 10 validity studies associated with A-dep (2017, p. 245). For example, one study partitioned A-dep scores into four components: dysphoria, self-deprecation, lack of drive, and suicidal ideation. There is nothing like this to be found regarding the Conners CBRS-SR Major Depressive Scale. (This is true even though the manual does report a concurrent validity study of the Conners CBRS Mood Disorder Indicator and the Childhood Depression Inventory as the criterion; see Conners, 2010).

The forgoing discussion notwithstanding, it is obvious that sometimes Conners CBRS self-report scores are used to help confirm or disconfirm extant hypotheses. For example, Juanita is a 10-year-old whose records, classroom observation, teacher interview, and parent interview all point toward anxiety. If Juanita’s Conners CBRS-SR profile contained an elevation on just one DSM scale, generalized anxiety, the credibility of the anxiety hypothesis logically would merit a boost. In contrast, if Juanita’s profile contained only average-range scores, then the prospect of anxiety would take a hit. Like broadband scales generally, however, (including the BASC-3) the Conners CBRS might produce a non-specific profile with lots of elevated scales. Hoping for clarity regarding just the prospect of anxiety, Juanita’s school psychologist found a Conners CBRS-SR printout containing several, not just one or two, elevated DSM-related dimensions. Her T-scores are listed below in parentheses:

  • ADHD-Predominately Inattentive (73)
  • ADHD-Predominately Hyperactive-Impulsive (68)
  • Conduct Disorder (73)
  • Oppositional Defiant Disorder (86)
  • Major Depressive Episode (90)
  • Manic Episode (86)
  • General Anxiety Disorder (90)
  • Social Anxiety Disorder (87)
  • Obsessive-compulsive Disorder (90)
  • Autism Spectrum Disorder (90)

The Conners CBRS-SR printout with this configuration might swamp a school psychologist with contradictions. The manual’s guidance regarding DSM symptom scales for T-scores above 65 indicates, in part, the following: “The diagnosis should be given strong consideration” (Conners, 2010, p. 46). Consequently, Juanita, with 10 T-scores above 65, might be considered to be experiencing many mental health conditions simultaneously, including autism spectrum disorder (ASD) and Conduct Disorder (CD). Critically, she appears to have two conditions that most psychologists would consider to be incompatible. Specifically, how does a 10-year-old with impaired social awareness and insistence on routine (ASD) engage in actions whose purpose is the social domination and manipulation of others for personal gain (CD)? In fact, empirical studies fail to confirm that ASD heightens risks for conditions like conduct disorder (Kantzler, Fernell, Westlund, Hagberg, Gillberg, & Miniscalco, 2016; Tsai, 2014).

What’s more, simple guidelines for interpreting score elevations may fail to jibe with diagnostic realities. The Conners CBR’s manual suggestion for interpreting score elevations, like the manuals of many other broad-band scales, represents a rather simplistic one-size-fits-all approach. Regardless of the Conners CBRS scale, the manual advocates the same statement, “significant features of the disorder are present in that setting” anytime an elevation is detected (Connors, 2010, p. 46). But are such blanket proclamations really creditable? After all, are we to believe that a T-score above 65 is just as likely to convey a risk of ADHD as anxiety (i.e., for equivalently elevated scores on the respective DSM-related scales)? It seems implausible that this could be true. You will soon see diagnostic likelihood ratios (in in Table 6.7). The values are anything but identical. How might scale elevation interpretations be so uniform?

Conners CBRS-SR and Classification Validity

Users of the self-report Conners CBRS experience the same uncertainty regarding classification validity as do users of comparable Conners teacher and parent versions. Although there are abundant sensitivity and specificity statistics, cut-scores used to create them are not reported (see Table 6.7 for values associated with the DSM symptom scales). Further caution is advised because cross-validation studies are not apparent in the manual. Moreover, no post-publication classification validity studies were found concerning the Conners CBRS-SR during a literature search. Practically speaking, this would suggest that only clearly non-elevated scores might be trusted to prompt entry of DLRnegative values into a nomogram. One might surmise below-the-cut-point scores would include “high average” scores (i.e., T = 60-64) as well as any scores lower than this (i.e., “average” and “low average”). Similarly, only clearly elevated scores (e.g., “very elevated” score > 70) might signal use of a DLRpositive values in a nomogram. This seems to call for unnecessary guesswork.

A DSM symptom scale score for anxiety above the cut-line, for example, would be more likely to verify anxiety than a DSM symptom scale score above the cut-line for ADHD to verify ADHD. Diagnosticians are encouraged to use all sources of information as part of an ongoing HR process (see Chapter 2). They might also use probability nomograms along the way (also seen in Chapter 2).

Table 6.7 Effect Sizes and DLR Values for Conners CBRS DSM-related Scales Predicting Clinical Samples vs. Representative Controls

Scale Effect size associated with positive score Diagnostic Likelihood Ratio Positive  Effect size associated with negative score Diagnostic Likelihood Ratio Negative 
ADHD Hyperactivity-Impulsivity Minimal 1.4 Moderate .19
ADHD Inattentive Small 2.1 Small .48
Conduct Disorder Moderate 7.2 Small .37
Oppositional Defiant Disorder Moderate 6.2 Small .23
Major Depressive Episode Small 3.7 Small .22
Manic Episode Small 3.3 Small .40
Generalized Anxiety Disorder Small 3.0 Small .37
Separation Anxiety Disorder Small 2.1 Small .42
Social Phobia Small 3.3 Small .42
Obsessive-compulsive disorder Small 2.6 Small .45

Achenbach System of Empirically Based Assessment (ASEBA) Youth Self-report (YSR)

The ASEBA Youth Self-report (YSR) is for those age 11 to 18 years. The YRS closely parallels the ASESBA CBCL and TRF you read about in Chapter 5. In other words, based on 112 items (each rated 0 = not true, 1 = somewhat or sometimes true, 2 = very true or often true), the familiar ASEBA school-age scales are found (e.g., factor analysis related [syndrome] scales ranging from anxious/depressed to aggressive behavior; DSM-5 ranging from depressive problems to conduct problems). Thus, scores on YSR syndrome and DSM-5 scales can be compared straight up with CBCL and TRF results. Information about score levels (e.g., “borderline” T scores range from 65 to 69 and “clinical range” score ≥ 70) also hold for the YSR. Of course, some youngsters referred for evaluation (especially for academic problems) are poor readers. Although YSR items are estimated to require fifth-grade reading proficiency, the manual indicates that items can be administered orally. As with the CBCL and TRF, multicultural options and numerous foreign language translations exist.

The first two pages of the four-page YRS record form sometimes set the stage for follow-up student interview. This section can also promote idiographic considerations. Items on these pages yield numerical competency scores (i.e., Activities, Social, School and Total Competence). But these pages also include some entirely open-ended items that may glimpse a student’s qualitative inner world. Specifically, students describe and appraise themselves regarding the following: sports, hobbies/activities, clubs/groups, jobs/chores, friends, cooperation, self-amusement, and academics.

Consider the idiographic possibilities that might arise if Franklin, the teen seen in Chapter 2, had completed YSR competency items. Franklin’s problem was hypothesized to concern classroom-specific adjustment problems. But his adjustment difficulties appeared  only in classes taught by two  teachers (with relatively few problems evident in four other classes, as well as in most extra-school aspects of life). Context-related problems represent an idiographic hypothesis, and responses on the YSR might help determine whether this type of speculation holds up in the face of information provided by Franklin himself. Logically, if this hypothesis were true, then self-reports about getting along with siblings, parents, and other children (YSR item VI) might reveal no (few) problems. Using similar logic, Franklin’s subject-by-subject rating of academics (item VII) might be expected to contain indications of classroom-specific, rather than generalized, concerns. Even more personal and uniquely illuminating information regarding school adjustment problems might have been evident on the YSR item prompting self-description of “concerns or problems.” This item is entirely open-ended (i.e., free of Likert-ratings). All of this implies that scanning the first two pages of a student’s YSR record form might prevent over-reliance on scores while simultaneously revealing an idiographic world rich in possibilities.

Of course, school psychologists using the HR Worksheet are encouraged to answer most referral questions by using complementary idiographic and nomothetic foci. In fact, some of the YSR open-ended items may strike diagnosticians as similar to portions of a conventional clinical interview (e.g., when the YSR prompts a student to describe the best things about himself). Moreover, these items might offer a window into students’ strengths and supports. Accordingly, the final page of the HR Worksheet, which calls for diagnosticians to record students’ strengths and assets, might fit hand and glove with this portion of the YSR. Because school psychologists are universally pressed for time, a self-report tool that streamlines a clinical interview (as seen in Chapter 9) seems appealing. By the way, routine post-testing clinical interviews are encouraged by the YSR’s author (Achenbach, 2019).

ASEBA YSR and Classification Validity

The YRS’s idiographic capabilities notwithstanding, comparing a referred student to representative peers (i.e., a nomothetic consideration) is the YRS’s most obvious use. Further, when such individual-to-normative group comparisons engender a classification, then we are brought back to the topic of classification validity. And like the ASEBA’s CBCL and TRS, classification validity of the YRS is provided without equivocation. Information about the YRS involves correct classification of referred (clinical) as distinct from non-referred (control) youth. The cut-point appears to be T-score = 65. Odds ratios can be found in Table 6.8. As you saw regarding other ASEBA tools in Chapter 5, odds ratios can be treated like DLRs when diagnosticians choose to employ probability nomograms.

Table 6..8 Odds Ratios Associated with Scores on the ASEBA YSR 

 

 

Odds ratio

Empirically Based Scales

Anxious/Depressed 5
Withdrawn/Depressed 4
Somatic Complaints 4
Social Problems 4
Thought Problems 4
Attention Problems 5
Rule Breaking Behavior 4
Aggressive Behavior 6
Internalizing 4
Externalizing 4
Total Problems 5

DSM-Oriented Scales

Affective Problems 6
Anxiety Problems 3
Somatic Problems 4
ADH Problems 5
Oppositional Defiant Problems 4
Conduct Problems 6
Source: Achenbach and Rescorla (2001)

Cases and Considerations in Using Broadband Self-report Scales and Personality Inventories

As with informant completed scales, mastery of self-report scales requires practice. As already seen, each scale mentioned involves a lot of particulars. And even when a scale’s name is already familiar from teacher and parent completed options, when a student rates herself, there are a host of unique considerations. Cases can help. Two follow.

Self-report to Address Hypotheses-the Case of Chance

You learned in Chapter 2 that several bits of information, each considered deliberately, is a better strategy than merely examining any singular source of information. And when school psychologists make the mistake of over-reliance on a single source, the prime culprit apt be a rating scale (such as a self-report instrument). Consider the case of a 15-year-old high school sophomore named Chance. He was referred with questions about why he had begun to miss school. His parents were at a loss to explain the change, although they noted that he had recently acquired a new peer group. That said, his mother harbored a deep concern that she only reluctantly shared with the school psychologist. Chance has a first cousin who is five years his senior, and that individual has recently been diagnosed with schizophrenia. The cousin had historically done well in school, only to suffer deteriorating functioning during early adulthood  followed by symptoms of florid psychosis. Both of Chance’s parents understood that there is a heritable (genetic) influence regarding schizophrenia. Parents disclosed to the school psychologist that there were no changes in their family life, nor had any health problems occurred. School records revealed a long history of high average scores on high stakes achievement tests in all content areas. Report card grades were similarly above average, and attendance had never been an issue. Chance had no discipline referrals. An interview with teachers indicated the following: several mentioned that he appeared to be tired during morning classes (no such complaints occurred among those teaching classes after lunch); he sometimes appeared poorly groomed; several teachers (both morning and afternoon) cited disinterest in learning; one teacher said that Chance looked “a little depressed.”

This information was sufficient to prompt several hypotheses. The first, implied by Chance’s parents, concerns early harbingers of schizophrenia (prodromal phase). A second involves depression. A third involves acquisition of a new social network characterized by classmates with little academic interests who model the same behavior for Chance. A fourth possibility is that Chances behavior merely represents common changes associated with adolescents (i.e., no significant psychological problem). It was decided to use a BASC-3 SRP to help distinguish among these possibilities. Results appear in Table 6.9.

The standard interpretation of a BASC-3 SRP might be envisioned to parallel that of teacher and parent versions. Validity scales would be considered first, just as would be the case with the BASC-3 TRS and PRS. There is relatively little in Chance’s scores to suggest that his self-ratings are invalid. The one exception concerns a caution level indication on the L Index. The manual offers three possibilities for this score level: psychological naiveness, unwillingness to disclose information about himself, or a tendency to present one’s self in an idealized manner. With a bit a caution, the school psychologist chose to proceed with interpretation. As seen above, the BASC-3 manual omits advice about hierarchical interpretation. Nonetheless, if the ESI were considered first, then it would be logical to conclude that Chance’s average range score speaks against the prospect of any emotional problem. The same conclusion (lack of personal endorsement of social-emotional problems) would be buttressed by Chance’s average range scores on four other composites (i.e., School Problems, Internalizing Problems, Inattention/Hyperactivity, and Personal Adjustment).

But as you learned in Chapter 2, test scores are best viewed within the context of various sources of assessment information in a process that is reiterative. For Chance, earlier iterations have promoted hypotheses about schizophrenia, depression, changing peer alliances, and attitudes and behavior that although novel (and perhaps objectionable) fail to represent anything more than garden variety adolescent adjustment. Thus, Chance’s scores in Table 6.9 might be analyzed in light of case specific hypotheses rather than with a generic rubric. Let’s start with the prospect of schizophrenia. Now the school psychologist must don her rational problem solver hat. Onset of schizophrenia might be expected to present with 1. deteriorating ego functioning (i.e., diminished reality testing, clouded personal identity, poor emotional competence) or 2. progressive adaptive impairments or both 1 and 2. Consequently, aberrant scores on Ego Strength or Personal Adjustment might be viewed as supporting the schizophrenia hypothesis. According to the manual’s guidance, Chance has average range scores on both of these (T-scores of 49 and 50, respectively). More straightforward is the depression hypothesis. This is so because the BASC-3 SPR contains a Depression scale (T-score = 42). Once, again Chance’s average range Depression score weakens this hypothesis. What about the final two hypotheses and Chance’s BASC-3 results? Self-rating scales are largely silent regarding these hypotheses. Instead, the school psychologist would need to continue to rely on logic and persist in her detective work. This would mean an in-depth clinical interview with Chance as well as talking further with his teachers and parents. For now, however, nomothetic explanations in the form of a mental health diagnosis is weakened. Idiographic explanations related to peers are viable. So are somewhat idiographic explanations related to adolescence (e.g., changing sleep patterns, changing interests). Diagnosticians need to be prepared to problem solve, think, and flexibly collect data. In Chance’s case, self-report scores help but on their own they resolve little.

Table 6.9 BASC-3 SRP scores for Chance

Scale name

Score range
F Index Acceptable
Response Pattern Acceptable
Consistency Acceptable
L Index Caution
V Index Acceptable

Scale name

T-score
School problems 45
Internalizing problems 42
Inattention slice hyperactivity 47
Emotional symptoms index 42
Personal adjustment 50
Attitude to school 38
Attitude to teachers 45
Sensation seeking 57
Atypicality 57
Locus of control 41
Social stress 41
Anxiety 37
Depression 42
Sense of inadequacy 42
Somatization 42
Attention problems 43
Hyperactivity 51
Relations with parents 49
Interpersonal relations 53
Self-esteem 53
Self-reliance 45
Anger control 39
Mania 43
Test anxiety 41
Ego strength 49
Functional impairment 43

A Language Conundrum-a Monolingual Farsi Speaker

Kaspar Shirazi presents as a perfectly groomed 15-year-old young man with a pleasing smile. He has attended school for two-weeks in the United States after he recently left Iran with parents and two younger siblings. He was referred to the school psychologist for temper outbursts at school. In fact, this has occurred three times since his arrival at school in the U.S. His mother, a former high school teacher, speaks some English. None of the other family members, however, speaks or reads in a language other than their mother tongue of Farsi. The referral question involves understanding Kaspar’s presentation.

With difficulty, Kaspar’s mother recounted in English the family’s story and perceptions of her son’s situation. Once prosperous, the Shirazis had suffered financial loses for the past three years as the Iranian economy deteriorated. A once thriving family business was lost. Unlike his younger siblings, Kaspar reportedly always worried about the welfare of his family. He seemed especially concerned about the prospect of coming to the United States. According to his mother, Kaspar needed nearly constant reassurance that he would not be separated from his parents or siblings. Evidently, Kaspar entertained the frightening fantasy of being detached from parents in transit and forced to fend for himself in a foreign airport. Repeated reassurances failed to alleviate this worry. School had always come easy for Kaspar; he had historically scored high in all academic subject areas. In Iran, he had relatively few school friends. But he had no emotional problems in school, and no temper outbursts.

The local school district has access to a Farsi interpreter, but that individual’s consultant contract means that she would not be available for a face-to-face meeting for approximately two weeks. Consequently, with only this thumbnail sketch, the school psychologist chose to quickly acquire a Farsi language YSR. The logic is that a YSR might offer insight into the nature of Kaspar’s presentation pending an interpreter-assisted interview. An interview informed by a YSR profile (plus responses to YSR open-ended items) is a strong strategy (Achenbach, 2019), more of which is discussed in Chapter 9. Kaspar’s scores appear in Table 6.10. Societal norms have been collected on the Farsi YSR, allowing application of proper norms for score generation (the process is detailed by Achenbach, 2019). You already saw how societal norms for other ASEBA instruments were used in Chapter 5. This is essentially the same process, although self-ratings rather than informant-ratings were used with Kaspar. Even before an interview is possible, potential areas of concern are apparent. Armed with this information, additional questions for Kaspar’s mother and, especially for Kaspar himself, started to emerge.

Table 6.10 ASEBA YRS Scores for Kaspar

Scale

T-score

Anxious/depressed 70
Withdrawn/depressed 83
Somatic complaints 64
Social problems 80
Thought problems 77
Attention problems 63
Rule-breaking behavior 56
Aggressive behavior 67
Affective problems 76
Anxiety problems 68
Somatic problems 56
Attention deficit/hyperactivity problems 54
Oppositional defiant problems 70
Conduct problems 55

Summary

Personality inventories and self-report scales arguably fill a niche left empty by behavioral rating scales whose scores rely on the viewpoint of parents and teachers. Self-ratings access the respondent’s internal world in a way impossible for informant-completed scales. Moreover, adult personality inventories enjoy a long and illustrious history in the form of the MMPI. This instrument help demonstrate the limitations of clinicians’ judgment and reveal the possibilities of rule-driven interpretation. At a practical level, with teens, the MMPI-A continues to expose some of the limitations in popular rating scales such as the BASC-3 and Conners CBRS. These include relatively simplistic interpretation and limited evidence supporting classification validity. With adolescents, and children as young as 8 years, self-report scales find routine usage in school-based practice.

 

 

 

 

 

 

 

 

 

 

 

 

License

Social-Emotional Assessment in Schools Copyright © by David L. Wodrich. All Rights Reserved.

Share This Book