Journal of Psychoeducational Assessment recent issues

Concordance of the Children's Executive Functions Scale With Established Tests and Parent Rating Scales

Goulden, L. G., Silver, C. H. — Wed, 18 Nov 2009 23:37:35 PST

An increase can be seen in the number of tests designed to assess executive dysfunction in children, but few address ecological validity by attempting to measure functioning in the natural environment. This study examined the utility of the Children’s Executive Functions Scale (CEFS), a 99-item parent-report measure. The CEFS was designed to measure a variety of behaviors related to executive functioning, including social appropriateness, inhibition, problem solving, initiative, and motor planning. A sample of 59 children was evaluated with the CEFS, established tests of executive functioning, Child Behavior Checklist, and Conners’ Rating Scale. Correlations between the CEFS and the test variables were modest and produced a pattern slightly different from correlations obtained for the other parent report measures. The CEFS was significantly correlated with the other parent report measures; however, the magnitude of the coefficients suggests that the CEFS is providing unique information.

Relationship Between Parenting Stress and Ratings of Executive Functioning in Children With ADHD

Joyner, K. B., Silver, C. H., Stavinoha, P. L. — Wed, 18 Nov 2009 23:37:35 PST

Executive functioning is important to assess in children with attention deficit/hyperactivity disorder (ADHD). Parent report is used to obtain information about a child’s executive functioning; however, parent report can be influenced by many factors. This study’s hypothesis was that higher ratings of children’s executive dysfunction are associated with parenting stress. Parents of 32 children (ages 8-12 years) with ADHD completed the Parenting Stress Index and two executive function rating scales, the BRIEF and CEFS. Stress produced by child and family system characteristics was significantly correlated with composite scores and subscales from the executive function rating scales. Findings suggest that parent ratings of their children’s executive functioning appear related to the stress they experience being parents, particularly with regard to their children’s challenging behaviors.

Psychometric Assessment and Reporting Practices: Incongruence Between Theory and Practice

Slaney, K. L., Tkatchouk, M., Gabriel, S. M., Maraun, M. D. — Wed, 18 Nov 2009 23:37:35 PST

The aim of the current study is twofold: (a) to investigate the rates at which researchers assess and report on the psychometric properties of the measures they use in their research and (b) to examine whether or not researchers appear to be generally employing sound/unsound rationales when it comes to how they conduct test evaluations. Based on a sample of 368 articles published in four journals in the year 2004, the findings suggest that, although evidence bearing on score precision/reliability and the internal structure of item responses remains under-reported, researchers appear to be assessing the relationships between test scores and external variables relatively more frequently than in the past. However, findings also indicate that, all told, very few researchers are assessing and reporting on internal score validity, and score precision/reliability, and external score validity, and in that sequence, suggesting that applied researchers may not always be adopting sound test-evaluative rationales in their psychometric assessments.

Self-Perceptions, Discrepancies Between Self- and Other-Perceptions, and Children's Self-Reported Emotions

Nuijens, K. L., Teglasi, H., Hancock, G. R. — Wed, 18 Nov 2009 23:37:35 PST

Self and others’ perceptions of victimization, bullying, and academic competence were examined in relation to self-reported anxiety, depression, anger, and global self-worth in a non-clinical sample of second- and third-grade children. Previous studies document links between negative emotions and self-perceptions that are less favorable than others’ perceptions. However, the current study suggests that the impact of discrepant self—other-perceptions (in bullying, victimization, and academic competence) on emotions is complex, sometimes involving interactions between perceptions of self and other informants.

An Exploratory Investigation of the Factor Structure of the Reynolds Intellectual Assessment Scales (RIAS)

Dombrowski, S. C., Watkins, M. W., Brogan, M. J. — Wed, 18 Nov 2009 23:37:35 PST

This study investigated the factor structure of the Reynolds Intellectual Assessment Scales (RIAS) using rigorous exploratory factor analytic and factor extraction procedures. The results of this study indicate that the RIAS is a single factor test. Despite these results, higher order factor analysis using the Schmid—Leiman procedure indicates that all subtests are aligned with their theoretically consistent factors. All analyses in this study, including the minimum average partial test, parallel analysis, the Schmid—Leiman procedure, as well as principal factors with orthogonal and oblique rotation, support interpretation at the composite intelligence index level and suggest caution when moving to interpretation at the verbal and nonverbal index levels. The memory subtests should continue to be separated from the main IQ battery because of poor g-loadings and contribution to cross loadings of the intelligence subtests. Interpretation at the subtest level should be eschewed.

Book Review: Sattler, J. M. (2008). Assessment of children: Cognitive foundations (5th ed.). San Diego: Author. Sattler, J. M. (2008). Resource guide to accompany Assessment of Children: Cognitive Foundations (5th ed.). San Diego: Author

Floyd, R. G. — Wed, 18 Nov 2009 23:37:35 PST

Convergent Validity of the Reynolds Intellectual Assessment Scales (RIAS) Using the Woodcock--Johnson Tests of Cognitive Ability, Third Edition (WJ-III) With University Students

Krach, S. K., Loe, S. A., Jones, W. P., Farrally, A. — Wed, 23 Sep 2009 00:54:23 PDT

Validity studies with the Reynolds Intellectual Ability scales (RIAS) indicated that RIAS composite intelligence index (CIX) and verbal intelligence index (VIX) scores have moderate-to-high correlation with comparable scores on other instruments. The authors of the RIAS described the VIX scale as a measure of crystallized ability and the nonverbal index (NIX) as a measure of fluid ability, but no studies were available comparing the VIX and NIX scores with an established measure of these abilities, nor studies specifically addressing the utility of the RIAS with a university population. The study examined RIAS scores of university students in comparison to scores on the Woodcock—Johnson Tests of Cognitive Ability (third edition; WJ-III). Consistent with previous studies, there were moderate-to-high correlations between the CIX and VIX scores and corresponding scores on the WJ-III. A substantially lower correlation was evident between the NIX and WJ-III fluid ability scores.

Impact of Test-Taking Behaviors on Full-Scale IQ Scores From the Wechsler Intelligence Scale for Children--IV Spanish Edition

Oakland, T., Harris, J. G. — Wed, 23 Sep 2009 00:54:23 PDT

Research on children’s counterproductive test behavior supports a three-factor model for behaviors: inattentiveness, avoidance, and uncooperative mood. In this study, test behaviors measured by the Guide to the Assessment of Test Session Behaviors (GATSB) are rated on a sample of 110 Hispanic Spanish-speaking children included in the Wechsler Intelligence Scale for Children—IV (WISC-IV) Spanish standardization to further understand the relationship of test behavior to test performance in Spanish-speaking children tested in their native language. GATSB scores generally are within the average range. Correlations between WISC-IV Spanish Full-Scale IQ and GATSB Total Score are significant for the total group, females and males, and ages 13-16 but not for ages 6-8 or 9-12. The amount of variance accounted for by the GATSB score is substantially less than in prior studies of Hispanic children tested in English with the WISC-III. Possible reasons for these discrepant findings are discussed.

The Importance of Relying on the Manual: Scoring Error Variance in the WISC-IV Vocabulary Subtest

Erdodi, L. A., Richard, D. C. S., Hopwood, C. — Wed, 23 Sep 2009 00:54:23 PDT

Classical test theory assumes that ability level has no effect on measurement error. Newer test theories, however, argue that the precision of a measurement instrument changes as a function of the examinee’s true score. Research has shown that administration errors are common in the Wechsler scales and that subtests requiring subjective scoring are the most prone to errors. In this study, graduate students scored fictitious Wechsler Intelligence Scale for Children— Fourth Edition Vocabulary subtests that were constructed to yield actual scaled scores of 4, 10 and 16. Results support the conditional error variance hypothesis: Participants made significantly more scoring errors in the low and high ability conditions than they did in the average ability condition. Implications of these findings with regard to the interpretation of extreme scores are discussed.

Psychometric Properties of the Devereux Early Childhood Assessment in a Head Start Sample

Lien, M. T., Carlson, J. S. — Wed, 23 Sep 2009 00:54:23 PDT

Effective and efficient methods are needed for identifying children who are at risk for later social—emotional challenges. Internal consistencies on the Devereux Early Childhood Assessment (DECA) within this sample (N = 1208) are comparable to the DECA standardization sample. Mean t scores and standard deviations on the behavior concerns subscale are significantly higher, t(1,292) = 2.71, p = .007, than the DECA community sample, t = 55.70, but significantly lower than the DECA identified sample. Total Protective Factors scale scores are inversely related to Behavior Concerns scale scores. The same three protective factors structure exists in the Head Start sample; however, there are differences in item loadings when compared to the DECA standardization sample. Overall, the psychometric properties of the DECA for this Head Start sample provide additional support for its inclusion within the mental health services provided to Head Start populations.

The Incremental Validity of Positive Emotions in Predicting School Functioning

Lewis, A. D., Huebner, E. S., Reschly, A. L., Valois, R. F. — Wed, 23 Sep 2009 00:54:23 PDT

Proponents of positive psychology have argued for more comprehensive assessments incorporating positive measures (e.g., student strengths) as well as negative measures (e.g., psychological symptoms). However, few variable-centered studies have addressed the incremental validity of positive assessment data. The authors investigated the incremental validity of positive emotions relative to negative emotions in predicting adolescents’ adaptive school functioning. Positive emotions demonstrated significant incremental validity in predicting school satisfaction, adaptive coping, and student engagement, but not self-reported GPA. The findings offer some support for the utility of positive measures in psychoeducational assessments.

The Average Distance Between Item Values: A Novel Approach for Estimating Internal Consistency

Sturman, E. D., Cribbie, R. A., Flett, G. L. — Wed, 23 Sep 2009 00:54:23 PDT

This article presents a method for assessing the internal consistency of scales that works equally well with short and long scales, namely, the average proportional distance. The method provides information on the average distance between item scores for a particular scale. In this article, we sought to demonstrate how this relatively simple statistic could be calculated and present examples that show its advantage over traditional methods. Simulation and empirical tests were conducted to establish standards for the average proportional distance of scores. The implications for test construction are discussed with a particular emphasis on the advantages of developing shorter scales for psychological and educational research.

Stability of Adolescents' Multidimensional Life Satisfaction Reports

Antaramian, S. P., Huebner, E. S. — Wed, 23 Sep 2009 00:54:23 PDT

Eighty-four students were administered the Multidimensional Students’ Life Satisfaction Scale (MSLSS) on three occasions, 1 year apart (Grades 8, 9, and 10). The 1-year stability coefficients ranged from .29 to .59, whereas the 2-year stability coefficients ranged from .41 to .59. MSLSS mean scores were consistent across administrations, with the exception of satisfaction with living environment in which scores were lower in Grade 10 relative to the previous two grades. Although a fair degree of stability was observed across most MSLSS domains, differences in the magnitudes of the coefficients across domains and time intervals suggested the importance of contextual factors in understanding the nature of the life satisfaction construct among adolescents as well as implications for the use of total and domain-specific life satisfaction reports as outcomes measures.

Test Review: Wechsler, D., & Naglieri, J.A. (2006). Wechsler Nonverbal Scale of Ability. San Antonio, TX: Harcourt Assessment

Massa, I., Rivera, V. — Wed, 23 Sep 2009 00:54:23 PDT

Development and Evaluation of the Social-Emotional Learning Scale

Coryn, C. L. S., Spybrook, J. K., Evergreen, S. D. H., Blinkiewicz, M. — Fri, 24 Jul 2009 01:33:34 PDT

This article presents the development and evaluation of a measurement device designed to assess elementary-aged students' social-emotional learning needs. A sample of 633 fourth-, fifth-, and sixth-grade elementary students from 11 public schools in a midsized Midwestern U.S. city was used to evaluate the reliability and validity of the 20-item Social-Emotional Learning Scale (SELS) for the sample. A correlated three-factor model consisting of the factors Task Articulation, Peer Relationships, and Self-Regulation was fit using maximum likelihood estimation and found to be adequate. For the sample, the SELS demonstrated evidence of both precision and accuracy, including internal consistency as well as convergent and discriminant validity. Potential applications for the SELS and further research are discussed.

Multitrait-Multimethod Analysis of FCAT Reading and Writing: Or Is It Writing and Reading?

Shermis, M. D., Long, S. K. — Fri, 24 Jul 2009 01:33:34 PDT

This study investigated the convergent and discriminant validity of the high-stakes Florida Comprehensive Assessment Test (FCAT) in both reading and writing at grade levels 4, 8, and 10. The data from the 2006 FCAT administration were analyzed via traditional multitrait-multimethod (MTMM) analysis to identify the factor structure and structural equation models (SEMs), to determine the weights of the influential variables underlying the tests. The MTMM analyses suggested that across all grade levels, the correlation between the multiple-choice reading and multiple-choice writing tests of the FCAT approached the reliability coefficients for each test separately. These correlations were higher than the correlations between the multiple choice and performance sections for each trait. The SEM analyses, however, provided support for both the convergent and discriminant validity of the test scores. The fit for the SEMs improved from a CFI of .84 for 4th graders (a poor fit) to .97 for 10th-grade students.

Evaluation of a Learning Strategies Scale for Middle School Students

Liu, O. L. — Fri, 24 Jul 2009 01:33:34 PDT

Learning strategies have been increasingly recognized as a useful tool to promote effective learning. In response to the lack of available learning strategies measures for middle school students, this study evaluated an instrument assessing cognitive, behavioral, and metacognitive strategy use among middle school students. The instrument, called the Middle School Learning Strategies (MSLS) scale, was examined in terms of factorial structure, reliability, and correlation with course grades. Results confirmed the factor structure based on cognitive, behavioral, and metacognitive strategy use. Scores on the MSLS scale and the three subscales showed satisfactory reliability coefficient (i.e., >.70). Student self-reported grades in language arts, social studies, math, and science were reasonably correlated with the three subscales. Females reported using more behavioral strategies than males. No significant difference was identified in strategy use across grades. Potential uses for the MSLS instrument in middle school educational settings are discussed.

Examining the Underlying Factor Structure of a Statewide Science Test Under Oral and Standard Administrations

Kim, D.-H., Schneider, C., Siskind, T. — Fri, 24 Jul 2009 01:33:34 PDT

This study investigates the factorial invariance of the statewide science tests at the middle school level across the oral administration accommodation and nonaccommodated groups of students with and without disabilities. Results indicate that the factor loadings and error variances are invariant across groups, although the oral administration and general education groups in Grade 8 support partial invariance in error variances. On the whole, this study supports that the science test scores for the oral administration group have the same meaning for those for the nonaccommodated groups. Aside from providing supporting validity evidence, this study also introduces the analytical method that can be used for studies of the same nature in other states.

Factorial Structure of the Anxiety Control Questionnaire in Chinese Adolescents

Wang Shujuan, , Qian Meihua, , Zhang Jianxin, — Fri, 24 Jul 2009 01:33:34 PDT

This article examines the psychometric structure of the Anxiety Control Questionnaire (ACQ) in Chinese adolescents. With the data collected from 212 senior high school students (94 females, 110 males, 8 unknown), seven models are tested using confirmatory factor analyses in the framework of the multitrait—multimethod strategy. Results indicate that the structure of the ACQ based on Chinese adolescents data is different from the one proposed by R. M. Rapee, M. G. Craske, T. A. Brown, and D. H. Barlow (1996). Method effects are found and the ACQ is organized according to three factors, one substantive factor and two method factors associated with the positive- and negative-worded items, respectively. The implication of the method effect is discussed.

Test Review: Woodcock, R. W., Schrank, F. A., Mather, N., & McGrew, K. S. (2007). Woodcock-Johnson III Tests of Achievement, Form C/Brief Battery. Rolling Meadows, IL: Riverside

Grenwelge, C. H. — Fri, 24 Jul 2009 01:33:34 PDT

Mathematical Disabilities--An Underestimated Topic?

Gregoire, J., Desoete, A. — Tue, 23 Jun 2009 04:21:02 PDT

Math Disabilities and Reading Disabilities: Can They be Separated?

Swanson, H. L., Jerman, O., Xinhua Zheng, — Tue, 23 Jun 2009 04:21:02 PDT

This article synthesizes some of the published literature that selectively compares the cognitive functioning of children with math disabilities (MDs) with average-achieving children and poor readers (children with reading disabilities [RDs] or comorbid disabilities [RDs + MDs]). All studies in the synthesis report reading, IQ, and math scores for children with MDs and poor readers. A random coefficients model of effect sizes (ESs) show that (a) ESs between MD and normal achievers were moderated by variations in working memory and literacy, (b) ESs between MD- and RD-only children were moderated by working memory and problem solving, and (c) ESs between MD and MD + RD children were moderated by long-term memory and IQ scores. No support was found for the notion that the differentiation between MD children and poor readers (RD and MD + RD) was related to variations in reading across the reviewed studies.

Mathematics Anxiety and the Affective Drop in Performance

Ashcraft, M. H., Moore, A. M. — Tue, 23 Jun 2009 04:21:02 PDT

The authors provide a brief review of the history and assessment of math anxiety, its relationship to personal and educational consequences, and its important impact on measures of performance. Overall, math anxiety causes an "affective drop," a decline in performance when math is performed under timed, high-stakes conditions, both in laboratory tests as well as in educational settings. This means that math achievement and proficiency scores for math-anxious individuals are underestimates of true ability. The primary cognitive impact of math anxiety is on working memory, particularly problematic given the important role working memory plays in math performance. The authors conclude with a discussion of risk factors for math anxiety and some factors to be kept in mind when working with math-anxious students.

Math Anxiety and Math Ability in Early Primary School Years

Krinzinger, H., Kaufmann, L., Willmes, K. — Tue, 23 Jun 2009 04:21:02 PDT

Mathematical learning disabilities (MLDs) are often associated with math anxiety, yet until now, very little is known about the causal relations between calculation ability and math anxiety during early primary school years. The main aim of this study was to longitudinally investigate the relationship between calculation ability, self-reported evaluation of mathematics, and math anxiety in 140 primary school children between the end of first grade and the middle of third grade. Structural equation modeling revealed a strong influence of calculation ability and math anxiety on the evaluation of mathematics but no effect of math anxiety on calculation ability or vice versa—contrasting with the frequent clinical reports of math anxiety even in very young MLD children. To summarize, our study is a first step toward a better understanding of the link between math anxiety and math performance in early primary school years performance during typical and atypical courses of development.

Individual Differences in Early Numeracy: The Role of Executive Functions and Subitizing

Tue, 23 Jun 2009 04:21:02 PDT

In this study, the question was raised how basic cognitive processes are related to math abilities and how it can be best determined which children are at risk for developing those disabilities. The role of four distinct basic processes in the development of early mathematics was investigated: executive functions, fluid intelligence, subitizing, and language. The counting skills of 115 five- and six-year-old children were also assessed. The results showed that both executive functions and number sense were important factors in children's development of counting skills. Both executive functions and subitizing explained a significant part of variance in children's counting skills. IQ scores could not add further explanation to the variance in early math. The implications of this study are that it seems promising to use the concept of executive functions for the early identification of children at risk for math learning difficulties.

Predicting Arithmetic Abilities: The Role of Preparatory Arithmetic Markers and Intelligence

Stock, P., Desoete, A., Roeyers, H. — Tue, 23 Jun 2009 04:21:02 PDT

Arithmetic abilities acquired in kindergarten are found to be strong predictors for later deficient arithmetic abilities. This longitudinal study (N = 684) was designed to examine if it was possible to predict the level of children's arithmetic abilities in first and second grade from their performance on preparatory arithmetic abilities in kindergarten. Furthermore, the role of intelligence in the assessment of developing mathematical reasoning and numerical facility was investigated. The results showed that good predictions of beginning arithmetic abilities can be made. Even in the long term, it was possible to make predictions of later arithmetic performances, especially for the domain of arithmetical reasoning. Logic thinking abilities such as seriation and classification and counting knowledge may be assessed in kindergarten as possible markers for at-risk development. Furthermore, it is important to take into account intelligence when looking for useful predictors of arithmetical reasoning.

Classification, Seriation, and Counting in Grades 1, 2, and 3 as Two-Year Longitudinal Predictors for Low Achieving in Numerical Facility and Arithmetical Achievement?

Desoete, A., Stock, P., Schepens, A., Baeyens, D., Roeyers, H. — Tue, 23 Jun 2009 04:21:02 PDT

Previous research stresses the importance of seriation, classification, and counting abilities that should be assessed in kindergarten, when looking for crucial predictors of mathematical learning disabilities in Grade 1. This study examines (n = 158) two-year-long predictive relationships between children's seriation, classification, procedural counting knowledge, and conceptual counting knowledge, and their numerical facility and arithmetical achievement in Grades 3, 4, and 5. Path analyses reveal a relationship between procedural counting knowledge in Grades l and 3 and numerical facility, and between conceptual counting knowledge in Grade l and arithmetical achievement in Grades 1 and 2. In addition, procedural counting knowledge correlates with seriation, and conceptual counting knowledge correlates with classification as well as with seriation in Grade 1. Clinicians are encouraged to select early arithmetic subtests with caution when assessing and predicting arithmetic and arithmetical disabilities.

Predicting Mathematical Achievement and Mathematical Learning Disability with a Simple Screening Tool: The Number Sets Test

Geary, D. C., Bailey, D. H., Hoard, M. K. — Tue, 23 Jun 2009 04:21:02 PDT

The Number Sets Test was developed to assess the speed and accuracy with which children can identify and process quantities represented by Arabic numerals and object sets. The utility of this test for predicting mathematics achievement and risk for mathematical learning disability (MLD) was assessed for a sample of 223 children. A signal detection analysis of first grade Number Sets Test scores provided measures of children's sensitivity to number and their response bias. The sensitivity measure, d', but not the response bias measure was predictive of third grade mathematics achievement scores, above and beyond the influence of intelligence, working memory, and first grade achievement scores. Further analyses assessed the sensitivity and specificity of the test and revealed that first grade d' scores identified 2 out of 3 children diagnosed as MLD in third grade and correctly identified about 9 out of 10 children who were not at risk for MLD.