Development and validation of the Bar-On measures
This section discusses how the EQ-i™, EQ-360™ and EQ-i:YV™ were developed and validated.
The Development of the Bar-On Emotional Quotient Inventory (EQ-i)
As was previously mentioned, the EQ-i™ was originally constructed as an experimental instrument designed to examine the conceptual model of emotional and social functioning that I began developing in 1980. At that time, I hypothesized that effective emotional and social functioning should eventually lead to a sense of well-being. It was also reasoned that the results gained from applying such an instrument on large and diverse population samples would reveal more information about emotionally and socially intelligent behavior as well as the underlying construct of what I refer as emotional-social intelligence.
The development of the EQ-i™ proceeded in six phases over a period of 17 years: (i) identifying and logically clustering various emotional and social competencies, skills and behaviors thought to impact human performance and well-being based on my clinical experience and review of the literature; (ii) clearly defining the individual key clusters of competencies and skills that surfaced; (iii) initially generating approximately 1,000 items based on my clinical experience, review of the literature and on input from experienced healthcare practitioners; (iv) determining the inclusion of 15 primary scales and 133 items in the 1997 published version of the instrument based on a combination of theoretical considerations and statistical findings generated primarily by item analysis and factor analysis; (v) initially norming the final version of the instrument on 3,831 adults in North America in 1996; and (vi) continuing to norm and validate the instrument across cultures around the world which is an ongoing endeavor.
The original normative sample included individuals from every Canadian province and from nearly all the states in the United States (US). The gender and age composition of the sample included 49% males and 51% females from 16 to 100 years of age. The sample was 79% White, 8% Asian-American, 7% African-American, 3% Hispanic, and 1% Native American (2% of the respondents did not identify their ethnicity). For more detailed demographic information, including the educational and occupational background of the normative sample, you are referred to the 1997 Bar-On EQ-i™ Technical Manual. Additional information regarding the development, norming and validation of the EQ-i™ can be found in this technical manual as well as in numerous other publications that have appeared in the EI literature over the past two decades.
The EQ-i™ has been translated into more than 30 languages, and data have been collected at numerous settings around the world. Earlier pre-published versions of the inventory were completed by approximately 3,000 individuals in Argentina, Germany, India, Israel, Nigeria, South Africa and the United States. In addition to providing cross-cultural data, this preliminary piloting of the EQ-i™ was important for item selection and alteration, continued scale development and validation, and establishing the final nature of the response format.
In order to explore the possibility of publishing the Bar-On EQ-i™, I began approaching psychological test publishers in North America and the United Kingdom in 1992. As part of these efforts, I met with Dr. Steven Stein who is the CEO of Multi-Health Systems (MHS). The meeting took place in Israel in June 1995, at which I was asked to send Dr. Stein published and unpublished material supporting the early development and validation of the EQ-i™. After MHS’ Research and Development unit (under the direction of Dr. Gill Sitarenios) reviewed the EQ-i™ and supporting material, I was notified a couple of months later that they were interested in publishing this instrument; and in January 1996, I signed an agreement with them to publish the Bar-On EQ-i™. In the meantime, Daniel Goleman published Emotional Intelligence in October 1995. And from the beginning of 1996, I reconnected with Daniel Goleman (a childhood acquaintance from Stockton, California) and began to meet others who had been independently working in the loosely-defined field of “emotional intelligence” including Peter Salovey, John Mayer and many others in closely associated fields.
The Bar-On EQ-i™ was the first EI measure to be peer-reviewed by the Buros Mental Measurement Yearbook in 1999. It is the only EI measure to be referred to in a Congressional Report, submitted to the US Senate by the Unites States General Accounting Office in January 1998. In addition to being favorably reviewed by the Buros Mental Measurement Yearbook in 2001, the youth version of the Bar-On EQ-i™ (the EQ-i:YV™) was selected among 59 measures of emotional and social competence as the instrument of choice by researchers at the University of Oxford in 2003, who recommended its use in UK schools to the British Department of Education and Skills.
The EQ-i™ passed the one million mark during the first five years following its publication in 1997, making it the most widely used EI measure to date. As was previously mentioned, a mildly revised version of the Bar-On EQ-i™ – referred to as the “EQ-i 2.0™” – was renormed in 2011. Although some of the items were re-worded and others were added, the 15 factorial structure of the Bar-On model was re-confirmed, for the most part, in spite of the cosmetic changes that were introduced by the publisher (Multi-Health Systems, Inc.). As such, the revised version will need to be re-normed and re-validated across cultures around the world. As with the original version of the Bar-On EQ-i™, this will be a very long process to assure that it is standardized for use in other countries and that it predicts what it was designed to measure. And this, of course, will have to be documented in the literature. For more information about the EQ-i™ 2.0, contact Multi-Health Systems.
The Development of the Bar-On Emotional Quotient – 360 (EQ-360)
The development of the Bar-On EQ-360™ involved extensive research and the analysis of two major versions of this instrument, which is described below. The preliminary findings indicated that improvements could be made to increase the psychometric strength of the instrument. Building on these findings and the recommendations that were made, the revised version proved to be a psychometrically stronger instrument and resulted in the final published version. This process is described briefly below and in greater detail in the instrument’s technical manual (the Bar-On EQ-360™ Technical Manual).
The first version of the EQ-360™ consisted of 46 items with a 5-point response format ranging from “very seldom true of him/her” (1) to “very often true of him/her” (5). Each of the 15 factorial components of the Bar-On model of emotional-social intelligence was briefly described directly on the response sheet, followed by three questions that were designed to assess the ratee’s functioning in that particular area. There was also space provided on the response sheet for an optional open-ended response for each area.
The original version of the EQ-360™ was completed by a sample of 1,256 raters who included 214 managers, 729 co-workers, 305 direct reports and 8 spouses. Feedback was provided by the data-collection administrators, and detailed data analyses were conducted that led to a number of recommendations for improving the instrument to achieve a psychometrically viable tool. The recommendations focused primarily on the following four points: (i) the need for items that more closely paralleled the Bar-On EQ-i™ item content; (ii) the need to increase the number of items representing each scale to enhance scale reliability; (iii) the need to randomly scatter items from the same subscales so that they did not occur together or in close proximity; and (iv) the need to remove the scale definitions to avoid creating a mindset that would possibly bias the raters’ responses.
Based on the above-mentioned recommendations, construction of a revised Bar-On EQ-360™ began. A large number of Bar-On EQ-i™ items were mildly altered and reworded to change their self-report style to a multi-rater style. A few new items were created to reflect existing Bar-On EQ-i™ items as well as the definitions of the 15 factorial components of the Bar-On model as closely as possible. Additionally, four items that were created for the original version were used in the revised version of the instrument; these were primarily items that demonstrated the strongest psychometric properties. In the end, 88 items were included in the revised instrument, and the Bar-On EQ-360™ response sheet was reformatted to remove scale descriptions and randomly scatter the items throughout the inventory.
The new version of the Bar-On EQ-360™ was subjected to rigorous piloting. Once a total of 1,000 raters completed the inventory, a preliminary examination of the data was conducted. The psychometric properties proved to be very encouraging. The final sample used in norming the revised version of the instrument consisted of 1,900 raters who rated 745 individuals. The revised version of the Bar-On EQ-360™ met our expectations by producing the psychometric properties required for publication in my opinion. Of the 745 ratees, 385 (52%) were males and 353 (48%) were females. The mean age was 40.4 years.
The normative sample was international in composition with participants from Australia, Canada, Holland, Mexico, South Africa, Sweden, the United Kingdom and the United States. The participants also represented of a variety of different occupations as diverse as architecture, finance, healthcare and education from the private and public sector. A more detailed demographic breakdown of this sample is found in the Bar-On EQ-360™ Technical Manual.
The internal consistency reliability of the items was examined for all Bar-On EQ-360™ scales. Scale reliabilities were good to excellent, ranging from .77 for Assertiveness to .98 for total EQ. A complete breakdown for reliability measures, including Cronbach’s Alpha and Standard Error of Measurement (SEM), is found in the instrument’s technical manual.
The construct validity of the Bar-On EQ-360™ was carried out primarily by examining the bivariate correlation between this instrument’s scale scores with those of the Bar-On EQ-i™. I focused primarily on the ratings generated by family members and friends, reasoning that they knew the individuals being rated more extensively and for a longer period of time. All correlations are statistically significant, and most are in the moderately high range with the overall correlation between the Total EQ scores for both instruments being .69 (N=185); the lowest correlation is .33 (Emotional Self-Awareness), while the highest is .78 (Optimism). These results suggest that the Bar-On EQ-360™ is measuring what the Bar-On EQ-i™ is measuring for the most part (i.e., various aspects of emotional-social intelligence). This finding is especially significant in light of the fact that the EQ-i™ has been shown to be moderately to highly correlated with a number of other EI measures.
During the development of the Bar-On EQ-360™, the scale scores for significant differences were compared between self-report and other-observer assessment versions of the instrument. It was found that when there were significant differences, self-report was always higher than other-observer assessments. However, relatively few differences were found — only 11% of the time, meaning that 89% of the time, there were no significant differences between self-report and other-observer assessments. This means that there are no significant differences, for the most part, between self-report and multi-rater observer assessments using the Bar-On psychometric model to assess emotional-social intelligence. These results are described in detail in the Bar-On EQ-360™ Technical Manual.
The Development of the Bar-On Emotional Quotient Inventory: Youth Version (EQ-i:YV)
There were 7 major stages in the development of the Bar-On EQ-i:YV™, and they are as follows:
Stage 1 involved the preliminary scale development. A small group of experts in the area of child and adolescent assessment examined the 133 items from the adult Bar-On EQ-i™ for appropriateness of use with respondents aged 7 to 18 years. Based on the recommendations of these experts, approximately 25% of the items from the Bar-On EQ-i™ were retained for the pilot version of the youth instrument. Approximately 25% of the remaining items were shortened or modified to be more appropriate for younger respondents. A new set of items was written and added to the item pool, producing a total of 96 items.
State 2 involved exploratory factor analyses of the 96-item scale. In this stage, a sample of 371 children and adolescents (160 males and 211 females) between the ages of 7 and 18 years was used. The mean age for the sample was 13.4 years. Participants were from several different schools in the United States and Canada. A 4-factor model emerged using exploratory factor analysis with a varimax rotation. This model, which used 30 items, was identical in the younger (12 years and younger) and older (13 years and older) respondents. This model matched closely with four of the five meta-factorial dimensions from the adult Bar-On EQ-i™ (intrapersonal, interpersonal, stress management, and adaptability). A separate series of exploratory factor analyses with a set of happiness and optimism items produced a 12-item general mood factor. An additional separate series of exploratory factor analyses with a set of positive impression (validity) items produced a 6-item Positive Impression factor. Based on these exploratory factor analyses, 48 items from the original item pool were retained for subsequent scale development.
Stage 3 involved scale re-development. During this stage, a new set of 33 items was written, tapping a cross-section of the key components of the Bar-On model of emotional-social intelligence. These new items were added to the pool of 48 items to produce a revised 81-item instrument.
Stage 4 involved a series of exploratory factor analyses of the 81-item scale. This phase used a sample of 800 children and adolescents between the ages of 7 and 18 years. The sample included 354 (45%) males, 428 (55%) females. The mean age for the sample was 13.3 years. The participants were selected from several different schools in the United States and Canada. Using exploratory factor analysis with a varimax rotation, the same 4-factor structure emerged as in the previous sample: Intrapersonal, interpersonal, stress management, and adaptability factors. This model used 40 items from the 81-item pool. Once again, a separate factor analysis of the general mood items produced a 14-item general mood factor; and a separate factor analyses with the positive impression items produced a 6-item Positive Impression factor. Based on this set of exploratory factor analyses, 60 items were retained for final scale development.
Stage 5 involved confirmatory factor analyses of the Bar-On EQ-i:YV™. This stage used a sample of 280 children and adolescents between the ages of 6 and 18 years — 133 males and 147 females. The mean age for this sample was 12.5 years. As with previous phases, participants came from several different schools in the United States and Canada. Using confirmatory factor analysis, the 4-factor model comprising 40 emotional-social intelligence items was empirically supported. Empirical support was also found for the 1-factor Positive Impression scale. Based on the analyses from the previous phases, the final version of the Bar-On EQ-i:YV™ contained 60-items.
Stage 6 involved creation of the Inconsistency Index. The Inconsistency Index is one of the two validity measures of the Bar-On EQ-i:YV™, which was designed to assess response consistency. It was found that the Bar-On EQ-i:YV™ contains several pairs of highly correlated items that have similar content, which were used to gauge response consistency. Ten pairs of Bar-On EQ-i:YV™ items were thus selected to be used in this index — two pairs for each of the five primary scales — in that they demonstrated the highest correlations within each of these five scales. The Inconsistency Index score is computed by calculating the absolute value of the difference in the response to each pair, and then summing these 10 differences. Some degree of response inconsistency is natural and expected, especially since there are subtle differences between the items in each pair that were selected. As such, it was determined that a score of 10 or greater should be treated as atypical in terms of response inconsistency. Highly inconsistent responding typically occurs with noncompliant or unmotivated respondents and can compromise the validity of the results when the degree of inconsistency is significantly high; and the most likely interpretation for high scores on the Inconsistency Index is that the individual is responding haphazardly or in an unmotivated fashion, and/or he or she is deliberately trying to invalidate his or her results for one reason or another. However, elevated scores on the Inconsistency Index can also mean that the respondent is experiencing difficulty understanding the subtle differences between some of the items. High scores on this index may also be indicative of a lack of self-awareness. The construction of this scale, including the cutoff score, followed a complicated process which is described in detail in the instrument’s technical manual.
Stage 7 involved creating the short versions of the Bar-On EQ-i:YV™ [Bar-On EQ-i:YV(S)™]. The EQ-i:YV(S)™ was developed for situations where time is of the essence, where the individual being tested has limited reading or comprehension abilities, where the respondent might be fatigued from a longer set of items, or where multiple administrations of the instrument are desired. The short form was developed using the large normative sample of the Bar-On EQ-i:YV™ (n=9,172). The goal was to develop a scale that would include a sufficient number of items (6 per scale) to reliably assess intrapersonal, interpersonal, stress management and adaptability competencies (the sum of which would render a total EQ scale). It was determined that the EQ-i:YV(S) ™ would also include the same 6-item Positive Impression scale included in the Bar-On EQ-i:YV™. Thus, the EQ-i:YV(S)™ would have 30 items. The 6-item Intrapersonal scale was retained from the long form. For the Interpersonal, Stress Management and Adaptability scales from the long form, the item-pool for each scale was subjected to confirmatory factor analysis testing a uni-dimensional model. To create 6-item versions of each subscale, only the 6 items with the highest loadings with the latent variables were used.
The Bar-On EQ-i:YV™ (long and short forms) were developed over several months from a large pool of items that assess a cross-section of competencies, skills and behaviors relevant to the emotional-social intelligence construct. The resulting forms were then normed and examined further for reliability and validity.
The Bar-On EQ-i:YV™ was normed on a large sample of children and adolescents attending several different elementary, junior high and high schools in the United States and Canada (n=9,172). All of the respondents who participated in the normative sample were selected from regular classes, meaning that children in special education classes and children with special needs were excluded from this study.
References
To review the psychometric properties of the EQ-i™, EQ-360™ and EQ-i:YV™, you are referred to the numerous summaries of studies that have examined the Bar-On measures. Additionally, the psychometric properties of these EI measures are summarized in the following sources:
Bar-On, R. (1997). The Bar-On Emotional Quotient Inventory™ (EQ-i™): Technical manual. Toronto, Canada: Multi-Health Systems.
Bar-On, R. (2000). Emotional and social intelligence: Insights from the Emotional Quotient Inventory™ (EQ-i™). In R. Bar-On and J.D.A. Parker (Eds.), Handbook of emotional intelligence: Theory, development, assessment and application at home, school and in the workplace. San Francisco: Jossey-Bass, pp. 363-88.
Bar-On, R., & Parker, J.D.A. (2000). The Bar-On Emotional Quotient Inventory: Youth Version™ (EQ-i:YV™): Technical manual. Toronto, Canada: Multi-Health Systems.
Bar-On, R., & Handley, R. (2003). The Bar-On EQ-360™: Technical manual. Toronto, Canada: Multi-Health Systems.
Bar-On, R. (2004). The Bar-On Emotional Quotient Inventory™ (EQ-i™): Rationale, description, and summary of psychometric properties. In G. Geher (Ed.), Measuring emotional intelligence: Common ground and controversy. Hauppauge, NY: Nova Science Publishers, pp. 111-42.
Bar-On, R. (2006). The Bar-On model of emotional-social intelligence (ESI). Psicothema, 18, supl., 13-25.
Siskos, B., Papioannou, A., & Bar-On, R. (2012) Continued cross-cultural validation of the EQ-i:YV™ and the relationship between emotional intelligence, metacognition and conflict styles based on a Greek sample. An unpublished manuscript submitted for review.