EXPLORE Versus NAEP in Kentucky
Introduction
This article explores the relationship between the percentage of Kentucky eighth grade students scoring at or above the National Assessment of Educational Progress (NAEP) "Proficient" level in mathematics and reading and the percentages of the same student cohorts scoring at or above the ACT’s Benchmark Scores and College Degree Awards in Kentucky.
EXPLORE is a part of the ACT, Incorporated’s EPAS testing system. ACT’s EXPLORE Benchmark Scores indicate students are proceeding on a good preparation track for eventual success in college and careers. The EXPLORE Benchmark scores are traceable to empirical studies ACT performed in colleges and universities to determine those scores on its ACT college entrance test that corresponded to decent odds of passing the first related freshman college courses. All public school students in Kentucky take EXPLORE.
The findings detailed below indicate that NAEP Proficient scores for Kentucky eighth grade students correlate very well with the EXPLORE Benchmark percentages.
The relationship of Kentucky’s EXPLORE and NAEP results provides additional evidence that in comparisons of outcomes from state tests to NAEP, if the goal is preparation for college and careers, then NAEP Proficient, not NAEP Basic, is the appropriate score to use. The discussion below also adds additional evidence that Kentucky’s current state assessment, which will be disbanded shortly, is providing inflated data on the true development of the state’s students.
Specific observations on Kentucky’s NAEP and EXPLORE performance
Figures 1 and 2 below show the percentages of Kentucky eighth grade students that scored at or above NAEP Proficient and at or above the EXPLORE Benchmark scores for the years where results are available from both tests. For each school term, the same cohort of students took both assessments.
The NAEP to EXPLORE math agreement was the closest found in this study with only a 1-point differential in 2007 and a 2-point spread in 2009.
The NAEP to EXPLORE reading difference is 7 points in 2007, but this was reduced to only a 2-point spread in the 2009 testing.
The close agreement for both math and reading is remarkable given the facts that the development process for these two assessments and their scoring schemes are different. However, while the EXPLORE is from the ACT, Incorporated, it should be noted that the NAEP prime contractor is the Educational Testing Service, which creates the SAT. Thus, both tests come from organizations that have a strong understanding of college preparatory needs. That might make the agreement in Figures 1 and 2 a bit less surprising.
Also, keep in mind that NAEP is a sampled assessment. The NAEP results have measurement error, which is not shown on the graphs above. Once that sampling error is considered, the agreement in the data becomes even more remarkable.
The agreement is particularly noteworthy once currently reported proficiency rates from the Kentucky Core Content Tests (KCCT) in mathematics and reading are considered. The Kentucky 2008-09 Interim Performance Report (access from menu here) shows Kentucky Core Content Test results of 68.05 percent proficiency in eighth grade reading and 55.16 percent proficiency in eighth grade math.
Implications
Very clearly, just as Kentucky’s NAEP and EXPLORE eighth grade reading and mathematics results closely agree with each other, the combined message from those two testing systems tells us the Kentucky Core Content Tests in those subjects are seriously inflated.
The findings here also help to further address long-standing concerns about the real value and meaning of the NAEP Achievement Level Scores. Given the current emphasis on getting more students ready for college and careers, the Kentucky EXPLORE to NAEP comparisons indicate that NAEP Proficient is indeed a meaningful score and appears to be a far more appropriate comparison for states to use in evaluating the performance of their own assessment systems. In sharp contrast, those who would like to compare state assessments to NAEP Basic levels of achievement are setting their sights too low.
As a note, if a good psychometric case can be made that the 4th grade NAEP achievement level scores are well-correlated to the 8th grade scores, that would provide the ability to better determine how many elementary school students in each state are on track at that early level to be fully college/career ready. That would obviously be a very valuable additional piece of information.
Source Notes for Data in Graphs:
- EXPLORE scores come from this Kentucky Department of Education Excel report
- NAEP eighth grade scores for 2007 and 2009 come from Table A-19 in the 2009 NAEP Math Report Card and The 2009 Reading Report Card.
 |
 |
