Kentucky's No Child Left Behind Loopholes
Loopholes in Kentucky’s NCLB Academic Proficiency Rate Requirements As of Fall 2008
Contents |
Unstable, non-standard scoring over time
The first way validity of Kentucky’s No Child Left Behind program is impacted involves steady erosion over time of the level of work required to achieve a score of “Proficient.”
The academic assessments used to determine NCLB proficiency rates in the Bluegrass State are the Kentucky Core Content Tests (KCCT) in reading and math, which are part of the state’s Commonwealth Accountability Testing System (CATS).
One issue is that the KCCT are not given in any other state. Thus, Kentucky’s definition of “Proficient” does not correlate to scoring programs used elsewhere.
The use of a non-standard test might not be as big an issue if the quality of work required to earn a “Proficient” score in Kentucky had been kept stable over time, but this is not the case. A Bluegrass Institute paper, CATS In Decline: Federal Yardstick Reveals Kentucky’s Testing Program Continues to Deteriorate compares KCCT proficiency rates over time in math and reading to proficiency rates reported by the National Assessment of Educational Progress. In fourth grade reading, for example, the paper shows the rigor of the CATS scoring has steadily decreased so that now a performance the national assessment considers to be below the level of only partial mastery of this skill is graded “Proficient” in the KCCT.
Thus, Kentucky’s use of the KCCT for NCLB accountability provides neither a stable measurement over time nor a measurement that can be fairly compared to what other states use for NCLB reporting.
The published target proficiency rates are an illusion
Under NCLB, various student populations in a school including different racial groups, learning disabled students, and students in poverty must all separately achieve the same target proficiency rates each year. Those target rates are unique to Kentucky and were set in 2003 by the Kentucky Board of Education. These target proficiency rates, called the Annual Measurable Objectives (AMO), vary by school grade configuration as shown in Table 1.
For example, according to Table 1, the public is led to believe that a standard configuration elementary school had to achieve a proficiency rate of at least 47.27 percent in reading and 22.45 percent in math in the 2001-02 school year to meet the AMO requirements. The school had to achieve these rates not only with an overall average across all students, but also for every student subgroup in the school with sufficient numbers of students.
The true proficiency rate requirements of NCLB – MUCH less than Table 1 shows
Even taken at face value, the required proficiency rates in Table 1 in the early years of NCLB were not terribly demanding, especially for mathematics. Sadly, potent loopholes in Kentucky’s implementation of NCLB make a mockery of even the first year proficiency rate requirements in Table 1.
Two of these key loopholes are discussed in detail in in CATS: An Inadequate NCLB Basis for School Improvement, a report by the Bluegrass Institute.
Number of students required for reporting
The first major loophole is that Kentucky requires excessively high numbers of students to be present in a subgroup before NCLB scores are reported for that subgroup. Thanks to this loophole, over 60 percent of the state’s elementary schools with African-American students were not held accountable for that racial group’s reading proficiency rate in 2004. Over 80 percent of the schools with learning disabled students used this rule to avoid accountability for those students, as well.
Confidence Intervals
A second major loophole involves abuse of a statistical process called “Confidence Intervals.” Confidence intervals are supposed to be estimates of the “plus or minus” error bands around each school’s reported proficiency rates. However, Kentucky pushes the theory far too hard, creating excessively large proficiency rate error bands. For example, in 2004 a total of 134 elementary schools escaped accountability for poor reading performance with their poor students thanks to this artifice. The report documents how some schools with small enrollment can have their actual proficiency rates boosted by as much as 50 points before Table 1 targets are applied. Thus, even in 2014, a small school’s actual proficiency rate might only be 50 percent, but after the confidence interval is applied, the school will be treated as though its rate were 100 percent.
Readers who wish to learn more about the minimum student number and confidence interval loopholes are encouraged to read CATS: An Inadequate NCLB Basis for School Improvement. The bottom line is that the minimum student number and confidence interval loopholes mean true proficiency rates in a school reported as meeting NCLB targets can be lower, sometimes far lower, than the rates shown in Table 1 in this report.
Safe Harbor
Another loophole makes tatters out of Table 1
If a school misses its AMO even after using the confidence interval and minimum student number loopholes, Kentucky provides yet another way to escape NCLB accountability with a provision, called “Safe Harbor.” As NCLB nears maturity in 2014, safe harbor is likely to become an important way for schools to escape accountability.
Under safe harbor, “The school must have
- (1) reduced by 10% the percentage scoring below proficient of any group that did not meet the AMO target in reading or mathematics AND
- (2) demonstrated improvement on the academic index (improve or equal 100 or more) for the same student groups.”[1]
If a school reduces the percentage of students not scoring proficient in math or reading and simply maintains its performance level on the other CATS academic tests, it will meet the overall academic requirement in item (2) above. Thus, this rule’s major impact comes from its very modest requirement for improvement for math and reading – merely hold stable on other academic tests and make only a 10 percent reduction in the non-proficient students each year in math and/or reading to escape NCLB sanctions.
How does this rule work in practice?
North Drive Middle School in Christian County provides an interesting safe harbor example. [Note: The source report, NO CHILD LEFT BEHIND, ADEQUATE YEARLY PROGRESS REPORT – 2008, August 27, 2008, North Drive Middle School, Christian County, can be accessed at the Department of Education website by entering “North Drive Middle School” in the “Type School Name” window and then clicking on the search button. Next, click on the “School NCLB Report” link that appears to the right of the school name at the bottom of the screen.]
North Drive Middle School’s actual math proficiency rate for African-Americans was 26.22 percent in 2008. That rate is well below the 2008 middle school target AMO of 37.37 percent from Table 1.
The school had too many African-American students to take advantage of the minimum student number loophole. Furthermore, even though this school got 9.05 “bonus” points from the confidence interval loophole, that still wasn’t enough to keep this school out of NCLB trouble in 2008 for African-American math performance.
However, thanks to safe harbor, North Drive was credited as meeting the African-American math AMO in 2008, anyway. In 2007, North Drive Middle’s African-Americans scored only 17.11 percent proficient. So, 82.89 percent of these students were not proficient in 2007. Under safe harbor, the school needed to reduce this number of non-proficient students in 2008 to 82.89 minus 8.289 percent,or 74.601 percent. That corresponds to a proficiency rate of 25.399 percent. In fact, as noted above, the school achieved a proficiency rate of 26.22 percent and was able to use the safe harbor loophole to avoid accountability for its African-American students in math.
The real impact of safe harbor is likely to become more apparent in the future. Consider how little improvement North Drive Middle must make with African-Americans to escape NCLB accountability in future years. Table 2 starts with North Drive’s 2007-08 proficiency rate of 26.22 percent. The Table then calculates the percentage of students below proficient in 2008 in North Drive Middle, which is 100 percent minus 26.22 percent, or 73.78 percent. Next, the table projects a 10% annual reduction in that non-proficient rate for each year out to 2014. As shown in Table 2, North Drive Middle can stay out of trouble in NCLB in 2014 with a proficiency rate of only 60.79 percent, a figure far below what the public would likely accept as an end goal and far below what the public seems to be promised in Table 1.
Summary
To summarize this discussion, NCLB proficiency rates in Kentucky have been inflated over time and bear no relationship to scores in other states. Furthermore, true requirements in Kentucky’s NCLB program involve proficiency rates well below those ostensibly presented to the public in Table 1. In fact, no school in Kentucky needs to post a 100 percent proficiency rate for any student group in 2014, thanks to the various loopholes discussed in this section. It may be possible for schools to avoid NCLB sanctions in 2014 with actual proficiency rates on the order of only 50 to 60 percent.
