ACT College Entrance Test Results for 2009 High School Graduates

From FreedomKentucky
Jump to: navigation, search

The ACT, Incorporated, released its annual report on test performance for the High School Class of 2009 on August 19, 2009. For the first time, 100 percent of Kentucky’s graduates took the ACT, which makes score comparisons both to other states and to Kentucky’s past performance problematic. This section will explain some of the issues and present the state’s 2009 high school graduates' performance in context. Another section of the Wiki deals with the separate ACT report for Kentucky's 11th grade testing, which is an entirely different data set.

Getting StartedTakeActionButton.png

Why is the ACT important?

The ACT, Incorporated has a lot of information to answer this question.

It is obvious that the ACT has importance for students who wish to further their educations. For example, the ACT news release accompanying the new 2009 scores says, “Years of empirical ACT data indicate that students who meet or surpass the College Readiness Benchmarks are more likely than those who don’t to go to college, stay in school and graduate with a college degree.”

The news release explains that those Benchmark scores indicate “a student is ready to succeed (50 percent chance of earning a “B” or higher or about a 75 percent chance of earning a “C” or higher) in a typical first-year, credit-bearing college course in that subject area.”

Thus, the ACT provides an important indication to students of their readiness to enter, and successfully complete, postsecondary studies.

The implications of the ACT range far beyond the impact on individual students, as well. The ACT release points out that, “It has been widely argued that greater numbers of students need to gain advanced math and science knowledge to keep the U.S. labor force competitive.”

Even former Kentucky Commissioner of Education Gene Wilhoit recognizes the importance of the ACT. The ACT news release quotes him saying, “We applaud ACT for showing where we need to elevate state standards.”

As will be seen below, that is certainly true in Kentucky.

ACT participation rates vary dramatically in different states

An ACT information sheet titled Average ACT Scores by State, Graduating Class of 2009 indicates that the percentage of 2009 graduating students who took the ACT in each state varies dramatically. For example, in Kentucky and the states listed in the following table, participation rates on the ACT for 2009 graduates are very high.

Percent of Graduates Tested in High Participation States From Average ACT Scores by State, Graduating Class of 2009

PCT Grads Tested.jpg

In sharp contrast, some states had very low participation rates. Only nine percent of Maine’s graduates in 2009 took the ACT, as the SAT is the predominant test in that part of the country.

Different participation rates mean simplistic comparisons of scores between states is inappropriate

In general, in states where use of the SAT college entrance test predominates, ACT participation runs less than 30 percent, and is often less than 20 percent. In these cases, the ACT scores are not based on any sort of uniform student sample and cannot be simplistically compared from state to state.

In fact, the ACT warns in its information sheet that,

“The accompanying list of state average scores should not be interpreted as providing grounds for an explicit or implicit ranking of the various states' educational systems. In most states, students who take the ACT are self-selected and do not represent the entire state's student population (see notice below). Further, the percentages of students taking the ACT vary a great deal from state to state, as do those students' backgrounds and characteristics.”

This warning is routinely ignored by a number of governmental and private education research organizations in Kentucky.

Different student demographics in different states also complicate analysis

In addition to the interpretation issues created by widely varying participation on the ACT from state to state, another complicating issue is the fact that student demographics also vary widely across the country.

This next table shows the percentage of White students in states where there is high participation on the ACT (more than 90 percent participation).

Percent of White Students in High Participation States From Table 1.5 in Each State's Scale Score Report (ACT Profile Report for 2009)

PCT White.jpg

Notice that Kentucky has a notably higher percentage of Whites than is found in the other states in the listing. Because Whites tend to score notably higher than the minorities, a state with a high proportion of Whites in the tested group gets a significant advantage when ACT analysis is limited to only a comparison of overall average scores for all students.

This is similar to the problem identified by Richard Innes for test results from the National Assessment of Educational Progress, which is discussed in this Wiki item.

One way to work around this problem is to examine disaggregated scores for the different racial groups, as discussed below.

On paper, the percent of Kentucky students taking college prep courses looks good

Kentucky also should enjoy another advantage when ACT performance is analyzed. Compared to most of the other high participation rate states, a notably larger proportion of Kentucky graduates reported taking the minimum core course load that the ACT recommends for college preparation. Only Tennessee had a higher proportion of students taking college core courses.

Percent of 2009 High School Graduates in High ACT Participation States That Took the ACT's Core or More Course Load in High School From Table 1.4 in Each State's ACT Profile Report for 2009

PCT Taking Core or More.jpg

ACT analysis is still valuable

While ACT analysis isn’t as straight-forward as some would like to believe, because Kentucky is now one of the states that tests virtually all of its students with the ACT, it is possible to do some comparisons to the performance in those other high participation rate states. Here are some specific examples of that sort of analysis.

Don’t say “Thank goodness for Mississippi!”

For many years, whenever Kentucky’s public education system was discussed, someone would point out that Mississippi had even lower test scores. In fact, as the next table shows, if you only look at the overall average score for all students, Kentucky’s 2009 graduates did outscore those in Mississippi, 19.4 to 18.9.

However, when the data is disaggregated by race, a very different picture emerges. When the disaggregated data is examined, Mississippi outscores Kentucky on the ACT for every reported racial grouping.

This is a remarkable situation, which points out the necessity to do more than simplistic comparisons of overall test scores when state to state comparisons are being conducted.

ACT Composite Scores for Kentucky and Mississippi 2009 Graduates by Race/Ethnicity From Table 1.5 in Each State's Scale Score Report (ACT Profile Report for 2009)

KY - MS Table.jpg

In the comparison above, it should be noted that Mississippi tested 93 percent of its graduates while Kentucky tested all of its graduating class. That might give Mississippi a small advantage in some of the specific score examples above, but not very much. Since the ACT report has no information about the small number of Mississippi 2009 graduates that were not tested, there is no way to further refine the data.

On the other hand, it must be recalled that the Mississippi graduates of 2009 were in their early high school years when the Magnolia State received extensive damage from Hurricane Katrina. Much of the infrastructure in the Southern part of the state was devastated. The widespread destruction included many schools that were either heavily damaged or totally destroyed. The students involved experienced significant upheaval during their remaining high school years as the state struggled to rebuild and regain population.

Why is Mississippi pulling ahead of Kentucky?

This is a question that requires more research, but a few things can be noted.

Until this year, Mississippi was one of 40 states that had charter schools. While that law temporarily lapsed in 2009, it is reported that Mississippi will re-enact it in 2010.

One key area – funding – does not explain the Mississippi phenomenon. Trends in current expenditures per pupil in Kentucky and Mississippi for the 1987 to 2006 period were recently investigated by Dr. William H. Hoyt, Dr. Christopher Jepsen and Dr. Kenneth R. Troske in Educational Spending: Kentucky vs. Other States. These researchers from the University of Kentucky’s Department of Economics found that over this period,

Kentucky’s current expenditures grew at a much faster pace that the other three East South Central states (Alabama, Mississippi, Tennessee). By 2006, Kentucky spent nearly $1,000 more per pupil than each of the other three states. In that same year, Kentucky spent marginally more than the four West South Central states. Closer inspection of the trends…show(s) that in 1991, after the passage of KERA, Kentucky’s expenditures were second only to Texas. In subsequent years, Kentucky continued to spend more than all of the states in the South Central region, including Texas.

More recent data from Table 11 in the US Census Bureau’s Public Education Finances 2007 report indicate that in the 2006-07 school year total per pupil elementary-secondary school revenue in Kentucky was $9,630 while in Mississippi it was only $8,523, more than $1,100 less. Clearly, this extra per pupil expenditure of 13 percent did not enable Kentucky’s students to prosper in comparison to Mississippi on the ACT.

Comparison of overall student scores for high participation rate states

The series of graphs below were derived from each state's profile report and compare overall student scores by subject for the high participation rate states. Notice that in all cases Kentucky tends to score fairly low. However, keep in mind the earlier discussion with the Mississippi example that shows how Kentucky’s apparent advantages disappear when scores are disaggregated. While not shown here, a comparison of the data in each state’s ACT Profile Report indicates that on only a few cases does any Kentucky racial group outscore their counterpart racial groups in the other high participation rate states.

All Stud Composite.jpg
All Stud English.jpg
All Stud Math.jpg
All Stud Reading.jpg
All Stud Science.jpg

Comparison of similar racial group performance

This graph shows the Composite Score performance for White students only in the high participation rate states. With Whites comprising nearly four of five students in the Kentucky graduating class of 2009, the fact that they do notably poorer in Kentucky than Whites in all the other states shown is especially problematic for the future of the commonwealth in the rapidly more technical economy of the 21st Century.

White Composite.jpg

Finally, the next graph shows Composite Score data for African-American/Black students. Note that only Michigan, where the economy has been devastated by the collapse of the American automobile industry, has lower scores.

Black Composite.jpg

Comparison of ACT results for public schools versus private/home schools

This graph shows how the public school only ACT scores for Kentucky's 2009 high school graduates compare to the scores for students who were educated in other schooling systems in the state.

The public school only scores are found in the Kentucky Department of Education's ACT News Release.

The private school scores were algebraically calculated using student count and score data for public schools found in the Kentucky Department of Education News Release and the overall scores for all students found in the 2009 ACT Profile Report for Kentucky.

Public Versus Private.jpg

Of the 45,419 Kentucky graduates who took the ACT, 41,099 come from the public school system, and this number represents 100 percent of the public school graduates. Another 4,320 graduates who took the ACT came from other schooling arrangements, including Kentucky home school students.

While there are no highly reliable data on the total number of non-public school graduates, the most recently available data in Digest of Education Statistics 2008, Table 62 indicates that Kentucky had 3,720 private school high school graduates in 2004-05. Home schools may add about 1,000 more to this number (there is no current count of home school students - estimate based on old reports) for a rough estimate of 4,720 students, which would imply that roughly around 92 percent of the students in non-public schooling programs in Kentucky do take the ACT, which is relatively close to a universally tested sample.

Clearly, private school graduates enjoyed a considerable education advantage in the Bluegrass State.

Are Kentucky students ready for college?

Several years ago, the ACT, Incorporated conducted empirical studies to determine what ACT score in each subject corresponded to reasonable odds of success in college. Known as the ACT Benchmark Scores, these scores indicate a student has about a 75 percent chance of earning a “C” and a 50 percent chance of getting a “B” in the first corresponding college course.

The next table summarizes the percentages of students meeting the ACT Benchmark scores in each state. The ACT reports that fewer than one student in six in Kentucky is being fully prepared for a well-rounded liberal college education across all four academic areas. Only Mississippi does a poorer job, but as previously discussed, that state has a much higher proportion of minority students to educate, and these traditionally have presented more challenges than White students.

Percent of 2009 Graduates Meeting ACT Benchmarks, States with High ACT Participation Rates From Table 1.5 in Each State's ACT Profile Report for 2009

Benchmark Table.jpg

It is interesting to point out in this table that Mississippi does a better job of preparing its graduates for college English than Kentucky. Given the extreme emphasis on writing in Kentucky since the Kentucky Education Reform Act was enacted in 1990, that statistic is particularly disappointing.

Kentucky’s relative weakness in math and science is also troubling, as these are critical courses for the high technology jobs that provide the best livelihoods in the new century.

The ACT has both proponents and critics

While the ACT college entrance test has an established record of value in predicting college performance, it is not without critics.

Some of those critics dislike all multiple-choice tests, which is the format of the ACT, so this group dismisses the ACT out of hand. Other critics assert the ACT can only assess such lower level skills such as wrote memory and the simpler mathematics operations.

Critics also say that no test is 100 percent accurate, and no single test can evaluate every facet of an educational program.

Those claims are countered by ACT proponents. Proponents point out that the state of the art in multiple-choice testing has advanced and now it is possible to write questions using this format that do tap into higher order thinking.

Proponents point out that every college in Kentucky, including the entire public postsecondary system, finds the ACT of sufficient value to use it for both admissions and certain placement decisions concerning remedial course requirements. In addition, the ACT weighs heavily in the awards of Kentucky’s public college scholarships, which is an indication that the legislature also believes the assessment has credibility.

Without question, the ACT enjoys a much higher level of confidence than Kentucky’s CATS assessments. While the ACT continues to be used in many important ways in Kentucky, and will continue as a portion of the state’s revamped public school assessment program, the CATS has been officially disbanded due to growing mistrust of what now were obviously inflated scores that failed to paint an accurate picture of school progress.

Thus, taken together, the evidence on the ACT is that it is an important assessment with a good track record of predicting college performance. And, with the new economy focusing more attention on the idea that college preparation needs to be the dominant goal of our school system, the ACT seems like the best currently available measure to see if our schools are truly meeting the needs of the state and its children.

What ACT says we can do to improve our schools

The ACT News Release mentions a number of areas that Kentucky should consider as it reworks our education standards and assessment system as required by Senate Bill 1 of the 2009 Regular Legislative Session. These include:

• Adopt fewer—but essential—college and career readiness standards as their new high school graduation standards.

• Adopt a rigorous core curriculum for all high school graduates, whether they are bound for college or work.

• Define “how good is good enough” for college and career readiness.

• Strengthen the rigor of their courses.

• Begin monitoring academic achievement early to make sure younger students are on target to be ready for college and career.

• Establish longitudinal P-16 (preschool through college) data systems.

This is “news we can, and should, use.”

Scores from 11th Grade ACT Testing in Kentucky

A separate series of ACT test scores is also available for all Kentucky high school Juniors who took the ACT in March 2009. That information is available in this separate Wiki Item.

These 11th grade results generally represent the first time the students took the ACT. The 11th grade scores are lower than the graduates' scores reported above because graduates often take the ACT again in the fall of their senior year.

Furthermore, scores for graduates are higher because the ACT retains the highest scores if a student tests more than one time, and because those who retake the ACT as Seniors have spent more time in advanced high school courses and have more experience with the ACT itself.