ACT’s Benchmark Scores and College Degree Awards in Kentucky

From FreedomKentucky
Jump to: navigation, search


Getting StartedTakeActionButton.png

What is the relationship between the number of Kentucky students who score at or above the ACT, Incorporated’s “Benchmark Scores” and the number of two-year and four-year degrees that are awarded in Kentucky’s college system?

This Wiki item explores answers. The base Wiki was assembled by Richard Innes, the education analyst for the Bluegrass Institute For Public Policy Solutions.

Briefly, ACT’s Benchmark Scores represent performance that indicates the test taker has good odds of earning passing grades in the first related college course. As such, these scores are clearly of considerable importance to students, parents and teachers.

While the ACT Benchmark Scores are only intended to show the likelihood of passing the first related college course (e.g. for ACT Science, that course is freshman biology), it also seems reasonable to expect a decent correspondence between the benchmark performance and later awards of closely related degrees.

That leads to the next question.

How do Kentucky’s ACT Science Benchmark results relate to awards of Science, Technology, Engineering and Mathematics (STEM) degrees?

This graph shows the relationship between the number of students who get scores at or above the ACT Science Benchmark Score of 24 and the number of students who later earn degrees in Science, Technology, Engineering and Mathematics, often called the “STEM” subjects.

ACT Science Benchmarks Vs STEM Degree Awards Graph.jpg

The blue lines show the total number of STEM degrees awarded to the same cohort of students. For example, the 2008-09 line includes the number of Bachelors’ Degrees awarded in STEM areas to Kentucky students at the end of that school term plus the number of STEM Associates’ Degrees awarded two years earlier.

The reference years listed on the graph are for the on-time four-year degree graduation year for each student cohort. For example, students listed on the 2005-06 year line graduated from high school in 2001-02 and earned on-time Associates’ Degrees at the end of the 2003-04 term and on-time four-year degrees at the close of the 2005-06 term.

It is clear the agreement between Science Benchmark results and later awards of college degrees in the STEM areas is quite good. The disagreement amounts to only about one out of ten STEM degrees awarded. For social statistics, that is very good agreement.

Although there are limitations in the available data and in the assumptions used to create this graph (discussed below), the close agreement between benchmark scores and later awards of STEM degrees adds confidence to the earlier benchmark research from the ACT.

Overall, interested stakeholders in Kentucky can be reasonably confident that the ACT Science Benchmark Score generally works as intended and probably provides an additional benefit of indicating the likelihood of eventually earning a STEM-related college degree.

Such information is clearly of importance to Kentucky’s students, parents and educators.

Deriving the graph

This discussion references data for the high school graduating cohort of 2001-02 as an example to show how the graph was created.

Determining the number of students achieving ACT Science Benchmark Scores

Over the years paper copies of annual ACT reports on Kentucky performance (not on line) have been obtained by Richard Innes from the Kentucky Department of Education. These reports are titled, "ACT High School Profile Report, H S Graduating Class xxxx, State Composite for Kentucky, Code 180-000," where xxxx is the graduation year (e.g. for the 2001-02 year, xxxx is 2002). The data includes results for all Kentucky students, including public school, private school and home school students.

Innes is the Education Analyst for the Bluegrass Institute for Public Policy Solutions.

{Note: More recently, the ACT, Incorporated has published similar reports for more recent high school graduating classes on line. For example, the “ACT Profile Report – State, Graduating Class 2009, Kentucky” is available here. In these newer reports, data formerly published in Table 4 is now found in “Table 2.1. ACT Score Distributions, Cumulative Percentages (CP1), and Score Averages.”}

Examination of data in Table 4 from the various profile reports shows the ACT reported that 5,931 Kentucky high school graduates in the 2001-02 school year scored at or above the ACT Science Benchmark score of 24.

Data for all the high school graduation years covered by the graph are summarized in this table.

Number of Science Benchmark Scores Table.jpg

Deriving the STEM degree award data

Data on the STEM degrees awarded by year for Kentucky’s two- and four-year programs comes from the Kentucky Council on Postsecondary Education (CPE) and is also currently not on line.

This data was analyzed by Richard Innes to remove all known degree awards to public university students from out of state. It was not possible to eliminate degree awards made to out of state students by the independent Kentucky colleges as that data is not available at the CPE.

STEM Degree Awards Table.jpg

Continuing the discussion of the high school cohort of 2001-02’s college experience, two years after their high school graduation, at the end of the 2003-04 term, the Kentucky public and independent colleges awarded a total of 2,037 STEM-based Associates’ Degrees to this group.

Then, two more years later in 2005-06, the cohort received a total of 4,637 STEM-based Bachelors’ Degrees. Thus, this cohort earned a total of 6,674 STEM-based degrees, as shown under the 2005-06 four-year college graduation reference year for this cohort.

Limitations of the analysis

While this study is about the best that can be supported by currently available data, it is far from perfect.

Ideally, the study attempted here would be based on data from a high quality student tracking program. That database would track each student through the entire education experience, accurately recording test scores such as those from the ACT along with eventual awards of high school diplomas and college degrees. Then the data on all the individual students would be compiled to show overall performance for each graduating class.

Sadly, 20 years after the passage of KERA, such a database does not exist in Kentucky, though one is now in development. The best we can currently do is a cohort-oriented study that makes some important assumptions.

For example, the study discussed here assumes that all Kentucky high school graduates who scored at or above the Science Benchmark Score went on to college in Kentucky. Clearly, that isn’t correct. However, there isn’t any publicly available information on the proportion of Kentucky students who earned high ACT scores and then went to college elsewhere. This issue means the number of Kentucky students earning at or above the ACT Benchmark score reported here is somewhat too high.

On the other hand, not knowing the proportion of the STEM degree awards from Kentucky’s independent colleges to out of state residents creates a small error in the opposite direction. Out of state residents who earn STEM degrees from Kentucky colleges should not be counted in the graph at the beginning of this Wiki, either, although only the degrees awarded at Kentucky public colleges to out of state residents was corrected for in this study.

Most importantly, this study assumes that all the graduates from a given high school class earn all of their two-year degrees two years later and all of their four-year degrees four years after high school graduation. That also isn’t correct. Some students take as long as six years or more to earn a Bachelors’ degree. However, those delayed degree awards are compensated for to some degree by degrees awarded to other students who are not from the reference high school graduating class but never the less earn degrees in the reference years. The full impact of this issue cannot be resolved absent the currently unavailable high quality student tracking system.

Even given the various assumptions above, it still is apparent that the overall correspondence between the number of students scoring at or above the Science ACT Benchmark and the later award of STEM degrees is noteworthy.

More Information

ACT-based tests are important in Kentucky

The landmark passage of Senate Bill 130 during the 2006 Regular Legislative Session made Kentucky the first state in the country to adopt 100 percent testing of all public school students with the full EPAS test system from the ACT, Incorporated. EPAS includes the EXPLORE test given in eighth grade, the PLAN test for tenth grade, and testing of all Kentucky eleventh grade students with the ACT college entrance test.

Results from these tests provide useful feedback to parents, students and teachers about student preparation for postsecondary education. In addition, the ACT is an important part of the college entrance process for all Kentucky postsecondary institutions and good scores on the ACT entitle Kentucky students to higher levels of support from the Kentucky Excellence in Education Scholarship program.

Some of information provided in the EPAS test results in Kentucky includes information about how students perform against the ACT College Readiness Benchmark Scores.

What are the ACT Benchmark Scores?

The ACT answers that question in a document titled “What Are ACT’s College Readiness Benchmarks?” This document says:

“ACT’s College Readiness Benchmarks are the minimum ACT test scores required for students to have a high probability of success in credit-bearing college courses—English Composition, social sciences courses, College Algebra, or Biology. In addition to the Benchmarks for the ACT® test, there are corresponding EXPLORE® and PLAN® Benchmarks for use by students who take these programs to gauge their progress in becoming college ready in the eighth and tenth grades, respectively.”

The ACT publication further explains:

“Students who meet a Benchmark on the ACT…have approximately a 50 percent chance of earning a B or better and approximately a 75 percent chance or better of earning a C or better in the corresponding college course or courses. Students who meet a Benchmark on EXPLORE or PLAN are likely to have approximately this same chance of earning such a grade in the corresponding college course(s) by the time they graduate high school.”

The ACT benchmark scores for the three EPAS tests are listed in this table:

ACT Benchmarks Table.jpg

It should be pointed out that the Benchmark scores are designed to provide an idea of how well a student is likely to do in certain freshman college courses.

The ACT Benchmark Score Controversy

In early March 2010, The Prichard Committee for Academic Excellence, a Kentucky education advocacy group, attempted to raise doubts about the ACT Benchmark Scores.

Using an approach somewhat similar to the one discussed in this Wiki item, that group posted a poorly chosen comparison between the numbers of students achieving the ACT Benchmark Score in science versus the total number of all two- and four-year postsecondary degrees awarded in Kentucky. The most significant problem with Prichard’s analysis is that many two- and four-year degrees require little, if any, science. In consequence, it is far more appropriate to compare the Science ACT results to the STEM degree awards, something that apparently didn’t occur to Prichard.

Prichard’s comparison made it appear that the ACT Science Benchmark was set far too high. The advocacy group published a graph showing the number of two- and four-year degrees awarded by Kentucky colleges was around double the number of students who were achieving the ACT Science Benchmark score.

Prichard’s case isn’t supported by the more thorough investigation reported in this Wiki post. When ACT Science Benchmark data are fairly compared to appropriately related college degrees, it looks like the results for Kentucky students correspond quite well.

See Also