Mississippi 1,660 vs. South Carolina 1,436 – Mississippi Wins SAT War!

1460

BOTTOM LINE: Correlation between State-level SAT average scores and test scores of representative (the average) students is almost ZERO. Knowing the state SAT average tells you virtually NOTHING about the quality of state education. 

SAT State Average scores are WORTHLESS for comparing states.

An old saying among South Carolina educators – thank heavens for Mississippi.  No matter the issue — test scores, teacher salaries, class size — Mississippi was always worst.

But the 2011 combined SAT scores (created by adding the math-reading-writing average for each state) show Mississippi at 1,660 points, South Carolina at 1,436 points and second from the bottom!  Once again, a big fail!??  Below is a sample of eleven states.  See the problem!  (Click here for the original state-by-state data).

State Combined SAT
Index
Illinois 1,807
Minnesota 1,778
Iowa 1,777
Kentucky 1,711
Tennessee 1,710
Mississippi 1,660
New Hampshire 1,559
Massachusetts 1,549
North Carolina 1,475
Florida 1,447
Georgia 1,445
South Carolina 1,436
Maine 1,391

How can we account for these differences?  Should we send a swat team to Mississippi to see what they are doing to have such high scores?  How does South Carolina always come out near the bottom?

But now see the same table of eleven states with an added column – the percentage of graduating high school seniors who took the test.  In all but a few states, taking the SAT is voluntary.  If your local colleges and universities don’t use the SAT results, why bother – unless you are well-to-do and applying to a selective school in the Ivy’s, on the West Coast, or such.  The percentage of graduating seniors taking the test varies from a low of 3% in North Dakota, 4% or so in South Dakota, Iowa and Mississippi, to almost 100% in Maine, Connecticut, and Massachusetts.

 

State

% Srs. taking

SAT Index 2011

Illinois

6%

1,807

Minnesota

8%

1,778

Iowa

4%

1,777

Kentucky

6%

1,711

Tennessee

11%

1,710

Mississippi

4%

1,660

New Hampshire

85%

1,559

Massachusetts

97%

1,549

Florida

75%

1,447

Georgia

81%

1,445

South Carolina

71%

1,436

Maine

98%

1,391

The new column gives an estimate of the percentage of graduating Seniors who took the SAT.  Mississippi had only 4% of their Seniors taking the exam, while South Carolina had 71%.  If you look at this subset of states, it appears that the best way to have a high average for your state is to have a smaller percentage of your students actually taking the test.

So Mississippi apparently played 4% of “varsity” scholars against our 70% or so range of more average students.  Are we surprised that the average varsity scholar in Mississippi does better on standardized tests than the average South Carolina student?  We prevent that from happening in high school sports by putting similar schools in similar divisions, and having most schools actually play – but remember for many states taking the SAT is voluntary and the percentage taking them is extremely uneven, ranging from 3% to almost 100%.

The takeaway — SAT State Average scores are WORTHLESS for comparing states. The typical test taker in some states can be far from average, part of the scholastic elite, or in other states, which require most students to take the SAT, truly average.

If someone tells you that South Carolina has a bad educational system AS DEMONSTRATED BY SAT scores, just remind them of how good Mississippi is!  And know that the correlation between these average SAT scores and a true measure of academic performance on the “Nation’s Report Card” is zero.  Knowing the state SAT average score tells you NOTHING about the performance on standardized tests taken by a representative sample of students in that state

So do we have a better way to compare the academic performance of states if the SAT comes up so short, because who takes it varies so much from state to state?

FAIR STATE COMPARISONS

To actually compare states fairly we need a representative sample of students in each state taking the same test.  The SAT does not meet that standard, but there is a common test given by the U.S. Department of Education that does exactly that.  A representative sample of fourth graders and eighth graders in EVERY state takes the same exam most years as part of The Nations Report Card.  Check out the table below that the same eleven states in our SAT table above.  These scores are taken from the 2010 National Assessment of Educational Progress (NAEP) scores.  I simply added the 4th grade reading and math scores to the 8th grade reading and math scores to create a simple index to compare states.

NAEP Index for the 11 states
in SAT order

State

NAEP index

Illinois

1,007

Minnesota

1,037

Iowa

1,013

Kentucky

1,017

Tennessee

981

Mississippi

962

New Hampshire

1,046

Massachusetts

1,065

Florida

1,004

Georgia

1,000

South Carolina

994

Maine

1,025

 

The SAT scores unfairly put Illinois on the top and Maine on the bottom of this short list.  Notice now that Illinois is only slightly above average, Massachusetts way up, and South Carolina is no longer second from the bottom: in this short table Tennessee is now second from the bottom, and in the table for all 50 states thirteen states have a lower NAEP index score than does South Carolina.  See the table below for these same eleven states sorted by this NAEP index.


NAEP Index for the 11 states
in NAEP order

State NAEP index
Massachusetts

1,065

New Hampshire

1,046

Minnesota

1,037

Maine

1,025

Kentucky

1,017

Iowa

1,013

Illinois

1,007

Florida

1,004

Georgia

1,000

South Carolina

994

Tennessee

981

Mississippi

962

 

Not only is South Carolina no longer at the bottom – for all fifty states 13 have lower NAEP index scores than does South Carolina and another 3 states are only slightly higher.  So South Carolina is essentially tied for 35th in the nation (rather than 49th )..

One factor we know greatly influences a state’s  performance on standardized tests is social class or income level.  A smaller percentage of children in poverty typically lead to higher test scores.  The table below shows the same eleven states with scores arranged by the NAEP index, and a new column showing an estimate of the number of children living in poverty using data from the US Census Bureau.  Note that this measure is only for children at and below the official poverty line, and does not include reduced lunch children who are slightly above the official family poverty line but still qualify for lunch aid and often for Medicaid insurance.  These then are the poorest of the poor, and not also the near poor often included in the “free and reduced lunch” measures used in South Carolina.

 

 

Our 11 sample states with scores arranged by the NAEP Index,
with a column showing the Poverty Index (% children below poverty line)


State

NAEP
Index

Poverty Index

Massachusetts

1,065

14%

New Hampshire

1,046

10%

Minnesota

1,037

15%

Maine

1,025

18%

Kentucky

1,017

26%

Iowa

1,013

16%

Illinois

1,007

19%

Florida

1,004

23%

Georgia

1,000

25%

South Carolina

994

26%

Tennessee

981

26%

Mississippi

962

33%

 

Notice that as the poverty measure increases, the NAEP index decreases.  The average test score in poorer states is lower than in more well-to-do states.  (Though not covered here, this same impact can be seen between nations.  Since the U.S. has considerably more childhood poverty than other well-to-do countries, we often have lower average test results.)

But the small table I used for illustration left out 39 states.  The table below gives the NAEP index score and poverty rate for ALL thirteen states with a lower NAEP index than South Carolina.


 

States below South Carolina in the NAEP 2011 Combined Index
with information on the % under 19 in poverty

State

Comb 4th-8th grade Reading Math NAEP

% under 19 in Poverty

South Carolina

994

26%

Arkansas

992

28%

Oklahoma

992

25%

Alaska

989

13%

Hawaii

988

14%

Arizona

987

24%

Nevada

986

22%

Tennessee

981

26%

Alabama

979

28%

West Virginia

978

25%

California

973

22%

New Mexico

971

30%

Louisiana

969

27%

Mississippi

962

33%

 

Surprised?  On a test that measures a representative sample of students, South Carolina is not second from the bottom, but actually does better than many states with a lower poverty rate (and not just Mississippi, which is the poorest state by this poverty index measure).  It is simply not true that South Carolina ranks near the bottom of all states in educational achievement, despite the rhetoric that suggests our low SAT scores show the state schools do a poor job.  If we want higher SAT average scores, we might discourage local colleges and universities from using the SAT for admission so only kids going out of state would bother to take the test – we could aspire to get the percentage taking the SAT test down to under 10% and have as high an SAT score as Mississippi does.

 

 

 

Technical detail – fuller data for all 50 states, correlations and regressions

At the end of this section is a table with the data used for all 50 states.  To summarize the relationships, analysts often use simple descriptive statistics.

It turns out that there is a very close relationship between the percentage of High School graduates taking the SAT test and the average state-level SAT score.  The chart below shows a scatterplot for each state, with each blue point showing the percentage taking the SAT on the vertical axis, and the same state’s average SAT score on the horizontal axis.

CHART 1:  For each state, the Percentage taking the SAT (vertical axis)
by the Mean Composite SAT score (horizontal axis)

I labeled several of the points by the state name.  All the labels in black (like HI, NV, CA, AK, MS and TN) are states that have a lower NAEP index score than South Carolina.  Notice that many of them have a much higher SAT score than the SC 1,436 (like Arkansas with 1,692, Oklahoma with 1,683, and Mississippi with 1,660).  Other state, like Maine, New Jersey, Connecticut, Massachusetts, and New Hampshire, had lower SAT scores than these but much higher NAEP index scores.  Especially noteworthy, Maine had the lowest SAT average in the country at 1,391, but on the NAEP index finish in the top third of states

Looking at this chart should convince one that there is a strong relationship between the state SAT scores and the percentage of Seniors taking the SAT.  We can use descriptive statistics to formalize the strength of that relationship.

The solid line in Chart 1 is the best-fitting least-squares line, also known as the linear regression line.  The equation for that regression line is:

Predicted SAT = 1715 – 2.9 * % taking SAT

Some examples:

If 98% take the SAT, the predicted SAT score is 17.15 – (2.9 * 98%) = 1431

Actual states near this percentage, ME at 1,391, CT at 1,535 and MA at 1,549.

For 50% taking, the predicted SAT score is 1570 (a lower percentage predicts a higher SAT)

Actual states near this percentage, NV at 1,460, OR at 1,540 and AK at 1,513

For 3% taking the predicted SAT score is 1706.

Actual states near this percentage, SD at 1,737, MS at 1,660, and ND at 1,759

The Pearson correlation R = 0.87, the R-squared (variance explained) is 0.76

Check the table and graph to see how close the predictions come to the outcome.  So once one knows the percentage of seniors taking the SAT, there is not much left to explain.  Just that fact alone will give you a good estimate of the SAT result, even for a very poor state like Mississippi.  And knowing this, you are not likely to be taken in by those who claim the South Carolina SAT results show something significant about how South Carolina compares educationally to other states, except that we sure have a lot of kids taking the SAT, and the Princeton Testing Service who charges for the test sure must love South Carolina (and not Mississippi).

The second chart shows the scattergram for all 50 states for the NAEP index vs. the children’s poverty rate.

In the formal statistical analysis of the second chart, the formula for the linear line is:

Predicted NAEP index = 1075 – 3.22 * % Poverty

R= 0.73 and R-squared = 0.53.  So over 50% of the variance can be explained just by the poverty rate.

This correlation is lower than the one for SAT vs. percentage taking.  For example, in the lower left portion of the chart you see Hawaii and Alaska, with relatively lower poverty rates, are doing less well on their average than is South Carolina, while others (like MA, the highest on the Y-axis) do better than expected.  Figuring out why would be interesting.  While it might be interactions with ethnic identities that have little to do with actual school programs, some of the remaining differences might be explained by differences in actual school organization and instruction methods.  Looking for “outliers,” for exceptions to the general rule and understanding why these are exceptional can advance the state of understanding.

Some example calculations and states close to the predicted results:

At 10% poverty, the Predicted NAEP = 1075.  States close to this poverty level and their actual score — NH 1046, MA 1065, CT 1032

At 21% poverty, the predicted NAEP = 1007.  States close to this poverty level and their actual NAEP index scores — MO 1010, NY 1006, IN 1014

At 33% poverty, the Predicted NAEP = 969.  States close to this poverty level and their actual NAEP index scores — MS 962, NM 971, Alabama 979

The table below shows the data for all 50 states, sorted by the NAEP index (which was the sum of the 4th grade and 8th grade 2010 Math and Reading scores for each state).  Notice that the relationship between the state-level SAT scores and the NAEP index jumps all over the place.  In fact the formal linear correlation between the SAT and the NAEP index is 0.03 (R-squared = 0.00).  Knowing the SAT score tells you essentially nothing about the likely NAEP index score.

Full 50-state data

Key Data:  50 States with % Sr taking SAT, combined SAT 2011, NAEP 2010 index, % 18 and under classified as in poverty.  Sorted by NAEP Index high to low
Several southern states shown in bold


State

% Sr. taking SAT


SAT 2011


NAEP Index

Children in Poverty

Massachusetts

97%

1,549

1,065

14%

New Jersey

90%

1,508

1,048

14%

New Hampshire

85%

1,559

1,046

10%

Vermont

72%

1,538

1,041

17%

Maryland

83%

1,492

1,037

13%

Minnesota

8%

1,778

1,037

15%

Montana

27%

1,592

1,035

20%

Connecticut

97%

1,535

1,032

13%

North Dakota

3%

1,759

1,031

16%

Colorado

20%

1,699

1,031

17%

Virginia

78%

1,516

1,028

14%

Kansas

7%

1,734

1,027

18%

Pennsylvania

86%

1,473

1,027

19%

Wyoming

5%

1,692

1,026

14%

Maine

98%

1,391

1,025

18%

Ohio

23%

1,606

1,025

23%

Wisconsin

5%

1,767

1,022

19%

South Dakota

4%

1,737

1,021

18%

Washington

61%

1,560

1,019

18%

Kentucky

6%

1,711

1,017

26%

Idaho

21%

1,598

1,016

19%

North Carolina

77%

1,475

1,015

25%

Delaware

93%

1,455

1,014

18%

Indiana

75%

1,469

1,014

22%

Nebraska

5%

1,745

1,014

18%

Utah

6%

1,667

1,013

16%

Iowa

4%

1,777

1,013

16%

Rhode Island

82%

1,477

1,012

19%

Texas

64%

1,446

1,011

26%

Missouri

5%

1,764

1,010

21%

Illinois

6%

1,807

1,007

19%

New York

95%

1,460

1,006

21%

Florida

75%

1,447

1,004

23%

Oregon

52%

1,540

1,001

22%

Michigan

5%

1,760

1,000

23%

Georgia

81%

1,445

1,000

25%

South Carolina

71%

1,436

994

26%

Arkansas

5%

1,692

992

28%

Oklahoma

6%

1,683

992

25%

Alaska

47%

1,513

989

13%

Hawaii

78%

1,448

988

14%

Arizona

40%

1,539

987

24%

Nevada

57%

1,460

986

22%

Tennessee

11%

1,710

981

26%

Alabama

9%

1,622

979

28%

West Virginia

17%

1,512

978

25%

California

58%

1,513

973

22%

New Mexico

13%

1,618

971

30%

Louisiana

8%

1,651

969

27%

Mississippi

4%

1,660

962

33%

 

NO COMMENTS

LEAVE A REPLY