April 1993
Number 4
Leah M. Fay
Alison P. Kelly
David G. Tuerck
Choice or Chance:
The Hard Economics of Educational Empowerment
Good intentions have once again gone awry in the Massachusetts legislature. Nearing passage is an education bill that mortgages the Commonwealth with $5 billion in future taxes and no guarantee of commensurate benefits to Massachusetts school children. The bill embodies the chance hope that more spending by itself will improve the performance of Massachusetts schools.
Two versions of the Massachusetts Education Reform Act, House 1000 and Senate 1550, are currently under debate in a conference committee. Both versions strive to improve school administration. Both also provide for an increased financial commitment by the state to education funding that will reach $1.3 billion per year by the year 2000. Debate is centered on a "mandatory choice" component of the Senate version.
Proponents of both versions stress the importance of education reform for making high-quality education more accessible to economically-disadvantaged children. House sponsor Mark Roosevelt claims that the act is in "the best interests of the state and the best interests of all children."
There is a need for education reform: Seventy-five percent of the students taking a test administered under the Massachusetts Educational Assessment Program (MEAP) failed to demonstrate "minimum" competency in their performance.
What is unclear is whether there is a need for more education spending, absent any guarantee that measured competency will in fact improve. A key assumption underlying both bills is that the state can improve the performance of "underfunded" schools simply by giving them more money.
Our analysis shows, however, that the recommended increase in state spending will not improve the academic performance of more than 90 percent of those students whose performance is most in need of improvement. This analysis assumes, moreover, the existence of an "ideal" world in which spending alone determines academic performance. In reality, many socioeconomic factors relating to such matters as community income and crime rates are more important. Once we expand our analysis to take these factors into consideration, the proposed increase in spending turns out to have no effect at all on academic performance. In statistical terminology, spending becomes "insignificant" as an explanatory variable.
MEAP Scores
Massachusetts public schools administer the MEAP test to 4th, 8th and 12th-grade students. The test evaluates students according to their answers to a combination of open-ended, essay and multiple-choice questions. MEAP reports the fraction of students taking the test who, on the basis of their performance, fall into the following categories:
- Testing Education Spending and Performance
below Level 1 - students respond to very few multiple-choice questions and cannot interpret or respond to open-ended questions;
Level 1 - students can respond to simple familiar material that is presented in a highly structured format, but fail to recognize the requirements of unfamiliar tests;
Level 2 - students have mastered the basic components of the grade-appropriate curriculum; however, when asked to generate their own responses, their answers seldom go beyond the minimally acceptable and may indicate major misconceptions;
Level 3 - students have mastered the underlying principles of the grade-appropriate curriculum and, in addition, can reason and communicate clearly and can apply their knowledge in a variety of contexts;
Level 4 - students have attained a broad base of knowledge beyond their curriculum; their analytic ability is sophisticated for their age level, as is their ability to communicate their reasoning.
- A crucial issue in the debate over education reform is whether more money will actually improve student performance on standardized tests. Last year, BHI published a study that demonstrated a weak or insignificant relationship between education spending and performance on the Scholastic Aptitude Test (SAT). 1 In order to shed further light on this issue, we have performed a statistical analysis of the relationship between education spending and performance on the MEAP test. (See appendix.) In performing this analysis, we adopted the technique of allowing only one factor - education spending - to influence student performance, a technique that deliberately biases the results to show a positive relationship between spending and performance.
Our analysis estimates the effect on student performance of an increase in per-student education spending of $1,310 (the maximum amount of additional state aid per student mandated by House 1000). There are two phases to this analysis. In phase 1, we estimate the effect of the additional spending on students whose 1992 MEAP scores place them at or below level 1. In this phase, we estimate the number of these students who can be expected to move up to levels 2, 3 or 4 as a result of the new spending, and we estimate the cost to the state of each "improvement" thus achieved. This phase of the analysis aims to show the likely effectiveness of the proposed new spending for improving the performance of poor (level 1 and below level 1) students.
In phase 2, we estimate the number of students with MEAP scores placing them at level 2 or below who can be expected to move up to levels 3 and 4 and the cost per improvement. This phase of the analysis aims to show the likely effectiveness of the new spending for turning poor or average (level 2 and below) students into superior (level 3 or 4) students. The results are as shown in Table 1.
Table 1
Effects of New Spending on Academic Achievement
Grade a Phase Number of students at each level before new spending Number of students who improve (move to a higher level) Percent of students who improve (mover to a higher level Cost per improvement 1 27,060 2,307 8.5% $15,366 4 2 52,140 2,033 3.9 33,597 1 26,400 2,016 7.6 17,155 8 2 43,800 1,818 4.2 31,561 1 26,610 2,358 9.6 13,672 12 2 37,985 2,172 5.7 22,910 a:Total number of students (1992) in grade 4: 66,000; total in grade 8: 60,000; total in grade 12: 53,500.
Not in "the best interests of all children"
The implications are both clear and discouraging: In general, only about 4 to 10 percent of all students will improve, at a cost per improvement ranging from $13,672 to $33,597. Moreover, most of the improvement that does take place consists of movement up from the lower levels (level 1 and below). There is relatively little movement up to the higher levels (levels 3 and 4).
Including Socioeconomic Factors
Statistically, the use of only one explanatory variable - in this analysis, spending per student - exaggerates the importance of this factor in explaining performance. Our second approach overcomes this bias by expanding the analysis to incorporate three additional factors within each community - the crime rate, the proportion of the work force consisting of professionals or managers, and the median income.
The results are again clear. The socioeconomic factors significantly affect academic achievement in the expected direction: Higher income, lower crime rates and a higher proportion of professionals improve performance within a community. All three factors are statistically significant. (See appendix.) More spending, however, does not improve performance. Spending is statistically insignificant for all three grades.
These results imply that the socioeconomic conditions under which school children live and receive their education are so much more important than the amount spent on their schools as to reduce to insignificance the effect of additional spending on academic achievement. More spending of the kind mandated by the education reform bill is likely to be ineffective for improving this achievement.
Policy Implications
Does this mean that parents and government are powerless to effect change? No, government might, for example, address one socioeconomic condition against which it can be directly effective - crime. It might spend more on school crime-prevention programs, police patrols, or neighborhood watch groups.
Another, even more effective option is to empower parents through vouchers, school choice or tuition tax credits. This allows motivated students to attend schools of their choice that are likely to be more conducive to learning. It also applies the discipline of the marketplace to the performance of public schools - a discipline that is likely to be more effective than cumbersome administrative reforms, constrained, by political necessity, to protect the jobs of public school teachers.
House 1000 stands this empowerment philosophy on its head by suspending existing school choice programs, pending a review of such programs for their "educational and budgetary impact." The Department of Education is directed, in conducting this review, to determine whether school choice has produced "tangible educational improvements, including test scores." This is ironic in light of the absence, in both bills, of any language demanding "tangible educational improvements" as a condition for increased state spending.
Senate 1550, in contrast, institutes "mandatory choice": Parents may send their children at state expense to any Massachusetts public school with room to accommodate them. While this gives students an opportunity to learn, it also poses the danger that receiving schools will be forced to alter their standards in order to accommodate students from sending schools. In order for school choice to succeed, it is necessary to preserve the standards that make receiving schools an attractive choice.
In summary, we can expect more spending to improve academic achievement only when it is conditioned upon social reform. Barring such reform, the only hope for motivated students lies in an opportunity to attend schools that, because of their location or excellence, provide better conditions for learning. When it comes to education, empowerment is necessary for achievement.
APPENDIX
We estimated the following "reciprocal" regression models to analyze the relationship between spending per student and performance on the MEAP test:
Expected Percentage of Students at Lower Levels = §0 + ß1 (1/X1), (A.1)
Expected Percentage of Students at Higher Levels = ß0 + ß1(1/X1), (A.2)
where X1 = spending per student.
Model A.1 measures the movement of students who are expected to leave the lower levels - level 1 and below - owing to a change in spending per student. Model A.2 measures the movement of students who are expected to enter the higher levels - levels 3 and 4 - owing to a change in spending per student. Both models are "simple" regressions: there is only one explanatory variable - spending per student. Table 2 shows the results:
Table 2
Simple Regression Models
Variable 4th Grade 8th Grade 12th Grade Phase 1 Phase 2 Phase 1 Phase 2 Phase 1 Phase 2 Constant 18.921* 38.678* 25.079* 42.869* 25.053* 46.085* 1/Spending 66689* -58807* 64130* -57898* 76891* -69479* * denotes significance at the one-percent level.
As noted in the main text, an analysis of school performance that relies solely on one factor, spending per student, overestimates the importance of this factor for determining academic performance. In order to correct for this bias, we also estimated multiple regression models that incorporate socioeconomic factors, in addition to spending per student:
Expected Percentage of Students at Lower Levels = ß0 + ß1(1/X1) + ß2X2 + §-3X3 + ß4X4 , (A.3)
Expected Percentage of Students at Higher Levels = ß0 + ß1(1/X1) + ß2X2 + ß3X3 + ß4X4 , (A.4)
where X1 = spending per student, X2 = crime rate, X3 = proportion of work force consisting of professionals or managers, and X4 = median income. The results for the multiple regression models are shown in Table 3:
Table 3
Multiple Regression Models
Variable 4th Grade 8th Grade 12th Grade Phase 1 Phase 2 Phase 1 Phase 2 Phase 1 Phase 2 Constant 52.045* 13.9* 73.188* 4.428 61.848* 13.521* 1/Spending 5051 -14527 -23663 13100 -2458 3463 Crime Rate 1.3938* -0.7202* 0.7822* -0.5176** 1.13* -0.7271* Prof/Man -19.25*** 22.835** -46.79* 42.91* -30.9** 31.31* Income -0.0003* 0.0002** -0.0003* 0.0002* -0.0003* 0.0002*
* denotes significance at the one-percent level, ** denotes significance at the five-percent level, and *** denotes significance at the ten-percent level.
All data for this study were obtained from the Massachusetts Department of Education. Some of the above material appeared in BHI FaxSheet, "House 1000: Not Much Improvement for the Money, " March 1993.
Leah M. Fay is research economist, Alison Kelly is resident scholar and David G. Tuerck is executive director at the Beacon Hill Institute. Alison Kelly is also assistant professor of economics and David G. Tuerck is professor and chairman of economics at Suffolk University.
