March 30, 2007 | Volume 4, Issue 1
Education Policy Research and Professional Training at SUPA and the Heinz School
A review of educational policy research and training within the Heinz School today, including what the Heinz School has done to help K-12 education in Western PA, and what they hope to do in the future.
Editor’s Note: This is the first paper in an updated section of the Review, In Practicum, that details individual Heinz School faculty research, educational innovations, and public policy activities. Professor Strauss joined the School and Urban and Public Affairs in 1979 from the University of North Carolina at Chapel Hill where he was a member of the Department of Economics for ten years. More about Professor Strauss can be found on his personal web page: www.andrew.cmu.edu/user/rs9f
1.0 Introduction
What attracted me to the School of Urban and Public Affairs (SUPA)1 in 1979 was Bill Cooper and Toby Davis’ vision to create a new kind of training for public administrators. Their world-wide reputation as researchers themselves, and a vision of a new curricula based on the research results and tools developed by its faculty and doctoral students made SUPA a very interesting opportunity. Also, the idea of SUPA was embraced and generously endowed by Richard King Mellon. In conveying the first endowment check, Joe Hughes of the RK Mellon Charitable Trusts, wrote at the close of 1968:
“It is the hope of the Trustees that this School . . . will become a nationally prominent contributor of men and ideas to the field of urban affairs with a particular interest and emphasis on Pittsburgh and Western Pennsylvania problems. We see a great opportunity for this School to provide assistance to the city, the state and the nation in the massive effort which is needed to educate, train and motivate managers in the field of urban affairs. ...The Trustees hope the income from this grant will help you to undertake the research and educational programs designed to develop new tools and new approaches to attack the whole problem of the cities in all their complexities as discussed by Dr. Cooper”
Central to this vision was not only faculty engaged in serious research, theoretical, applied, and methodological, but also a small, apprentice-ship PhD program whose doctoral students’ published research would ensure their subsequent employment in academe or the broader society.
Several early SUPA doctoral dissertations made substantial theoretical and empirical contributions, and had a broad impact on many fields of enquiry and practice. First, the development of non-parametric methods of measuring economic efficiency was accomplished by Eduardo Rhodes in his thesis under SUPA Dean Bill Cooper and CMU Professor of Mathematics Abe Charnes.2 Data Envelopment Analysis or DEA, as it became known, applied linear programming to the analysis of economic efficiency and is now a standard method recognized by operations researchers and economists as a way to analyze the productivity of private, non-profit, and governmental units. Second, Robert Carbone, writing under Professor Richard Longini, devised a new method for tracking real estate markets in order to accurately predict the assessed values of real estate.3 Third, Tony Boardman wrote an influential dissertation in 1975 under SUPA Dean Toby Davis that re-analyzed the data in the 1966 Coleman Report, and reached much more optimistic conclusions about the beneficial impact that incremental resources could have on student achievement.4 There have been other notable contributions to method and policy by faculty and doctoral research here, but below I shall discuss the long term attention paid to matters surrounding education at SUPA and Heinz, and describe some of my most recent research in this area for the Review.5
Dean Toby Davis’ interest in discovering through scholarly research with his students what might make public education more productive was mirrored by an interest in using the Pittsburgh region as a laboratory. Dr. Harry Faulk was recruited from the ranks of the most successful school superintendents in the region to develop a research activity with the region’s school districts who subscribed to the program. SUPA served as a hub and place of interaction between the education community and the research and outreach efforts of the School.
This outreach activity evolved over time from collaborative applied research to professional training of educators. Dr. Faulk developed two innovative professional masters programs to that end. He first developed the Educational Leadership program to train experienced educators interested in becoming principals. It was established for mid-career educators who sought to become school principals; however, unlike principal preparation programs in neighboring and other schools of education, the Ed Leadership curricula was based on the philosophy in SUPA‘s core masters program that emphasized general managerial training in accounting, statistics, in organization and strategy, micro-economics, and information systems. These general management courses were then augmented through courses in curricula supervision, school law, and school budgeting. This masters degree program was formally approved by the Pennsylvania Department of Education and has been re-approved several times. The first masters degree was awarded in 1992.6
The second and more recent training program involved the preparation of education technology managers who are in high demand not only in public school districts, but more generally by institutions engaged in professional development and who wish to utilize technology in the training process. Again, the idea was to develop a curricula that emphasized core management and analysis skills, and then augment them with courses that focus on technology, evaluation and research, and technology and education restructuring. This program was also formally approved by the Pennsylvania Department of Education. The first masters degree in this area was awarded in 2002.
Systems synthesis projects at the Heinz School have also been a vehicle for research on public education. One project, supervised by myself and SUPA PhD student Ashok Srinivasan in the mid-1990’s, devised and analyzed different mechanisms to schedule and move students among small school districts in order to enrich their educational opportunities in mathematics and science.7 Another year long-project, with Dr. Harry Faulk, performed an education technology audit for the Brentwood School district in 2000–1, and provided a template for the analysis of other districts technology programs.
My own interest in the relationship between education inputs and student achievement began while on the faculty at Chapel Hill in the late 1970’s. North Carolina was one of the early states to test its teachers using the Educational Testing Service’s National Teacher Exam. Newspaper accounts about wide variability in teachers own proficiency on their standardized tests led me to pursue the district level data from the North Carolina Department of Education. Getting the right data on teacher test scores proved crucial in explaining variations in student average achievement scores and the percentage of students testing below grade level.8 Elizabeth Sawyer, a PhD. economics student at UNC, and I found that a 1% improvement in average teacher test scores was associated with a 3 to 5% reduction in the fraction of students testing below grade level, and much more modest effects, elasticities of around .8 on average student achievement. These Cobb-Douglas educational production function results held constant the socio-economic and ethnic background of the students, student teacher ratios, and the insured value of the capital stock.
More recent work with Pennsylvania data by Professor Bill Vogt and myself confirmed this general finding, and found even stronger positive effects of general knowledge of teachers on student achievement.9 Using Pennsylvania data that differentiated between general knowledge, and knowledge about pedagogy, and more sophisticated econometrics and a structural model of the hiring decision, we found that a 1% improvement in general knowledge is associated with as much as a 10% improvement in an index of student achievement.
I have also been involved in curricula innovation, and designed and taught a course on statistics, and educational assessment, for students in the Masters in Education Technology program. Subsequently, Norma Chang, a doctoral student in CMU‘s Psychology Department and now on the faculty at UC-Berkeley, and I co-taught a version of the course, to principals drawn from the Pittsburgh Public Schools. Several of those principals are now in major leadership positions in the Pittsburgh Public Schools.
2.0 Recent Research on Teacher Preparation and Selection and Related Issues
Having confirmed through earlier research that more knowledgeable teachers improve their own students’ performance, I undertook in 1991 a program of research on the preparation and selection of classroom teachers throughout Pennsylvania, and the impact of teacher characteristics on student classroom performance. That line of enquiry was initially supported by the Pew Charitable Trust of Philadelphia,10 and led to a series of regional and state-wide projects. Most important was a multi-year project for the Pennsylvania State Board of Education that resulted in Pennsylvania’s teacher preparation standards and testing requirements being moved from among the lowest in the US, to the most demanding in the US in 2000.11 Not only were schools of education required to raise their admissions and graduation standards, but teachers prepared in the new regime would have to take the same specialization courses as majors do in arts and sciences. That is, since 2000, prospective math teachers are required to take the same required mathematics courses as undergraduate math majors. That project was supported by the Heinz Endowments, the Grable and Buhl foundations, and the State Board of Education. Besides actually changing state law and regulations, the research led to various publications12 and state and federal public testimony, and enabled other research, on such matters as the nature of the classroom teacher retirement decision,13 the market for substitute teachers in South West Pennsylvania, and the supply and demand for school administrators in Pennsylvania.
Among the major findings of my research for the Pennsylvania State Board of Education were:
1) Only half have written personnel policies, and that the districts are haphazard in the way they hire teachers;
2) There is huge variation in the standardized test scores of graduates of different schools of education. When these scores are transformed into the percent correct, the medians across schools of education, or across school districts in terms of their employed teachers, are on the order of 2:1;
3) There is a positive and statistically significant correlation between more professional personnel policies and student achievement among Pennsylvania’s school districts; and,
4) Pennsylvania school districts, on average, employ former students for 40% of their teachers; the greater this measure of insularity, the lower is general student achievement in a school district.
The observed variability in teacher quality among wealthy and poor school districts, and the lack of correlation between hiring selectivity and school district wealth led me to consider how matters of governance and institutional design might explain this, and to investigate what alternative governance mechanisms might look like that would still permit local control. I first explored this issue with Ruth Kolb, a practicing attorney, who came to Heinz for a masters in public policy,14 by looking at state ethics laws governing school board members in 1998–9. Roger Severino, another Heinz masters student, who subsequently graduated from Harvard Law School, and I broadened that enquiry by comparing rules supervising the conduct of directors of publicly traded corporations to those that apply to elected school board directors.
Figuring out that schools do not ensure that their students learn because their board members are not required, through their oath of office, to make sure that this happens, and are also not precluded from indirect self-dealing, does not, however, lead to immediate change. The current morass in school governance exists for reason. That is, it should come as no surprise that potentially corrupt governance mechanisms can result in dysfunctional or corrupt activity, and that changing that equilibrium requires an improvement in general governmental ethics.15
In December, 2005, the Pennsylvania State Board of Education requested that I undertake an analysis of the effects of the teacher preparation reforms they made in 2000. This three year project, currently underway, is being supported by the William Penn Foundation of Philadelphia and the Howard Heinz Endowments of Pittsburgh. Better data on teachers and student achievement are being collected and analyzed, and collaboration with the Educational Testing System of Princeton, New Jersey is also being developed. A number of Heinz and Tepper students are providing research assistance, and will be presenting papers at a research conference next month. The research is currently engaged in a survey of hiring practices, analysis of the relationship between school and teacher characteristics and student achievement, and a review of the curricula of Pennsylvania’s 90+ schools of education to determine if they have complied in fact with the requirement that prospective teachers take the same courses as do true majors at each university, and to compare across universities the depth and breadth of their preparation..
Concurrent with this state-wide project has been an ongoing project with the Pittsburgh Public Schools to look at how classroom resources and peer students affects student performance across all grade levels. Ms. Haijing Hao, a Heinz doctoral student, is involved in this research.
3.0 Summary
Public policy research on any subject involves understanding current law and how people, resources, and institutions intertwine to produce public services through government or its agents. Successful, independent public policy research by faculty and students in a university setting can not only lead to new knowledge, but also the transmission of such knowledge to professional management students who will take both tools and findings to their new place of work. Over time, one may reasonably hope that the broader society will learn and adopt changes in policy that reflect such knowledge.
What I hope the reader takes from this historical and personal essay about education policy research at the Heinz School is that this model of research, findings, training, and changes in public policy so informed envisioned by Bill Cooper and Toby Davis, and supported by the RK Mellon Foundation is alive and well.
1 SUPA became the H. John Heinz III School of Public Policy and Management in 1992.
2 A. Charnes, WW. Cooper, and E. Rhodes( 1978), “Measuring the efficiency of decision-making units,” European Journal of Operational Research, 2:429–44.
3 Robert Carbone, Richard L. Longini(1977), “A Feedback Model for Automated Real Estate Assessment,” Management Science, Vol. 24, No. 3 (Nov., 1977), pp. 241–248.
4 A.E. Boardman, O.A. Davis and P.R. Sanday, “A Simultaneous Equations Model of the Educational Process,” Journal of Public Economics 7, 1977, 23–49.
5 The education policy research discussed below deals entirely with issues of efficacy and various education labor markets, and does not deal with my work addressing school finance, accounting, real estate assessment and taxation, and related political economy issues in Pennsylvania, New York and elsewhere. For examples of research on these second set of education policy issues, see: “Reforming School Finance in Pennsylvania,” Testimony before the Select Committee on Public School Funding, Pennsylvania General Assembly, Indiana, Pennsylvania October 10, 2001; “Distributional and Economic Effects of Pennsylvania’s Local Property Taxes,” State Tax Notes 18, (May 15, 2001), reprinted from Testimony before the Pennsylvania Senate Education Committee, Harrisburg, Pennsylvania. March 22, 2000. Also, see “Community Choice and Local Public Services: A Discrete Choice Approach,” with Thomas Nechyba, Regional Science and Urban Economics, 28, 1 (January, 1998), 51–74. (NBER Working Paper 5966). “States of Mind: Why Homeowners Hate the Property Tax,” State Tax Notes 13, (June 16, 1997), pp. 1802–1806; “Reducing New York’s Reliance on the School Property Tax,” Journal of Education Finance, 21, 1 (Summer, 1995), 123–164. (Reprinted in State Tax Notes, 9, 5 (July 31, 1995), 339–360.) “Reforming School Finance in Illinois: Some Observations on Principles, Practicalities, and Politics,” State Tax Notes, 5, 7 (August 16, 1993), 351–60. “Factors Influencing School District Financial Reporting Practices,” with Rajiv D. Banker and Beverly Bunch, Research in Governmental and Non-Profit Accounting, 5 1989 , 27–56. “The Impact of Block Grants on Local Expenditures and Property Tax Rates,” Journal of Public Economics, 4, 1 (Fall, 1974), 269–284; “School Finance Reform: Moving from the School Property Tax to the Income Tax,” Proceedings, 88th Annual Conference on Taxation, Annual Conference on Taxation, National Tax Association-Tax Institute of America, San Diego, California (October, 1995), 84–91. “Reducing Reliance on the School Property Tax: Rationales and First Results,” in David H. Monk [editor]. Study on the Generation of Revenues for Education. (Albany, New York: New York Board of Regents, February 3, 1995), 107–151. “Discussion” of `Educational Vouchers and Social Policy’, by Henry M. Levin, in James Gallagher and Ronald Haskins, (editors), Care and Education of Young Children in American: Policy, Politics and Social Science, (Ablex Publishing Company, 1980), 56–62
6 President Clinton’s Education Secretary personally commended Dr. Faulk for this innovative approach to training school managers in 1994.
7 See “Improving Access to Science and Math Education,” with Ashok Srinivasan, Interfaces, 29, 4 (July-August, 1999), 82–95.
8 “Some New Evidence on Teacher and Student Competency,” with Elizabeth A. Sawyer, Economics of Education Review, 5, 1 1986 , 41–48.
9 See Vogt, William B. and Strauss, Robert P. “It’s What You Know, Not How You Learned to Teach It: Evidence from a Study of the Effects of Knowledge and Pedagogy on Student Achievement ” (George J. Stigler Center for the Study of the Economy and the State Working Papers Series, 168).
10 Who Should Teach in Pennsylvania’s Public Schools? A Research Report to the Pew Charitable Trusts, 1993.
11 See Pennsylvania Bulletin, October 3, 2000.
12 See Teacher Preparation and Selection in Pennsylvania: Ensuring High Performance Classroom Teachers for the 21st Century . A Research Report to the Pennsylvania State Board of Education. With the assistance of Lori R. Bowes, Mindy S. Marks, and Mark R. Plesko. (Harrisburg, Pennsylvania: Pennsylvania State Board of Education), June 4, 1998, pp. 245. (ERIC Clearinghouse Number TM029186.); “Improving Teacher Preparation and Selection: Lessons from the Pennsylvania Experience,” with Lori Bowes, Mindy Marks, Mark Plesko, Economics of Education Review, 19,4 (July, 2000), 387–415; “Who Gets Hired to Teach? The Case of Pennsylvania,” in Marci Kanstoroom and Chester E. Finn, Jr. [editors]. Better Teachers, Better Schools. Fordham Foundation Press, 1999, 103–130; “A Reverse Engineering Approach to Improving Teacher Quality: The Hiring Decision and State Laws Governing School Board Conduct and Ethics,” Testimony before the Committee on Education and the Workforce Subcommittee on Post-Secondary Education, Training, and Life-Long Learning U.S. House of Representatives May 13, 1999.
13 See “The Effects of Defined Benefit Pension Incentives and Working Conditions on Teacher Retirement Decisions,” with Joshua Furgeson and William B. Vogt, Education Finance and Policy, 3, 1 (Summer, 2006), 316–348.
14 See Kolb, Ruth and Robert P. Strauss (1999), “A Survey of State Laws Governing School Board Ethics,” A Paper Presented to the 1999 Annual Research Conference of the American Education Finance Association, Heinz Working Paper 1999–8.
15 See Roger T. Severino and Robert P. Strauss, “Improving Public Education through Strengthened Local Control,” Conference on Education and Economic Development, Federal Reserve Bank of Cleveland, (April, 2005) pp. 73–105.