Tuesday, February 18, 2014

The Hypocrisy of Accountability

“Free Mr. Clark! Free Mr. Clark!” Blackboard Wars was originally cast as a show that would display a Lean on Me-like turnaround for a dangerous, failing academic institution in New Orleans, Louisiana. That it wasn’t, however! A cultural icon never emerged. John McDonogh Senior High School, the charter high school operated by the Future Is Now, which gained fame as the focus of Oprah Winfrey’s Blackboard Wars, is closing. The news story (found here) reported several reasons for the school’s closure. Most of all, the story emphasizes the academic woes of the school. Apparently, the school turned in a dismal school performance score, a score that placed the school’s performance as the worst in the state of Louisiana – even lower than the scores of alternative schools.
            Charter schools arrived in New Orleans in a different package than they arrived in some other cities. Charter schools were hardly about community engagement in the New Orleans area although some charter schools have made efforts at engaging the community. In fact, it appears that charters are only granted for schools in the New Orleans area when those charters are supported by large-scale charter management organizations (seemingly the opposite of local community control). Charter schools in New Orleans, the now epicenter of the charter movement, are about school accountability. Traditional public schools were routinely closed to make room for charter schools that would ultimately turn in better academic performance than the traditional public schools.
            There is something hypocritical about the accountability movement, which is disguised in the charter school movement. While traditional public schools in New Orleans are shuttered for a lack of performance, charter schools are hardly shuttered for similar reasons. In a recent conversation with a Louisiana Department of Education worker, the worker attempted to assert that many New Orleans area charter schools had been closed for failure to meet the academic promises of the schools’ charters. This assertion is little more than a fib. Of all the charter schools that have been shuttered in the New Orleans area, I can’t remember one charter school that was closed solely because of academic performance. Most, if not all, closed charter schools have been closed due to financial woes. The Future Is Now-operated charter school at John McDonogh is yet another example of an academically failing charter school that is closing because of financial woes more than academic failures. It is notable that the school ran financial deficits of more than a million dollars for two years straight, which is probably the state’s reason for closure more than the school’s academic problems.

            The truth of the matter is something had to happen in New Orleans Public Schools. As a graduate of the fledgling school district, I was aware that people were laughing at the school district and not with the school district. Although I don’t agree with the extreme accountability – based nearly completely on test scores – I understand the desperation of parents in New Orleans. Schools had to become accountable for failing generations of students. My problem is the double standard. Why are some schools closed for failure to meet standards of adequate academic progress, but other schools are only closed when they fail to meet adequate academic progress and a host of other problems plague the school? This is nothing short of hypocrisy, and parents in New Orleans deserve better! They deserve true and evenly distributed accountability that holds all schools responsible for failing to satisfactorily educate students.

A Response to U.S. Secretary of Education Arne Duncan’s The Threat of Educational Stagnation and Complacency

by Joseph C. Krajacic

On December 3, 2013, United States Secretary of Education Arne Duncan made strong statements to the Organisation for Economic and Co-operative Development (OECD) upon release of the 2012 Programme for International Student Assessment (PISA) results. “The big picture of U.S. performance on the 2012 PISA is straightforward and stark: It is a picture of educational stagnation.”  The Secretary goes on to say, “That brutal truth, that urgent reality, must serve as a wake-up call against educational complacency and low expectations.” But is this truly the case?  One would like to believe that prior to pontificating, the United States Secretary of Education did some research himself. 

In his speech to the OECD, Secretary Duncan professed progress “On the 2013 National Assessment of Educational Progress (NAEP), or what's known as the nation's report card, reading and math scores edged up nationally to new highs for fourth and eighth graders.”  What the Secretary failed to mention is the long term NAEP showed significant gains for all students for fourth and eighth grades in reading and mathematics from 1973 to 2012, but little statistically significant change for all students regardless of race, ethnicity or gender from 2008 to 2012.  Let the reader take note that Arne Duncan was sworn in as U.S. Secretary of Education in January of 2009. 

In his article, Poor ranking on international test misleading about U.S. student performance, Stanford University reporter Jonathan Rabinovitz details from the revised report, What do international tests really how us about U.S. student performance? by Martin Carnoy and Richard Rothstein that “score trends on these different tests can be very inconsistent, suggesting need for greater caution in interpreting any single test.  For example, declining trends in U.S. average PISA math scores do not track with trends in TIMSS (Trends in International Mathematics and Science Study) and NAEP, which show substantial gains for all U.S. social classes.”  If  “PISA is an important comparative snapshot of U.S. performance,” then the TIMSS and NAEP are two other important “snapshots” with different views. The inconsistent findings amongst these tests make it difficult to understand Secretary Duncan’s firm assertion of “educational complacency and low expectations.”

Secretary Duncan proclaimed, “The PISA is an important, comparative snapshot of U.S. performance because the assessment is taken by 15-year olds in high schools around the globe.” He goes further to discount naysayers who blame poor and minority students for the U.S. performance by claiming white students also lag behind their global peers.  Is the “snapshot of U.S. performance” the Secretary refers to really representative of the United States?  In Asia Society’s Asia Blog by Jessica Kehayes, Has Recent Coverage of Global School Rankings Missed the Point? Tony Jackson states, If we take the top 10 percent of the most privileged students in the United States, their mean score would be near the top of the international rankings. As we go down the socioeconomic ladder, skills drop. This means that American students do not have equal access to a good education.” 

The Stanford Graduate School of Education and the Economic Policy Institute (EPI) reports, “Socioeconomic inequality among U.S. students skew international comparisons of test scores…when differences in countries’ social class compositions are adequately taken into account, the performance of U.S. students in relation to students in other countries improves markedly.”   In a response to Andreas Schleicher’s comments regarding the 2013 EPI report, What do international tests really show about American student performance?, Carnoy and Rothstein theorized that the disproportionately higher concentration of disadvantaged students who participated in PISA may be due to their schools’ reliance on federal funding.  More affluent schools tend to rely less on federal funding and more on local tax dollars, thus declining PISA’s voluntary participatory acceptance.  In addition, schools in more affluent areas would most likely already have significant benchmark testing.

As stated on ED.gov biography of Secretary Arne Duncan, “Under Duncan's leadership at the Department, the Race to the Top program has the incentives, guidance, and flexibility it needs to support reforms in states. The Department also has focused billions of dollars to transform struggling schools, prompting nearly 1,000 low-performing schools nationwide to recruit new staff, adopt new teaching methods, and add learning time.”  If this is truly the case, then why does Secretary Duncan believe “The big picture of U.S. performance on the 2012 PISA is straightforward and stark: It is a picture of educational stagnation?”

Joseph C. Krajacic is a doctoral student in the Educational Policy, Organization and Leadership Program at the University of Illinois at Urbana-Champaign.  In his twenty-fourth year in secondary education, Joseph is presently an Associate Principal at John Hersey High School in Arlington Heights, Illinois.              

Teacher Working Conditions in Charter Schools

Research suggests that charter schools have much higher teacher turnover rates than traditional public schools (TPS). Teachers in charter schools are more likely to transfer to another school or leave the profession altogether when compared to their TPS colleagues. Since teachers’ perceptions of in-school working conditions have an important influence on their willingness to remain at a school, it is important to understand whether and how charter schools influence teacher working conditions.

In theory, it is possible to argue either more or less supportive workplace conditions for teachers in charter schools compared to TPSs. On the one hand, charter schools are autonomous and enjoy substantial flexibility in hiring teachers who are committed to a school’s instructional mission and help establish collaborative school environments. Also, charter school teachers tend to have more discretion to innovate educational programs and collectively participate in decisions regarding school design and organization. On the other hand, many charter schools are not bound by collective bargaining agreements. Without the union’s protection in areas such as workload, salaries and benefits, and due process rules, charter school teachers may experience more stress and doubts than teachers in unionized TPSs.

Empirical evidence comparing teacher working conditions in charters and TPSs is limited. Based on surveys in Colorado, a study finds that charter and TPS teachers perceive similar levels of collegiality and sense of shared missions. While charter school teachers exert more influence over classroom-related issues, they have the same or less influence on school governance and policies than their peers in TPSs. Another study compares the weekly hours worked by charter and TPS teachers and found little difference nationwide.

Using data from the
Schools and Staffing Survey (SASS), my study compares working conditions in charter schools and TPSs. I find that charter school teachers report substantially more influence in school-wide policies but heavier workloads than TPS teachers. Other than that, teachers in both charter schools and TPSs perceive similar levels of principal leadership, sense of community and collegiality, classroom autonomy, opportunities for professional development, and adequacy of instructional supplies. The findings support previous research that, although charter school teachers have greater influence in school policies, the decision-making processes seem to significantly adds to teachers’ workload. Also, the autonomy in charter schools does not necessarily foster changes regarding classroom instruction, professional development, and teacher collaboration that are more closely related to student learning.

Among all charter schools, I find district-granted charter schools tend to provide more supportive working environments than charter schools granted by other organizations, including state boards of education and postsecondary institutions. Compared to TPSs, teachers in district-granted charter schools perceive more power in school-wide decision making but similar workloads. It implies that autonomy and workload are not necessarily in conflict: it is possible to empower teachers in decision making and avoid overworking them at the same time. Since the designation of which organizations are authorized to grant charters is defined by individual state’s laws, the results also imply that state policy can have some indirect influence over charter school working conditions.

Given the existing literature that supportive working conditions increase teacher satisfaction and reduce teacher turnover rates, an important next step is to examine what aspects of working conditions matter most to charter school teachers as they make decisions about their careers. In addition, due to the finding that district-granted charter schools tend to provide more supportive teaching working conditions than other types of charter schools, further research is needed to explore how different authorizers affect working conditions for teachers.

Further readings:

Gawlik, M. A. (2007). Beyond the charter schoolhouse door: Teacher-perceived autonomy. Education and Urban Society, 39(4), 524–553.
Johnson, S. M., & Landman, J. (2000). “Sometimes bureaucracy has its charms”: The working conditions of teachers in deregulated schools. Teachers College Record, 102(1), 85–124.
Malloy, C. L., & Wohlstetter, P. (2003). Working conditions in charter schools: What's the appeal for teachers? Education and Urban Society, 35(2), 219–241.

Friday, February 14, 2014

Measuring Competition

The most interesting aspects of school choice, to me, are the outcomes associated with traditional public schools near newly opened “schools of choice.” School choice was a fairly alien topic to me until graduate school. I grew up in a town with one high school and the only school of choice was a Catholic grade school. I am likely primarily interested in the effects of how the competition generated by schools of choice change (or do not change) traditional public schools due to the fact that my experiences as a student in primary, secondary, undergraduate, and graduate school and my experiences as a teacher have all been in public schools. Moreover, despite the focus of the media, policy-makers, and researchers, schools of choice are, and will remain, a small segment of the educational landscape.

Through reading the literature on the effects of competition, I was surprised to learn the wide range of findings associated with the effects of competition. In summary, studies have found that school competition may improve[i], impair[ii], or have no effect[iii]  on traditional public schools. Through examining these studies, it became clear that while all of the studies claim to examine the effects of competition, a wide range of measures and methods are being used to define competition.
Thus, the problem. Educational researchers are seeking to answer questions and inform policy makers about issues surrounding school choice, but without agreed definitions and consistent measures, researchers will likely continue to find conflicting evidence. This problem is not exclusive to competitive effects, or even school choice, but it continues to be a problem nonetheless. Obviously educational programs and policies do not exist in a vacuum and solutions of chemistry or physics do not translate to studies of school choice, but how can educational researchers build an evidence base with a foundation that includes common definitions and measures?
I do not seek to answer this question here, but to start a conversation that may one-day result in answers. What are your thoughts? What is the next step? Is this even a problem?

Bettinger, E. P. (2005). The effect of charter schools on charter students and public schools. Economics of Education Review, 24(2), 133-147. Doi:10.1016/j.econedurev.2004.04.009
Bifulco, R., & Ladd, H. (2006). The impacts of charter schools on student achievement: Evidence from North Carolina.  Education Finance and Policy, 1(1), 50-90. Doi:10.1162/edfp.2006.1.1.50
Bohte, J. (2004). Examining the Impact of Charter Schools on Performance in Traditional Public Schools. The Policy Studies Journal, 32(4), 501-520.
Booker, K., Gilpatric, S. M., Gronberg, T., & Jansen, D. (2008). The effect of charter schools on traditional public school students in Texas: Are children who stay behind left behind? Journal of Urban Economics, 64(1), 123-145. Doi:10.1016/j.jue.2007.10.003
Buddin, R. J., & Zimmer, R. W. (2005). Is charter school competition in California improving the performance of traditional public schools?. RAND.
Carr, M., & Ritter, G. (2007). Measuring the competitive effect of charter schools on student achievement in Ohio’s traditional public schools. Research Publication Series, National Center for the Study of Privatization in Education, Teachers College, Columbia University. Retrieved November 24, 2007.
Holmes, G. M., DeSimone, J., & Rupp, N. G. (2003). Does school choice increase school quality? (No. w9683). National Bureau of Economic Research.
Hoxby, C. M. (2003). School choice and school productivity: Could school choice be a tide that lifts all boats? In C. Hoxby (Ed.), The Economics of School Choice (pp. 287-342). Chicago, IL: University of Chicago Press.
Imberman, S. A. (2007). The effect of charter schools on non-charter students: An instrumental variables approach. University of Houston.
Imberman, S.A. (2011). The effect of charter schools on achievement and behavior of public school students. Journal of Public Economics 95,(2011), 850-863. Doi:10.1016/j.jpubeco.2011.02.003
Ni, Y. (2009). The impact of charter schools on the efficiency of traditional public schools: Evidence from Michigan. Economics of Education Review, 28(2012), 571-584. Doi:10.1016/j.econedurev.2009.01.003
Sass, T. (2006). Charter schools and student achievement in Florida. Education Finance and Policy, 1(1), 91. Doi:10.1162/edfp.2006.1.1.91
Winters, M. (2012). Measuring the effect of charter schools on public school student achievement in an urban environment: Evidence from New York City. Economics of Education Review, 31(2012), 293-301. Doi:10.1016/j.econedurev.2011.08.0

[i]  (Bohte, 2004; Booker et al., 2008; Holmes, DeSimone, & Rupp, 2003; Hoxby, 2003; Sass, 2006; Winters, 2012)
[ii] (Bettinger, 2005; Carr & Ritter, 2007; Imberman, 2007; Ni, 2009)
[iii] (Bifulco & Ladd, 2006; Buddin & Zimmer, 2005)

Wednesday, February 05, 2014

How Policymakers Define “Evidence”: The Politics of Research Use in New Orleans

Note: This blog is adapted from an under-review article generated from research funded by the William T. Grant Foundation’s program on Understanding the Acquisition, Interpretation, and Use of Research Evidence in Policy and Practice. This blog was written by Priya La Londe. A further analysis of the below topic will be available in a forthcoming publication by Huriya Jabbar, Priya Goel, Elizabeth DeBray, Janelle Scott, and Chris Lubienski. Please contact Huriya Jabbar with any questions.

Public school systems across the U.S. have seen unprecedented expansion of incentive-based reforms, such as teacher performance pay, school choice, new governance forms, and alternative pathways to teaching and leadership. New Orleans, with its virtually overnight transformation of schooling post-Katrina, is perhaps the reform hotbed, a key incubator of the market principles that underpin reforms. Since Katrina, education reforms in New Orleans have emphasized incentives, choice, competition, and privatization, as well as new governance forms and alternative pathways to teaching and leadership (Buras, 2011). Advocates of incentive-based reforms point to improved aggregate test scores in New Orleans as markers of success and are promoting the model at the national level. As New Orleans-style reforms spread in the absence of strong supporting research, we ask what types of evidence are policymakers using to justify and expand these reforms, and how do they access it?

To answer this question, we build on previous conceptions of research utilization to examine how intermediary organizations broker research on what we call “incentivist” policies (Lubienski, Weitzel, & Lubienski, 2009); policies such as school choice and teacher incentive pay. In this new policy terrain, intermediary organizations link the supply and demand for research evidence, and they work in support of, and in opposition to, these highly contested education policies. We interviewed key district and state policymakers, as well as representatives from intermediary organizations in the area, who, we argue, are also shapers of policy.

We find that, in light of scant empirical evidence on the New Orleans reforms, policymakers primarily used personal anecdotes to justify their positions and explain the success of reforms. They also relied heavily on Education Week and other online media sources to translate research, especially blogs that aligned with their perspectives,. Peer-reviewed research was seldom used. When it was used, it was typically passed to policymakers via an echo chamber of intermediary organizations or personal contacts. Furthermore, we find that rather than being inundated with data and reports, as we commonly perceive policymakers at the national level to be, state-level policymakers in Louisiana did not receive much research or evidence directly. They did not believe they were seeing, much less being inundated with, reputable research; when they did receive information, it came from representatives at intermediary organizations that they worked with, or from individual researchers with whom they had a personal connection.

Such findings, at the very least, complicate traditional notions of research utilization among policymakers. We hope the findings shed light on what Louisiana policymakers should, going forward, institute as they weigh the effects of the New Orleans school reforms: an independent, credible research entity.


Buras, K. L. (2011). Race, Charter Schools, and Conscious Capitalism: On the Spatial Politics of Whiteness as Property (and the Unconscionable Assault on Black New Orleans). Harvard Educational Review, 81(2), 296–331.

Lubienski, C., Weitzel, P., & Lubienski, S. (2009). Is There a “Consensus” on School Choice and Achievement? Advocacy Research and the Emerging Political Economy of Knowledge Production. Educational Policy 23(1), pp. 161-193.