Trial and Error in Criminal Justice Reform: Learning from Failure

Trial and Error in Criminal Justice Reform: Learning from Failure

by Greg Berman, Aubrey Fox

Paperback(Revised Edition)

View All Available Formats & Editions
Choose Expedited Shipping at checkout for guaranteed delivery by Friday, June 21


In this revised edition of their concise, readable, yet wide-ranging book, Greg Berman and Aubrey Fox tackle a question students and scholars of law, criminology, and political science constantly face: what mistakes have led to the problems that pervade the criminal justice system in the United States? The reluctance of criminal justice policymakers to talk openly about failure, the authors argue, has stunted the public conversation about crime in this country and stifled new ideas. It has also contributed to our inability to address such problems as chronic offending in low-income neighborhoods, an overreliance on incarceration, the misuse of pretrial detention, and the high rates of recidivism among parolees. Berman and Fox offer students and policymakers an escape from this fate by writing about failure in the criminal justice system. Their goal is to encourage a more forthright dialogue about criminal justice, one that acknowledges that many new initiatives fail and that no one knows for certain how to reduce crime. For the authors, this is not a source of pessimism, but a call to action. This revised edition is updated with a new foreword by Cyrus R. Vance, Jr., and afterword by Greg Berman.

Product Details

ISBN-13: 9781442268470
Publisher: Rowman & Littlefield Publishers, Inc.
Publication date: 03/15/2016
Series: Urban Institute Press Series
Edition description: Revised Edition
Pages: 166
Product dimensions: 5.94(w) x 9.05(h) x 0.50(d)

About the Author

Greg Berman is the director of the Center for Court Innovation, a public-private partnership that seeks to reduce crime, aid victims, and improve public trust in justice.

Aubrey Fox is director of special projects for the Center for Court Innovation.

Read an Excerpt


Men are greedy to publish the successes of [their] efforts, but meanly shy as to publishing the failures of men. Men are ruined by this one sided practice of concealment of blunders and failures. (Burlingame 1997)
—Abraham Lincoln

Name a social problem and chances are that someone has written a book about how to solve it. Interested in improving failing schoolsfi Check out Wendy Kopp's One Day, All Children: The Unlikely Triumph of Teach for America and What I Learned along the Way (2003). Eradicating povertyfi Try Banker to the Poor: Micro-Lending and the Battle against World Poverty by Muhammad Yunus (1999). Creating new jobs while saving the planetfi There's The Green Collar Economy: How One Solution Can Fix Our Two Biggest Problems by Van Jones (2008). We could go on, but you get the picture. In general, the world of public policy is inundated with books about "best practices" and "evidence-based programs"—exemplary ideas and initiatives that promise guaranteed success.1 In truth, the world probably does not need another book that markets success. But failure . . . well, that's a different story. Failure is typically discussed only in hushed whispers in the world of civic affairs. In his effort to usher in "a new era of responsibility," President Obama has demonstrated a refreshing candor about his own mistakes. But this is still the exception and not the rule in public policy. Fear of failure is, of course, part of human nature. And experience tells us that the instinct for self-preservation can kick into overdrive when careers, reputations, and dollars are on the line. The higher the stakes, the less likely you are to hear the "F-word."

But there are real consequences when we fail to talk about failure. Most obviously, it leads to an environment that stifies innovation. Without a willingness to try new ideas and risk failure, it is impossible to imagine how we are ever going to challenge conventional wisdom or address our most difficult social problems.

Unfortunately, the field of criminal justice has been particularly slow in embracing the value of trial and error. In other disciplines, most notably science and medicine, solving problems is viewed as an iterative process.2 There is an old quote, attributed to Thomas Edison, that neatly encapsulates the importance of the scientific method: "I have not failed 5,000 times. I have successfully discovered 5,000 ways that do not work and I do not need to try them again." More recently, Eli Lilly & Company's chief science officer was renowned for holding "failure parties" as a way of acknowledging outstanding scientific work even if it ultimately failed to lead to new drugs for the company.3

Needless to say, there are no failure parties in criminal justice. Criminal justice officials are rarely afforded the opportunity to engage in a trial-and-error process, partly because the consequences of failure are so immediate (people can die) and partly because the media and political environment that surround the crime issue will not let them (people get fired). Is it any wonder that a "cover your ass" mentality dominates many criminal justice agenciesfi Policing expert and University of Wisconsin Law School Professor Michael Scott describes the problem this way: "Very seldom do police chiefs say, 'We had a great idea that didn't work. We're going back to the drawing board to do it differently.' That's what a scientist would say without batting an eye, but a police chief often doesn't feel that he or she has that kind of latitude." (Scott 2008).

To be fair, it is not accurate to say that there is no discussion whatsoever about failure in the criminal justice system. There are, in fact, built-in mechanisms for dealing with certain kinds of failures—most notably, civilian complaint review boards and the appellate review process. Partially because of these kinds of institutions, public discussion about the shortcomings of the criminal justice system tends to focus on such hot-button issues as police misconduct and the death penalty. As important as these topics may be morally and philosophically, they involve very few people and have only a marginal impact on public safety. They also offer few meaningful lessons for policymakers and would-be innovators.

Learning lessons from the past is not a particular strength of the criminal justice system. Indeed, the history of criminal justice in the United

States can be read as a swinging pendulum, as policymakers have veered from punitiveness to leniency and back again, without pausing to remember why they initially favored one approach over the other. The reluctance of criminal justice policymakers to talk openly about failure has helped fuel this dynamic. It has stunted the public conversation about crime in this country and stified new ideas It has also contributed to our inability to address such problems as chronic offending in low-income neighborhoods, an overreliance on incarceration, the misuse of pretrial detention, and the high rates of recidivism among parolees. Unless criminal justice policymakers and practitioners are given the time, space, and encouragement to learn from their predecessors and analyze their own foibles, the field will forever find itself haunted by Georges Satayana's dictum that "those who cannot remember the past are condemned to repeat it."

We have chosen to write a book on failure in the criminal justice system in an effort to escape this fate. Our goal is to encourage a more forthright dialogue about criminal justice, one that acknowledges that most new initiatives fail and that no one knows for certain how to reduce crime. For us, this is not a source of pessimism but a call to action. By openly discussing and even celebrating failure, we seek to help foster a climate that encourages the development and implementation of new ideas based on rigorous refiection.

To be clear, when we use the term "failure," we do not mean errors of incompetence or corruption—the judge who mistakenly rules a crucial piece of evidence inadmissible or the police officer who is on the payroll of local drug dealers. Nor are we focused on the kinds of societal failures—poverty, racism, the breakdown of the American family— that are so often intertwined with the criminal justice system.

Rather, this book is devoted to examining well-intended efforts that for one reason or another fell short of their stated objectives. These efforts include high-profile national programs, such as the Drug Abuse Resistance Education (D.A.R.E.) program, which has sent thousands of police officers into local schools to educate students about drugs without making a dent in teen substance abuse. We also look at lesser-known local initiatives, such as the St. Louis police department's creative, but ultimately flawed, program to reduce gun violence by asking the worried parents and guardians of teenagers for permission to inspect their homes for firearms in return for a promise not to make any arrests based on what they found.

By design, we examine several initiatives that have been widely hailed as successes, including drug courts (which link addicted offenders to drug treatment in lieu of incarceration) and Operation Ceasefire (a strategy to send a more coherent message of deterrence to individuals who are responsible for the lion's share of violence in a given jurisdiction). While drug courts and Operation Ceasefire have achieved impressive results in many places, in some locations they have struggled to succeed. Our implicit message in looking for failure amid success is that the line between the two is not as clear as some may think.

In writing this book, we drew upon three years worth of research into the failed criminal justice experiments of the past. This research, which was underwritten by the U.S. Department of Justice's Bureau of Justice Assistance, included a literature review, roundtables, on-site observations of innovative projects, and interviews with dozens of leading practitioners and scholars in the field.

This research has been buttressed by our own life experience. For the past decade, we have worked at the Center for Court Innovation, a not-for-profit organization that seeks to help the justice system reduce crime, aid victims, and promote public trust in justice. The Center for Court Innovation has won national prizes for innovation from the Drucker Institute, American Bar Association, National Criminal Justice Association, National Association for Court Management, Ford Foundation, and Harvard University's John F. Kennedy School of Government.

In particular, the Center has been honored for its success in conceiving, planning, and implementing model projects that seek to focus the energies of the justice system on addressing the problems of addiction, mental illness, domestic violence, and quality-of-life crime. These "problem-solving courts" include two projects that the two of us served as lead planners for: the Red Hook Community Justice Center and Bronx Community Solutions.

The Red Hook Community Justice Center is a community court located in a Brooklyn neighborhood that has long been notorious for drugs, crime, and disorder. In response, the Justice Center—a joint project of the Center for Court Innovation, the New York State Unified Court System, and the Kings County District Attorney's Office—has sought to reduce fear and improve public safety.

Operating out of a refurbished Catholic school that previously had been vacant for decades, the Justice Center is home to a multifaceted courtroom, presided over by the Hon. Alex Calabrese, that handles minor criminal matters, landlord-tenant disputes, and juvenile delinquency cases. Whenever possible, the Justice Center seeks to link defendants and litigants to social services, including drug treatment, job training, and mental health counseling, with the goal of helping them avoid a return to court. At the same time, the Justice Center serves as a launching pad for an array of crime prevention and community engagement initiatives, including but not limited to a youth court, youth baseball league, police-teen theater workshop, and an AmeriCorps community service program.

The Justice Center's novel approach to justice has contributed to the transformation of what was once a dreary neighborhood. Crime is down, investment in the community is up, and researchers have documented improved attitudes towards government.

In 2005, we brought some of the ideas piloted by the Red Hook Community Justice Center to the Bronx, but with a twist. Instead of working in a single neighborhood with just one judge, we have attempted to go to scale with problem-solving justice by providing the four dozen judges who handle misdemeanor criminal cases in the Bronx, a borough of 1.4 million people, with the same kind of sentencing options that Judge Calabrese in Red Hook utilizes.

Our goal in the Bronx is a simple one: to make justice in low-level cases more meaningful. While this experiment is still relatively young, there are encouraging results to report. The project already has changed sentencing practice in the Bronx—the use of short-term jail has been replaced by community restitution and social services. Bronx Community Solutions also has won considerable neighborhood support by contributing tens of thousands of community service hours to the Bronx each year, putting minor offenders to work sweeping the streets, painting over graffiti, and cleaning local parks.

Both Red Hook Community Justice Center and Bronx Community Solutions have been hailed as national models by the U.S. Department of Justice. Each year, these projects are visited by hundreds of criminal justice officials from across the country and around the world. Replications have been spawned in places both near (New Jersey and Connecticut) and far (Canada, Australia and England).

As gratifying as this attention has been, we know that every success that the Red Hook Community Justice Center and Bronx Community Solutions have experienced has been the result of a laborious trial-and-error process that inevitably involves disappointment along the way. We have had a role in creating technology applications that frustrated end users, mentoring programs that struggled to find mentors, antiviolence initiatives that proved impossible to sustain over time . . . the list goes on and on.

In reflecting upon our own failures as well as the failures of others, we have identified four principal themes that come up time and time again. These are the themes that animate this book.

1. Not all failures are alike. Failure is usually the product of a complicated chemistry involving a specific time, a specific place, and specific personalities. While every failure has its unique elements, failures generally fall into four distinct groups. The first two are relatively straightforward: failure of concept (a bad idea) and failure of implementation (poor execution). Sometimes, reformers just get it wrong, fundamentally misunderstanding the nature of the problem they are trying to address or failing to pay the necessary attention to service delivery. Two other kinds of failure are less obvious: failure of marketing and failure of self-reflection. These are essentially opposite sides of the same coin. On the one hand, innovators will not get very far if they do not manage politics well or if they are incapable of winning the necessary resources to implement their ideas. On the other hand, some reformers become so intent on drumming up support that they fail to assess their own weaknesses or to respond quickly as facts on the ground change. In chapter 1, we use the Consent to Search program in St. Louis to highlight the various ways in which a criminal justice experiment can fail.

2. Failure is rarely black-and-white. While the press and politicians tend to want bottom-line answers ("Does this program work or not?"), the reality usually is more nuanced: some initiatives work for some people some of the time. Moreover, the definitions of success and failure often depend on where one stands—and what one values. This is particularly true in the criminal justice system, where the principal actors often have conflicting agendas. Michael Schrunk, the elected prosecutor in Portland, Ore., offers this example: "If pretrial services succeeds in getting more people out of jail, they might define that as a success, whereas the local police force or prosecutor might not see that as being in their interest. . . . It is very difficult to get everybody at the table to agree on specific strategies because a lot of times they see it as, 'If you win, then I lose' " (Berman 2008). In our final chapter, we explore the difficulty of defining failure in greater detail, through the prism of the controversy over the D.A.R.E. program.

3. Politics plays an enormous role in the success or failure of any reform. Most innovations require government support or endorsement at some level, so criminal justice reformers constantly must grapple with the political realities of elected officials and high-ranking bureaucrats. It is tempting, particularly for "good government" types, to portray reform initiatives as somehow rising above the political concerns that typically drive government decisionmaking. This is a recipe for disaster. On the other hand, in order to mobilize political support and generate funding, reformers often find it necessary to overstate the potential impacts of their work. According to criminologist Joan Petersilia, when it comes to criminal justice reform, "there's a long history of over-promising and under-delivering":

There is nothing in our history of over 100 years of reform that says that we know how to reduce recidivism by more than 15 or 20 percent. And to achieve those rather modest outcomes, you have to get everything right: the right staff, delivering the right program, at the right time in the offender's life, and in a supportive community environment. We just have to be more honest about that, and my sense is that we have not been publicly forthcoming because we've assumed that we would not win public support with modest results (Berman and Fox 2008).

Petersilia is describing a common failure trap that reformers stumble into repeatedly. Unfortunately, failure is often as much a matter of perception as reality: even the best designed and implemented initiatives will be viewed as failures if reformers are not careful about managing expectations. The importance of politics— with both a big "P" and a small "p"—is woven throughout this book, most explicitly when we examine the checkered histories of drug courts in Minneapolis and Denver and tell the remarkable story of how officials in Connecticut were able to resist calls for "three strikes and you're out" legislation despite public outrage over one of the most shocking crimes in recent years.

4. Implementation is as important as ideas. Translating concepts into concrete programs is not easy. The truth is that the majority of new policy initiatives fail, just as the majority of new businesses fail. One of the most common obstacles is the challenge of context: what works in Los Angeles might not work in Chicago, let alone in a rural parish in Louisiana.

According to Lisbeth Schorr, professor at Harvard University, "Context is the most likely saboteur of innovations. The biggest mistake is thinking that because a program is wonderful, the surroundings won't destroy it when you plunk it down in a new place" (Berman and Fox 2002). Or, as the singer Billy Bragg puts it, "You can borrow ideas, but you can't borrow situations."4 Our chapter on the struggles of Operation Ceasefire to maintain and replicate its success in Boston in the 1990s illustrates this point. Reformers also must understand the importance of winning support from frontline workers (police officers, judges, prosecutors, parole officers). As difficult as it is to win battles among policy elites at the top levels of government, often the most important policy decisions are made at the ground level by "street-level bureaucrats" (Lipsky 1980). This lesson is underlined in our examination of the battle to reform parole in California.

In selecting our failure case studies, our goal was neither to assign blame nor to engage in critiques that are only possible with the benefit of 20-20 hindsight—indeed, we came away from our research with renewed respect for all those with the courage to attempt to change an institution as vast and complex as the criminal justice system. Nor is our goal to send the message that change is impossible. Quite the contrary. We seek to foster criminal justice innovation by acknowledging the reality that failure, while not desirable, is often inevitable and even acceptable, if it is properly analyzed and used as a learning experience. Even initiatives that fall short of their goals can provide valuable information and guidance as we look to improve the criminal justice system in the future.

While this book is about failure, it is worth noting that in many respects we are living in a period of remarkable success for the American criminal justice system. During this time, historic crime drops have made citizens safer, reduced victimization, and pushed public safety from the top to the bottom of the public agenda. Numerous theories have been advanced to explain the crime reductions of the past twenty years. Some have focused on metatrends, such as changing birth rates, an economy that until recently was expanding, the decline of crack cocaine, and fluctuations in abortion rates. Others have pointed to discrete criminal justice innovations, such as drug courts, sentencing reforms, the CompStat crime mapping and management system, and community and problem-oriented policing. No matter which explanation one favors, it is safe to say that the field of criminal justice has come a long way from the days of "nothing works" and "ungovernable cities."5

While there is much to be proud of, there is still more to be done. Despite significant advances in knowledge and practice over the last twenty years, American criminal justice agencies are still struggling to understand how to tackle problems like gangs, gun crime, and domestic violence. The system still relies on incarceration as a default setting, even in cases where it might be cheaper and more effective to use community-based alternatives. Roughly two-thirds of ex-offenders commit a new offense within three years of leaving prison. Probation and parole supervision are often empty rituals because underresourced officers are routinely asked to supervise dozens of offenders simultaneously. There is no shortage of problems still to be addressed.

Unless criminal justice officials are willing try new approaches and risk failure, we are unlikely to make a dent in these problems. According to Kevin Burke, a Hennepin County (Minnesota) trial court judge whom we will meet again in our chapter on drug courts, "We have to fight against the tendency to always choose a traditional approach to problems. A more common source of failure in criminal justice is an unwillingness to try anything different" (Berman and Fox 2008).

The need for consistent and persistent risk taking was perhaps best communicated by a Nike commercial made at the height of Michael Jordan's career. In the spot, Jordan says, "I've missed more than 9,000 shots in my career. I've lost almost 300 games. Twenty-six times I've been trusted to take the game-winning shot . . . and missed. I've failed over and over and over again in my life. And that is why I succeed."6

In writing this book, we hope to encourage criminal justice reformers to fail over and over again so they will ultimately succeed.

1. Our point is not to disparage such books. Indeed, we helped write a book that explicitly seeks to sell a new idea (Berman and Feinblatt 2005).
2. While criminal justice has a lot to learn from fields like medicine, it should be noted that there are good reasons why a more scientific approach does not prevail in criminal justice, including the logistical and financial obstacles to conducting randomized trials on a routine basis.
3. Thomas M. Burton, "Flop Factor: By Learning from Failures, Lilly Keeps Drug Pipeline Full." New York Times, April 21, 2004, p. A1.
4. Billy Bragg, "North Sea Bubble," Don't Try This At Home. Lyrics available at
5. In 1974, sociologist Robert Martinson and several coauthors published a metaanalysis of the evaluations from various rehabilitation programs; Martinson then wrote a widely read essay on the topic in The Public Interest (Martinson 1974). The conclusion was that there was no appreciable evidence of the programs' impact on recidivism. While many scholars have subsequently debunked this research, the idea that "nothing works" to change the behavior of offenders had a lasting impact, helping to discredit the idea of rehabilitation. Similarly, the urban problems of the 1970s and 1980s led many to conclude that American cities like New York were "ungovernable" (Cannato 2001).
6. The commercial can be found on YouTube at

Table of Contents

1 The Four Types of Failure
2 Failure amid Success
3 The Complicated Legacy of Operation Ceasefire
4 The Billion-Dollar Failure: Parole and the Battle for Reform in California
5 Beyond Simple Solutions:Mastering the Politics of Tragedy in Connecticut
6 Defining Failure
About the Authors

What People are Saying About This

Malcolm M. Feeley

Greg Berman and Aubrey Fox demonstrate the need for experimentation—trial and error—in developing successful problem-solving programs. Their voice stands in sharp contrast to the bombastic cries and exaggerated claims of most so-called reformers. (Malcolm M. Feeley, Author, Court Reform on Trial and The Process is the Punishment, and Claire Sanders Clements Professor Jurisprudence and Social Policy Program, Boalt Hall School of Law, University of California, Berkeley)

Ira A. Jackson

Peter Drucker once observed that Babe Ruth struck out 1,330 times—but also hit 714 home runs. We needn't be discouraged by Berman and Fox's findings; in fact, there's probably more to learn by diagnosing and embracing and learning from failure than in thinking we can easily or glibly replicate success. At a time when clarity and transparency and trust are so lacking and so needed in policy and politics, this book can help us all see more clearly the seeds of failure and the ingredients necessary for clear-eyed and sustainable success in criminal justice reform that really works. (Ira A. Jackson, Henry Y. Hwang Dean and Professor of Management, Peter F. Drucker and Masatoshi Ito Graduate School of Management, Claremont Graduate University)

Joan Petersilia

This provocative book charts a promising path for criminal justice reform in this country. I can think of no other book like it, and I urge front-line practitioners, policymakers, and scholars to read it. As the nation faces severe budget cuts, the lessons learned from past failures seem more important than ever. (Joan Petersilia, Adelbert H. Sweet Professor of Law, Stanford University Law School)

Bill Bratton

Free of jargon and ideology, Trial and Error in Criminal Justice Reform: Learning from Failure offers a valuable contribution to the continuing discussion and debate about crime and what to do about it. It's an informed and realistic contemporary review of what reform efforts can, and cannot, achieve. It's not only a good read but a must read. (Bill Bratton, Former Commissioner, New York and Los Angeles Police Departments)

Customer Reviews

Most Helpful Customer Reviews

See All Customer Reviews