Obvious statement of the day: The criminal justice system can be really difficult to navigate. That is, if you can figure out what its various processes are in the first place—and they can sometimes be so unintelligible that a person might wonder if they’re actively designed to be confusing.
This is a particular problem in the pretrial court process. A judge decides whether a person should be released from custody while awaiting their day in court with guidance from a data-driven algorithm called a release assessment. But this crucial step of the criminal justice process is a black box: a mystery to the general public and difficult for court employees to use. So New York’s Criminal Justice Agency tapped the Brooklyn design agency Hyperakt to make a user-friendly interface that’s intended to help everyone—from lawyers to families of people who have been arrested—better understand the process.
The fact that CJA redesigned its site and associated tools isn’t completely out of left field. It’s part of a growing push to reform criminal justice in the United States, from a bipartisan criminal justice bill passed in 2018, to cash bail reform in California, New York, and New Jersey, to the election of progressive district attorneys in Boston, Philadelphia, Dallas, San Francisco, and more. At the same time, many tools that power the criminal justice system are coming under fire for perpetuating bias. CJA’s interface shows the role design can play in working toward a fairer system.
Navigating an opaque system
Consider the current pretrial process in New York City. You’ve been arrested and fingerprinted, and you’re awaiting arraignment. Before that happens, you’ll undergo a pretrial interview to determine the likelihood you’ll return for your court date, and whether or not you should be released on recognizance in the interim.
This interview is conducted by a staff member of the New York City Criminal Justice Agency (CJA) I mentioned a little earlier. It’s an independent nonprofit and quasi-governmental agency with a mission to “assist the courts and the City in reducing unnecessary pretrial detention,” according to its website. Within New York City’s criminal justice system, CJA is mostly involved in the pretrial process, and makes pretrial release recommendations based on that interview mentioned above. The scope of CJA is more broad than this process alone, however; it conducts research, offers resources to navigate the cash bail system (which was restricted under new bail reform legislation on January 1), manages a supervised release program in Queens, and notifies every person who has a pending case. According to Aubrey Fox, CJA’s executive director, the organization makes about a million pending case reminders a year.
Like many courtrooms across the country, the pretrial release recommendation is based on the results of an algorithm that calculates how likely it is that a person will return to court (although it’s just that, a recommendation, and the judge still has latitude to make his or her own ruling, says Fox). According to CJA, the release assessment has been in place for about 40 years. “But a lot of people have an issue with it because it’s a black box,” Fox says. “A computer is making the decision, but you don’t know why or how. But it has to be transparent. We want to explain to the public what went into the algorithm.”
What with the cash bail reform legislation and conversation about closing Rikers Island, Fox says that he’s seen a lot of interest in how the court operates lately. Now, “Judges have to pick the least restrictive option and cannot select money bail on most cases,” Fox says. “So anyone would know what went into the algorithm and how it was produced.” It seemed like an optimal time to push toward transparency and improve the experience of the site. That’s when the organization brought on Hyperakt.
Breaking the black box open
The redesigned release assessment was part of an enormous research process that looks at over 1.5 million arrest records, according to Fox. So how does it work? The tool uses an individual’s past data like years since last bench warrant, felony convictions, and connection to community, to evaluate a scored questionnaire. The higher the score, the stronger the recommendation for release on recognizance. This updated release assessment “provide[s] judges and attorneys with impartial data so they can make more informed decisions,” and “a more fair and accurate pretrial decision making aid,” according to Hyperakt and CJA respectively. Overall, the assessment recommends that over 85% individuals be released on recognizance while they await their court date, says Fox. The other 15% fall into release recommendation categories which may require pretrial conditions, like supervised release. It doesn’t necessarily mean that other 15% will simply be detained.
Data aside, user experience was still an issue. Whether due to budgetary or human resource restrictions, the CJA had the same problem as a lot of government websites: It was confusing and cumbersome to navigate. That doesn’t make for a very transparent experience. Hyperakt sees court employees, academics, and researchers as the primary users of the tool, but the redesign allows the general public to see how the assessment is made and perform the release assessment themselves. When taking on a complicated data set with public value, that’s the ultimate challenge: How do you make it accessible for the most people?
Hyperakt streamlined more than the Release Assessment. In addition to redesigning the website, it also redesigned the CJA’s Case Assessment Tool (CAT), which “allows users to explore data on prosecuted cases in New York City.” The previous version of the tool was cumbersome to those outside academia or criminal justice, and CJA wasn’t happy with its color scheme, design, or functionality, says Hyperakt. So Hyperakt worked to make the tool more intuitive and easy to navigate.
According to Deroy Peraza, creative director and partner of Hyperakt, the team followed a few core design principles to streamline the user experience of CJA’s tools:
- Simplify language and minimize use of jargon.
- Emphasize simplicity and efficiency through clear information hierarchy and purposeful interaction.
- Pace the delivery of information and break the experience down into steps that build on each other.
- Guide users with clear signposts.
Of course, simply putting the information out there isn’t enough. Internally, CJA is growing and saw the redesign as opportunity to show the value of what it does as a “neutral source of information” for the criminal justice system. “We felt a responsibility for information to be available in an attractive way that’s available to the public. Part of it is about giving people access to all this great data when there’s all these debates about data and criminal justice—it’s good for people to have information to improve the quality of that debate,” says Fox. “It’s not easy to figure out how to depict data.”
Transparency is great. But what if the data is flawed?
In fact, it’s been reported that relying on historical data that powers algorithms like the release assessment itself perpetuates bias in the criminal justice system. “Those patterns are statistical correlations—nowhere near the same as causations. If an algorithm found, for example, that low income was correlated with high recidivism, it would leave you none the wiser about whether low income actually caused crime. But this is precisely what risk assessment tools do: They turn correlative insights into causal scoring mechanisms . . . the algorithm could amplify and perpetuate embedded biases and generate even more bias-tainted data to feed a vicious cycle,” wrote Karen Hao in Technology Review. Meanwhile, Science Friday reported that empirical risk assessments are “often inaccurate and based on flawed data.”
I asked Fox what he made of this critique. “While my agency has been using a release assessment for over four decades and the updated version is a clear improvement by any conceivable metric, we learned a lot from critics of these types of tools that informed our final design,” he says. “So we are grateful for all the thoughtful critiques that are out there, and for the willingness of a wide range of people (including critics) to give feedback and help shape what became the final version of the update release assessment.”
The release assessment is “backed by local data, real knowledge, and experience. All of that is great,” Fox continues, “but what Hyperakt helped do is translate that into design principle.” A key component of design is to clarify and simplify. With greater visibility into how the pretrial process actually works, the public, professionals, and everyone in between can look into the black box and give a data-based justice system the evaluation it needs.