New Developments Regarding Risk Assessments

Risk assessment tools are starting to take root in the criminal justice system. They’re used to make decisions about pretrial release, sentencing, and the level of supervision or custody to which a defendant will be subject. Some of the results are encouraging. For example, Mecklenburg County uses a risk assessment developed by the Laura and John Arnold Foundation to help make pretrial release decisions. The pretrial services office there reports that the risk assessment has contributed to “transformational change” in how pretrial justice is administered, with fewer secured bonds being imposed the jail population falling with no harm to public safety. Based in part on Mecklenburg’s success, the North Carolina Commission on the Administration of Law and Justice encouraged the creation of a pilot project that would “implement and assess more broadly . . . an empirically derived pretrial risk assessment tool.”

Risk assessment tools are not without controversy, however. For example, in a 2014 speech, then-Attorney General Eric Holder warned that the use of risk assessments at sentencing “may inadvertently undermine our efforts to ensure individualized and equal justice.  By basing sentencing decisions on static factors and immutable characteristics – like the defendant’s education level, socioeconomic background, or neighborhood – they may exacerbate unwarranted and unjust disparities that are already far too common in our criminal justice system and in our society.”

Readers interested in risk assessments may be interested in two recent developments:

  1. The Supreme Court of New Mexico recently affirmed a trial judge’s order detaining a murder defendant without bail. In the course of its opinion, the court noted that the judge’s decision was supported by the results of “the Arnold Public Safety Assessment (PSA) . . . a nationally recognized scientifically validated risk assessment instrument that courts in an increasing number of jurisdictions use as an aid, though never as the only factor, in making detention and release decisions.” State v. Groves, __ P.3d __, 2018 WL 359473 (N.M. Jan. 11, 2018). The opinion is noteworthy because it reflects courts’ increasing comfort with and reliance on risk assessment tools.
  2. A new article in the journal Science Advances evaluated the accuracy of COMPAS, a widely-used risk assessment tool that uses “137 features about an individual” to assess the likelihood of recidivism. The tool’s overall accuracy in predicting recidivism is 65.2%. This is somewhat better than chance, but the authors found the risk assessment to be “no more accurate or fair than predictions made by people with little or no criminal justice expertise” who were provided with some basic information about the defendants for whom they were making predictions. The researchers also found that “a simple linear predictor provided with only two features,” age and criminal history, was as accurate as the more complex tool.

Obviously, this new research doesn’t establish that all risk assessments are inaccurate, much less biased in the way that Attorney General Holder suggested. But it appears that some risk assessment tools work better than others, and that determining whether a particular tool works well requires serious scientific scrutiny, which can be facilitated by transparency about how the tool works. I remain cautiously optimistic about risk assessments as a source of information to support judicial decision-making. But I’m interested in others’ thoughts and experiences, especially regarding how things are working in Mecklenburg County or in other jurisdictions where risk assessments are in use.

6 thoughts on “New Developments Regarding Risk Assessments”

  1. North Carolina’s risk assessment tool is fairly decent, however there is a portion that is based entirely on the answers provided by the defendant. Since the questions never change, many defendants have “figured out” how those questions are to be answered to “lower their risk” during probation or post-release supervision. This aspect of the tool should be revamped at some point in time.

  2. According to the National Institute of Justice:

    Bureau of Justice Statistics studies have found high rates of recidivism among released prisoners. One study tracked 404,638 prisoners in 30 states after their release from prison in 2005.[1] The researchers found that:

    Within three years of release, about two-thirds (67.8 percent) of released prisoners were rearrested.
    Within five years of release, about three-quarters (76.6 percent) of released prisoners were rearrested.
    Of those prisoners who were rearrested, more than half (56.7 percent) were arrested by the end of the first year.
    Property offenders were the most likely to be rearrested, with 82.1 percent of released property offenders arrested for a new crime compared with 76.9 percent of drug offenders, 73.6 percent of public order offenders and 71.3 percent of violent offenders.

    So can’t you just find they will all re-offend within 5 years and be 76.6% accurate?

  3. I’m a big fan of this new opportunity, especially for the use of Magistrates, and hope to see further implementation! Thank you for this article and please keep us posted on this topic!


Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.