Suspects in custody are “legally innocent people,” said Diwan, 30, who hopes to one day put his software to use in Ontario’s bail courts. “We just want to find a way to make the system better, faster, economical.”
Proponents of this kind of program say machine learning would save time and money by quickly identifying people who should be released, speeding up bail hearings, reducing the number of people in jails and freeing up courts to focus on defendants who should have a full, contested hearing. All that with less bias and without affecting the crime rate.
At least, that’s how it should work. Some researchers argue that risk-assessment algorithms can replicate bias already present in the criminal-justice system. Advocates are also concerned about the accuracy, reliability and transparency of such tools, as well as data collection and privacy.
Despite calls for caution, bail algorithms are now in use in certain U.S. states, including New Jersey, where a tool has resulted in more pretrial releases. Machine-learning software such as the proprietary COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) tool are also being used at sentencing hearings.
It is only a matter of time before machine-learning technology will make its way into Canadian bail courts, experts say, and we need to be prepared.
The efficiency promised by artificial intelligence may be especially seductive in the wake of Legal Aid Ontario budget cuts by the Ford government. Those cuts led to a much-criticized decision by Legal Aid Ontario to stop funding private lawyers for bail hearings, instead shifting those duties almost entirely onto duty counsel.
“My concern is that, in overcrowded bail courts where duty counsel are completely overwhelmed, this will enhance the appeal of algorithmic risk-prediction tools,” said lawyer Jill Presser, an expert in artificial intelligence and the law. “And I think that is something we should all be really concerned about, because these tools are fallible.”
The federal government is already piloting artificial-intelligence systems, but for a different purpose. The programs are intended to triage immigration and refugee applications into two streams — one for processing, one for review — but limited information has been made publicly available, according to a 2018 report from the University of Toronto’s CitizenLab.
The Government of Ontario did not say whether it is considering the use of machine-learning technology in bail court. Spokesperson Emilie Smith said the Ministry of the Attorney General “remains committed to continuously improving and to investigating modern methods of case evaluation, such as risk assessment tools, while ensuring public safety and the rights of the accused.”
Diwan said governments need to move forward with machine-learning technology because people are staying in provincial jails much longer than necessary on the taxpayer’s dime. Last year in Ontario, bail cases that resulted in a release order took an average of 3.5 days; those that resulted in a detention order took an average of 13.5 days.
Those can be very difficult days. Research in both the U.S. and Canada suggests being stuck in jail can make it hard to prepare for trial and even push innocent people to plead guilty to get out faster.
But it’s not just about speed. Such risk-assessment tools are touted to be more knowledgeable and less biased, relying on vast amounts of data only a machine could process.
“Crown counsel are just human beings at the end of the day, and human beings aren’t aware of all of the variables they take into account when they make a decision,” Diwan said.
“Our tool would be able to take all of the information into account and have the benefit of all of the information that came before it, rather than one person’s individualized experience.”
SmartBail is still in the earliest stages. Diwan, who graduated from McGill’s law school in 2017, had the idea last fall and joined Ryerson University’s Legal Innovation Zone at the end of January.
The software is currently training itself with data from the United States. For it to be useful in Ontario, it would need access to data from the Ministry of the Attorney General — something Diwan is hoping to get permission to use.
The data is the key, and the more data the software has, the better, Diwan said.
But data is also the biggest obstacle to overcome.
“If we were to train an algorithm on previously collected data, it would be trained on biased data,” said Akwasi Owusu-Bempah, a sociology professor at the University of Toronto specializing in race and the criminal-justice system.
“We’d have to do a fair bit of groundwork not only to train it on data but to collect reliable data upon which to train an algorithm.”
He said data on people violating bail conditions would have to include people who were caught as well as people were not caught — and the latter is a difficult group to study since it would require people to disclose their violations to government researchers.
Owusu-Bempah said he doesn’t disagree that an algorithm could assist in making unbiased decisions but the extent of the bias that exists in the bail process has not been studied in depth.
“To assume that it is not, I think, would be a rather foolish assumption based on what we know about bias in our society generally and in the criminal-justice system,” he said.
He notes that research by the Ontario Human Rights Commission into racial profiling by Toronto police found Black defendants may be released with more bail conditions, which could increase the chances of being arrested for violating those conditions.
Defendants with addictions are more likely to violate conditions that prohibit drug or alcohol use or going to specific areas, according to guidance from the federal prosecution service released earlier this year. That can lead to arrest and periods in custody that lower their tolerance for opioid use, putting them at heightened risk of an overdose upon release, the service warned.
And in the 2018 Supreme Court of Canada case of Ewert v. Canada, an Indigenous man argued that the long-standing risk-assessment tools used by the Correctional Services of Canada — which do not use machine-learning — for his parole hearing were unfair because they were tested on non-Indigenous populations. The court agreed that Correctional Services of Canada breached its obligation to research the validity of the tools despite long-standing concerns around cultural bias.
“While these technologies might seem very attractive and a quick fix, we actually have a lot of work to be done before one of them could be utilized effectively,” Owusu-Bempah said.
Complicating matters further, such algorithms continually change as they gather more information.
“If you have someone using a typical risk-assessment tool, those tools tend to be validated over years and years,” said Ryan Fritsch, a lawyer researching AI for the Law Commission of Ontario. “With machine learning, you might essentially have a different algorithm operating on every individual because it’s going to change and evolve over time. So practically speaking, does that mean I have to call an expert every time I want to challenge the AI?”
Diwan said SmartBail would be transparent and open for auditing, examination and critique.
That would avoid the problems encountered in the U.S. with COMPAS. In 2013, a defendant tried to challenge the harsher sentence he received due to his high-risk score, arguing “the proprietary nature of COMPAS prevents a defendant from challenging the scientific validity of the assessment.” The Wisconsin Supreme Court ultimately ruled against him but urged caution in the use of algorithmic risk-assessment tools.
Fritsch is also concerned about which factors are looked at to determine how “risky” a person is, how those factors are weighted, how such data is collected and from whom.
“Vulnerable groups in our society are subject to much greater surveillance than others, and it is simply because they come into contact with government services more, social services more, court services more, health care services more and police services more,” said Fritsch.
“And a health-care issue or a social-support issue now in the hands of the criminal-justice system becomes a risk issue.”
It’s also unclear where the collected data will surface. A Toronto Star investigation found that police record checks divulged information about 911 mental-health calls and unproven allegations, resulting in a new law coming into effect last year, Fritsch said.
Diwan said unnecessary or invasive data can be avoided if SmartBail considers only the information Crown counsel already uses to make bail determinations.
He added that a tool like SmartBail would not replace the decision-making of a justice of the peace or judge; it would just be one factor to consider. The presence of defence counsel, he said, provides an important balance and ensures that the necessary context is still available.
Even if the design is perfect, there are practical problems to overcome, including “automation bias” — relying too much on the tool in increasingly busy bail courts.
“The justice system needs to remain a human process,” said Presser, the lawyer who writes about artificial intelligence.
The level of acceptable accuracy for a tool needs to be carefully examined, Presser said, as population-level statistics cannot ever truly reflect an individual.
“When we talk about liberty, how much inaccuracy are we prepared to tolerate?”
For experts including Presser and Fritsch, the focus needs to be on proactively establishing regulations and legislation to govern the use of such tools before they are implemented rather than waiting on constitutional challenges to be launched.
“We litigate at the back end because the system has failed,” Presser said.
Earlier this year the Treasury Board of Canada released a directive on the ethical use of automated decision-making tools, including the introduction of an “algorithmic impact assessment” that would help identify and reduce the risks associated with the technology. Ontario does not yet have a similar set of rules.
“Rather than trying to fight new technology, I would rather ensure that we develop and implement technology intelligently,” Owusu-Bempah said.
“That means acknowledging a lot of what we don’t know now and working on making significant expenditures to implement it properly. And ensuring whatever technologies are developed are open so researchers can actually take a look at their inner working.”
He emphasized that the timing is short and the stakes are high to get these regulations right.
“I think the introduction of technologies like this is inevitable, and therefore we need to ensure they are implemented in a fair and just manner,” Owusu-Bempah said. “We have to remember that these are people’s lives.”