Australian legal AI startup Barrister AI scores higher than Harvey and Westlaw in new High Court research benchmark
BarristerAI
Key Facts:- Barrister AI has published the High Court Benchmark 3, in which its research pipeline averaged 65.6 out of 100 across ten decided High Court of Australia appeals, outperforming Harvey (51.4), Westlaw CoCounsel, and several frontier AI models including GPT-5 and Claude.
- The benchmark tested legal research and reasoning on contested questions from High Court appeals, with each tool answering questions before judgments were handed down or with post-decision knowledge removed from the platform.
- Answers were scored out of 100 by a panel of three AI models using a median score, rewarding identification of relevant authorities, the decisive issue, reasoning through both sides, and the advised result.
- Barrister AI's platform covers more than 500,000 cases from over 20 courts and tribunals and more than 35,000 Acts and instruments, with every citation linking to the exact paragraph of the referenced judgment.
- The company acknowledges limitations, noting the benchmark is its own, covers only ten decided appeals each answered once, and that answer keys and marking were produced by AI without lawyer review.
Barrister AI's research pipeline averaged 65.6 out of 100 across ten decided High Court appeals, against 51.4 for Harvey (nine cases) and 26.9 for Westlaw (CoCounsel), in a benchmark published with its logic.
Sydney, New South Wales - Barrister AI, an Australian legal technology platform for barristers and solicitors, has published the Barrister AI High Court Benchmark 3, a research benchmark that puts pending High Court of Australia appeals to its own system and to other legal research tools and AI models, including Harvey and Westlaw (CoCounsel). Across ten appeals, answered before they were decided or with knowledge of the result removed from the platform, Barrister AI's research pipeline averaged 65.6 out of 100, the highest average among the tools that answered at least four of the ten cases.
On the nine cases both answered, Barrister AI averaged 66.8 to Harvey's 51.4, a 30% advantage. It also averaged 144% higher than Westlaw (CoCounsel), and higher than the general-purpose frontier AI models GPT-5.6 Sol (by 37%), Claude Fable 5 (by 40%), GPT-5.5 (by 43%) and Claude Opus 4.8 (by 91%). It also averaged 30% higher than Legora over the seven cases both answered.
Every score and the method are published on the benchmark page.
“We think we are the best legal research platform on the market. We have built it from the law up, and trained it to think like a barrister thinks and expose its reasoning and what authorities it relied upon. These results speak to that,” said Matthew Kalyk, CEO of Barrister AI. The team also stressed that “of course there are limits to benchmarking, and the very best test is the one that lawyers do when they use our platform. However, these benchmarking results show just how well our platform compares on High Court questions against the analysis of the High Court.”
How the Benchmark Was Run
The benchmark tests legal research and reasoning on contested questions from High Court appeals, including issues on which the judges disagreed. Each tool answered each question once. All platforms were asked the questions before the Court handed down the judgments, or with their research limited to cases decided before each question's date. A key for each case was written by an AI model from the judgment and audited by a second AI model for accuracy and bias. Each answer was then scored out of 100 by a panel of three AI models from three labs, using the median score for each part of the answer.
The score rewards finding the right authorities, identifying the decisive issue, reasoning through both sides and the advised result, which is worth 15 of the 100 points, so it is a composite research score rather than a forecast of outcomes. The primary goal is not to predict what a judge will find, but to provide the legal reasoning and authorities that the judge will find relevant.
Benchmarking Methods and Logic Published With the Results
The benchmark is Barrister AI's own, and ten decided appeals, each answered once, is a small set. The team also recognises that benchmarking is very particular to what the platform is trying to focus on and doesn’t speak to general quality. “For us,” says CEO and founder Matthew Kalyk, “the focus is entirely on delivering high quality, verifiable legal reasoning on highly complex questions and measuring that reasoning product against the very high quality legal reasoning of the judges of the High Court on the very same questions.” They have seen a continued trend across the two published benchmarks that their unique architecture is consistently outperforming on these complex questions.
Research Grounded in Australian Law
Barrister AI provides Australian practitioners with legal research grounded in case law and legislation, with every pinpoint citation linking back to the exact paragraph of the cited judgment. Its Briefs feature maps a matter, answers questions from the matter's own documents and drafts from them. The platform's index covers more than 500,000 cases from more than 20 courts and tribunals, and more than 35,000 Acts and instruments, updated daily. Barrister AI is an AI-assisted research tool, and practitioners should read the cited sources and exercise independent professional judgment.
More information is available on the Barrister AI website at https://www.barristerai.com.au/benchmark/high-court-3.
What is the Barrister AI High Court Benchmark 3?
It is Barrister AI's research benchmark that puts key legal questions in ongoing High Court of Australia appeals to Barrister AI and to other leading legal research tools and frontier AI models, then, when the case is decided, scores every answer out of 100 against what the Court held. Ten of the twelve appeals in the set have been decided and scored so far.
How did Barrister AI compare with Harvey in the benchmark?
On the nine cases both answered, Barrister AI averaged 66.8 to Harvey's 51.4. With nine shared cases and rules revised after the answers were read, the paired t-test gives p = 0.002 and a 95% confidence interval of 7.3 to 23.4 points. Of the 15.4-point lead, 7.5 comes from automatic citation marks, and length is not tested.
How was the Barrister AI benchmark scored?
Each answer was scored against a key written from the judgment by an AI model, and marked by a panel of three AI models, using the median score. The score combines authorities, the decisive issue, reasoning, both sides of the argument and the advised result, which is worth 15 of the 100 points.
What are the limitations of the benchmark?
It is Barrister AI's own benchmark and covers ten decided appeals, each answered once. Barrister AI answered after the Court decided while most other tools answered before, and the answer keys and marking were done by AI with no lawyer review. The marking rules were revised after the answers had been read.
About us:
About Barrister AI
Barrister AI is an Australian legal technology platform built for barristers and solicitors. It provides AI-assisted legal research grounded in Australian case law and legislation, with pinpoint citations that link directly to the cited paragraph of a judgment or provision, alongside a Briefs feature that maps a matter, answers questions from its documents and drafts from them. The platform draws on an index covering Australian courts and tribunals, Acts and instruments, updated daily.
Contact details:
Company: Barrister AI
Contact: Alexander Kalyk
Email: [email protected]
Location: Sydney, NSW, Australia
Website: https://www.barristerai.com.au/