Hacking the Judge's Assistant: The First Case of "Prompt Injection" Sanctioned in Courts

Gaston Rey

 Introduction: The New Frontier of Procedural Bad Faith

Historically, procedural bad faith and misconduct in courtrooms relied on traditional ruses: hiding a document among thousands of pages, forging a signature, or delaying proceedings through manifestly groundless motions. However, digitalization and the sudden rise of generative Artificial Intelligence (AI) within the judiciary have opened an unprecedented front. Today, procedural traps no longer solely aim to deceive the human eye of the judge, but rather to subvert the cognitive capabilities of the algorithmic systems assisting them.

In May 2026, Brazil became the global stage for the very first precedent sanctioning this behavior. To the best of publicly available knowledge, this appears to be the first reported judicial sanction specifically imposed for a prompt injection attack directed at a court AI system. The case exposes the deep dangers of a cyber-technique known as "prompt injection" within the legal domain and raises urgent alarms regarding the ethical responsibility of legal practitioners when interfacing with modern technology.

The Concrete Case: Detection in Parauapebas

The epicenter of the regional milestone took place within the Regional Labor Court of the 8th Region (TRT8), specifically in the 3rd Labor Court of Parauapebas, Pará. Inside the labor lawsuit registered under docket number ATOrd 0001062-55.2025.5.08.0130, two litigating attorneys attempted to alter the natural workflow of the judicial support software.

Upon processing the initial petition, the court's generative AI system—named "Galileu"—detected a severe anomaly. Distributed covertly throughout the digital document was text rendered in white font over a white background. For any human reader (the judge, the clerk, or the opposing party), the text was completely invisible at first glance on a screen or when printed. However, for the underlying code-reader of the Large Language Model (LLM), the embedded instructions were perfectly legible.

The Anatomy of Deception: What Did the Hidden Command Say?

As a concept, the technique known as "prompt injection" involves embedding hidden or adversarial commands within text inputs to override or modify the original system instructions governing an AI application. In this particular case, the attorneys inserted distributed fragments which, when combined sequentially by the machine, formed a direct, fraudulent mandate aimed squarely at the court's AI software.

The recovered invisible text contained a blatant typographical error in the original Portuguese and read textually:

The objective was clear: they intended that when the software processed the complaint to assist the judge's chambers in drafting a summary, or when the opposing party utilized automated tools to evaluate the claims, the AI would suffer a "cognitive hijacking." The system would have generated a flawed analysis or a weakened response, deeply compromising the procedural equilibrium and equity of the adversarial system.

The Technological Reaction and Galileu's Counter-Defense

Far from succumbing to the fraudulent command, the Galileu system—originally developed by the Regional Labor Court of the 4th Region (TRT4, Rio Grande do Sul) and adopted nationwide by the Higher Council of Labor Justice (CSJT)—proved the efficiency of institutional safety guardrails.

The algorithm not only identified the invisible text and reconstituted the malicious command from its scattered parts, but it also automatically blocked the processing of the adversarial content to neutralize the exploit. Immediately after, it generated an explicit, high-priority technical alert for the human operator in the judicial chamber.

The Human Verdict and Procedural Sanctions

The presiding judge, Luiz Carlos de Araújo Santos Júnior, acted firmly under the fundamental premise of modern digital law: technology proposes, but humans dispose. Upon receiving the software's alert, the magistrate executed a manual inspection of the source file to verify the technical fraud with his own eyes.

Once the maneuver was verified, the court ruled that the conduct constituted an act injurious to the dignity of justice and a clear case of procedural fraud. The drafting and submission of legal petitions remains an exclusive domain of human accountability and professional ethics. Consequently, the magistrate imposed severe penalties:

·       A joint punitive fine imposed on both attorneys equivalent to 10% of the case's total value, amounting to approximately R$ 84,250 (roughly $15,000 USD).

·       An immediate referral to the Brazilian Bar Association (OAB - Pará Section) and the TRT8 Corregedoria to initiate formal disciplinary proceedings, which may result in temporary suspension from practicing law.

The Regulatory Shield: CNJ Resolution No. 615/2025

This ruling does not exist in a regulatory vacuum. Brazil has positioned itself as a global pioneer in AI governance within the judiciary under the strict guidelines of the National Council of Justice (CNJ). Specifically, the recent Resolução nº 615/2025 issued by the CNJ clearly demarcates the boundaries of generative tools in court settings, mandating:

·       Mandatory Human Oversight: No AI system can autonomously issue rulings, decrees, or draft minutes without strict human verification by a judicial officer.

·       Risk Mitigation and Transparency: Courts must implement robust auditing mechanisms capable of identifying adversarial attacks (such as prompt injections).

·       Proactive Accountability: The use of AI tools never absolves the litigating parties of their core duties of loyalty, honesty, and procedural good faith.

Thus, the fact that Judge Santos Júnior rooted his final decision in his own personal validation of the hidden text stands as a textbook example of the "human-in-the-loop" doctrine required by the CNJ.

Final Reflection: The Future of Legal Ethics

The ruling from Parauapebas marks a clear watershed moment in comparative digital law. It demonstrates that digital illiteracy is no longer an acceptable defense for litigants, and that traditional codes of professional ethics must be adapted immediately to the AI era.

Although the ruling emerged from the Brazilian labor judiciary, the underlying problem is universal. Any court, law firm, arbitrator, or regulator that relies upon LLM-based assistants becomes potentially vulnerable to adversarial instructions embedded in documents submitted by parties.

Attempting to manipulate court software via prompt injection is the modern equivalent of physically altering a paper file or trying to deceive a judge inside their private office. As judiciaries across Latin America and the world integrate language models, lawyers must realize that technological transparency is an unyielding ethical requirement. Those who attempt to beat the machine by cheating the code will ultimately lose against the law of human oversight.

More significantly, the court appears to have implicitly accepted that manipulating a judicial AI assistant is legally equivalent to attempting to manipulate the judicial process itself.

That is the true conceptual leap. The conduct was not punished because it damaged a computer. It was punished because it sought to interfere with the administration of justice by exploiting a technological instrument used in the adjudicative process. Therein lies the ruling's most significant doctrinal contribution.

PRIMARY REFERENCES FOR CONSULTATION & LINKING

• Conselho Nacional de Justiça. (2025). Resolução nº 615/2025: Dispõe sobre a utilização de inteligência artificial no Poder Judiciário. Available at: atos.cnj.jus.br

• Tribunal Regional do Trabalho da 8ª Região. (2026). Ruling by Judge Luiz Carlos de Araújo Santos Júnior in case ATOrd 0001062-55.2025.5.08.0130. Parauapebas, PA.

• Tribunal Regional do Trabalho da 4ª Região. (2026). 'Galileu: Sistema identifica tentativa de manipulação em petição e alerta magistrado.' Available at: trt4.jus.br

• OWASP. (2025). Top 10 for LLM Applications: LLM01 - Prompt Injection. Available at: genai.owasp.org

(*) Gaston Rey is an attorney and legal analyst specializing in digital law and new technologies applied to the judicial ecosystem.