From Novel Neural Networks to Algorithmic Due Process: The Architecture of Accountability.

Gastón Rey

Gastón Rey

— 7 min read
Share
From Novel Neural Networks to Algorithmic Due Process: The Architecture of Accountability.

1. The News: Rethinking AI Architecture from the Brain Up

A series of recent AI studies have increasingly examined how to transform neural networks as we know them. The dominant paradigm in deep learning has been density. Consequently, every neuron in one layer is connected to every neuron in the next, at the cost of consuming enormous amounts of energy. Sparse Autoencoders, mechanistic interpretability, adaptive sparse representations, and circuit-level analysis all point to the opposite direction. A study published in Neurocomputing, researchers from the Nature-Inspired Computation and Engineering (NICE) group at the University of Surrey introduced Topographical Sparse Mapping (TSM) and its advanced variant, Enhanced Topographical Sparse Mapping (ETSM).
Inspired by the vertebrate visual system and the spatial organization of the human brain, TSM connects artificial neurons predominantly to nearby or functionally related nodes rather than linking every neuron in one layer to all neurons in the next (which is the standard way in current models). The framework cuts energy consumption during training and inference to less than 1% of conventional dense networks. But this approach is not the only one.

The related works
Mechanistic interpretability. This is the research program that treats a neural network not as an oracle but as a mechanism: something whose internal parts can be identified, named, and causally tested. The ambition is not to describe what a model outputs but to reconstruct how it got there.

Circuit-level explanations. Once features are isolated, researchers can trace how they combine into circuits: identifiable computational pathways responsible for specific behaviors. This is where interpretability stops being descriptive and becomes causal.

2. The Current Paradigm: The Wall of Brute-Force and Opacity

To appreciate why these studies matter, we must look at the structural crises facing modern artificial intelligence. We can call that problem “The Energy and Compute Wall” or maybe the "fuerza bruta" (brute-force scaling) of large language models— they add billions of parameters, dense matrix operations, and consuming hundreds of thousands of kilowatt-hours of electricity for single training runs. In the main issue, this scale is ecologically and economically unsustainable. But in the legal side we have also the "Black Box" Opacity Paradox. Conventional deep neural networks are dense and polysemantic. Let me explain the main problem. Because every node is connected to every other node, individual neurons simultaneously participate in multiple, unrelated cognitive tasks. The result is an opaque system where it is virtually impossible to isolate why or how a specific output was generated.

3. The Broader Horizon: The 2026 literature: interpretability towards algorithmic due process

Here is the connection between this studes and current research. These AI research programs have converged on a common objective: mainly making neural networks not only more efficient but also more understandable. From Topographical Sparse Mapping to Sparse Autoencoders and mechanistic interpretability, a new generation of architectures is beginning to expose the internal pathways through which AI systems reach their conclusions. This shift has consequences extending far beyond computer science—it may provide a more promising substrate to the technical infrastructure necessary for procedural due process in algorithmic governance.

What convinces me that this is a mature trend is not merely the mere existence of these papers, but the standards they impose upon themselves. Let me walk through the five works I found most telling.

  1. Feature consistency as a scientific requirement. Song et al., in a position paper at ACL 2026 ("Mechanistic Interpretability Should Prioritize Feature Consistency in Sparse Autoencoders"), it confronts an uncomfortable fact head-on: SAE features have not always been reproducible across training runs. Their answer is not to abandon the tool but to impose a standard; features must converge reliably across independent trainings, and they propose a concrete metric (PW-MCC) to measure it, showing that high consistency is achievable with the right architectural choices. I want to underline this, because it is the intellectually honest core of my argument: the field is naming its own weaknesses and converting them into requirements. A feature that appears reproducibly across independent runs can become a reliable object of audit. A feature that appears only in one run cannot. That distinction (between anecdote and evidence) is precisely the distinction every legal system already knows how to handle.
  2. Interpretability as an adaptive capability: Yao, Zhang and Du (AdaptiveK, Findings of ACL 2026) introduce SAEs whose sparsity adjusts dynamically to the semantic complexity of each input. The more active features for complex contexts, the fewer for simple ones, with improvements in both reconstruction and interpretability over fixed-sparsity SAEs. So interpretability is no longer studied as a static property you either have or lack — it is being engineered as a capability that scales with the difficulty of the decision. Any lawyer will recognize the analogy: the intensity of required justification scales with what is at stake.
  3. Extracting the concepts that decide. The work I find most directly translatable into legal language comes from EACL 2026 ("Unveiling Decision-Making in LLMs for Text Classification"): using SAEs to extract the concepts that actually influenced a model's classification decision, and to evaluate the causality and quality of those explanations. Read that objective again as a jurist: which concepts were decisive for this decision?  That is not only a technical question  but a question every administrative judge in the world asks of a contested resolution.
  4. Interpretability as a governance tool. Safe-SAIL (Findings of ACL 2026) takes SAEs into safety territory, building a framework to identify the internal features associated with risk and safety-relevant behavior in large language models. The significance is institutional: interpretability is no longer pursued out of scientific curiosity alone. It is being built as practical infrastructure for governing AI systems.
  5. The area keeps self-correcting. Subspace-aware SAEs (2026) argue that traditional SAEs fragment certain internal representations, and propose architectures that better preserve the structure of concepts. Notice what the debate is about: the community no longer discusses whether mechanistic interpretability is possible but It discusses how to do it better. That is what a consolidating scientific field looks like.

4. A Normative Shift in Legal Governance: From Compute Efficiency to Accountability?

While the researches try to solve computational and environmental challenges, its implications for AI governance, auditability, and regulation can arguably be even more profound.

Until now, opacity was presented (in litigation, in regulatory debates, in compliance documents) as an inevitable limitation of the technology. "Deep neural networks are black boxes" functioned as a full stop: a fact of nature that law simply had to accommodate. Under the EU AI Act and analogous emerging frameworks, this created a structural tension: the law demands traceability, meaningful oversight, and explanation for high-risk systems, while the dominant architecture made granular auditing extraordinarily difficult, because their decision pathways cannot be audited in a granular way.

Sparse, topographically structured architectures fundamentally change this dynamic. We can identify two aspects of the problem that could be improve significantly.

a) Causal Traceability: By restricting activation to specific, isolated sub-circuits, sparse models allow auditors to reconstruct the exact path a model took to reach a conclusion.
b) Surgical Model Editing: If an audited model exhibits bias or generates unlawful outputs, engineers do not need to retrain the entire network or add fragile external patches. They can identify, prune, or reweight the specific faulty sub-circuit.

As one very wise guy once pointed out: you can't connect the disperse dots in the present but in the future. I would venture to say that it is not something limited only to individuals but to groups of people, in this case, researchers from the most diverse fields. At some point the dots can be connected in the future from totally different disciplines and paths.

This new neural architectural evolution of engineers may provide the exact technical foundation required to realize legal accountability frameworks developed by lawyers and policymakers. The latter has been designed by many researchers. In my case, I developed what I call “the Habeas Log” (its introductory article is currently under peer review). I mean, procedural mechanism designed to enforce due process in automated decision-making.

 The core objetive shifts the legal focus from static data protection ("what data do you store about me?") to dynamic procedural integrity ("how, under what specific logic, and through what verifiable pathway did the algorithm decide?").

I want to be careful here. Nothing in the 2026 literature "proves" those frameworks, and I am not claiming it does. What these research lines constitute is something more modest but, I think, more important: a technical substrate increasingly compatible with mechanisms of algorithmic due process.

a) Instead of returning abstract, uninterpretable heatmaps, sparse networks allow what I call the "Saliency Summary" to map onto concrete, monosemantic concepts, providing a genuinely readable explanation for affected citizens and judges.
b) Eliminating the "Technical Impossibility" Defense: System operators have historically evaded procedural accountability by claiming that modern AI is inherently unexplainable. Sparse, brain-inspired design proves that high-performance AI can be built with explicit pathways by design (auditability by design).

If the concepts that influenced a specific decision can be extracted and causally evaluated, then a saliency summary stops being an uninterpretable heatmap and becomes something a citizen (or a judge) can actually read. If sparse activation makes logging the inference path computationally trivial, the cost objection to procedural record-keeping collapses. And if interpretability is already being operationalized as safety and governance infrastructure, then demanding it legally is no longer asking the impossible — it is asking for the state of the art.

Which brings me back to the core thesis. The "technical impossibility" defense had its historical moment. That moment is ending — not because a court said so, but because the research community is dismantling its factual premise, one ACL paper at a time. When it fully ends, the failure to provide a verifiable account of an automated decision will no longer read as an unavoidable limitation of the technology. It will read as what it is: a deliberate failure of due process.

Sparse may solve the main problems of AI. At first glance, the question would be: is energy efficiency or explainability the primary driver for the next generation of neural architectures? Can be both at once?

But maybe they are the wrong questions. They are converging on the same architectures. The real question is whether our institutions will be ready to demand what the technology is learning to offer.

References:

Kamelian Rad, M., & Bauer, R. (2025). Topographical Sparse Mapping: Brain-inspired sparse neural architectures for efficient artificial intelligence. Neurocomputing. 

Le Bail, M., Dentan, J., Buscaldi, D., & Vanier, S. (2026). Unveiling Decision-Making in LLMs for Text Classification: Extraction of Influential and Interpretable Concepts with Sparse Autoencoders. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2026 Findings).

Song, X., Muhamed, A., Zheng, Y., Kong, L., Tang, Z., Diab, M. T., Smith, V., & Zhang, K. (2026). Mechanistic Interpretability Should Prioritize Feature Consistency in Sparse Autoencoders. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026).

Weng, J., Zheng, H., Zhang, H., Zhou, E., He, Q., Tao, J., Xue, H., Chu, Z., & Wang, X. (2026). Safe-SAIL: Towards a Fine-grained Safety Landscape of Large Language Models via Sparse Autoencoder Interpretation Framework. In Findings of ACL 2026.

Yao, Y., Zhang, H., & Du, M. (2026). AdaptiveK: Complexity-Driven Sparse Autoencoders for Interpretable Language Model Representations. In Findings of ACL 2026.

Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretability. (2026). arXiv preprint, arXiv:2606.06333. https://doi.org/10.48550/arXiv.2606.06333⁠

Read more