The Dark Factory is a Trap: Why AI Coding Needs Self Illumination
Maybe the future is not lights on or lights off.
Recently, the idea of the “lights off” software factory has gone viral. The concept is simple: AI coding agents plan, build, test, and review software, while no human writes or reads the code. The factory runs continuously, but it operates in the dark. You have to trust the agents, because you no longer know what is happening underneath.
The reference article, Why Software Factories Fail, argues this is exactly the problem. A lights off factory removes human code review. A “lights on” factory keeps humans as the final gatekeepers, ensuring someone understands the software and can fix it when it breaks.
I understand the desire to keep the lights on. If nobody understands the system, catastrophic failures are inevitable. But I remain skeptical. Models improve every single day. I have already seen Fable complete complex tasks in one shot, and we are only halfway through 2026. Who knows what models will do next?
This makes me wonder if the choice is really between keeping the lights on manually and turning them off completely. Why choose either? Why not build machines that are self illuminating?
The Four Zones of AI Coding
Think of this as a graph. The horizontal axis measures visibility: on the right, a human reviews the work (lights on); on the left, no human reviews it (lights off). The vertical axis measures the builder’s technical experience, from nontechnical at the bottom to highly technical at the top.
The top right is the current ideal. You are an experienced coder who reviews the generated code. Because you understand what was built, the risk of breakage is low.
The top left is a lights off factory run by highly technical coders. Consider Boris Cherny, the head of Claude Code. In an interview about his workflow, he noted he no longer writes code by hand, starting much of his work on his phone. Instead of writing lines, he sets up loops, agents, and routines. This is loop engineering. You design the system that prompts the agent, checks its work, and decides the next step. Claude Code builds Claude Code, while humans steer from above.
The bottom left is nontechnical vibe coding. People build without understanding the underlying mechanics and never check the code. When it breaks, they ask AI to fix it, hoping the model can untangle its own mess.
The bottom right is the most interesting. You are not technical, but you want to keep the lights on. How?
A nontechnical person cannot review code line by line. Even as a computer science undergraduate who loved algorithms, I hated debugging a misplaced comma. But today, a nontechnical person can ask AI to explain a folder, a file, and their relationships. They can review whether the structure makes sense, if the product functions correctly, and if best practices were followed. It is not line by line reading, but it is not blindly turning the lights off either.
What I Mean by Self Illuminating
Those four zones represent the present. A self illuminating software factory is the future.
I do not mean a factory left alone in the dark. I mean a system that builds and improves itself while making its work visible. It knows its codebase, its customers, its business model, and its intended outcomes. It can explain what it changed, why, what it tested, and if the result succeeded.
Self illumination is about legibility, not just autonomy. The system must learn, repair, and evolve without constant human intervention, but it must also clearly show the evidence of its completion.
This factory relies on two connected components:
1. Context Aware Models
It should not be one model doing everything. It should be a group of models routed according to the task and context. Cursor Router is an early example, directing requests to different models based on the work required. Over time, these models must deeply understand the company through proprietary context: the codebase, the customers, and the history of decisions.
2. The Router and Harness
The harness is the environment surrounding the models: the agents, tools, loops, checks, and memory. The model determines capability, but the harness turns that capability into useful work. Why Software Factories Fail correctly argues that this relationship is critical.
This is where the recent conversation about graph engineering is valuable. A graph forces us to make the process explicit. Which agent works? Which checks? What becomes memory? What happens upon failure? Graph engineering makes loops, handoffs, and feedback visible enough to improve.
Models Are Already Moving in This Direction
Kimi K3 is fascinating because its process feels different from standard generation. It appears to move through planning, decision making, component design, and sample code within its own reasoning process before returning an answer. An agentic workflow is happening inside the model itself.
This makes me wonder how outdated our definition of “review” will become. If models can plan and iterate internally, and agents can build and test in loops, what does a human actually need to review?
The Real Review Is the Specs and the Evals
In the agentic coding era, humans will not read codebases line by line forever. The most important review happens before and around the code. The human role shifts from writing and reviewing syntax to defining the boundaries and the tests.
They ask four essential questions:
Target: What exactly are we trying to build, for whom, and in what context?
Acceptance: What must be true for this to count as done? (Ideally written as binary, observable conditions.)
Evidence: How will we prove those conditions are true? (The test cases, environments, and pass thresholds.)
Guardrails: What must never happen, and when must the system stop or escalate?
Notice what is missing from that list: methods. Humans specify outcomes, proof, and boundaries; the model and harness choose methods. The harness, the specifications, and the goals are paramount. The better we define these, the less we need manual code review.
“Lights on” does not have to mean reading every code diff. “Lights off” does not have to mean trusting an opaque system. A self illuminating software factory keeps the work visible through specs, evals, memory, and feedback loops, even as it achieves full autonomy.
That is the factory every company should build.
Till next time, Cheers!
Previous column articles can be found here: