log in | about 
 

I have some thoughts related to the recent craze to automate everything with agentic skills. On a plus side, I think it can be very useful when done correctly. On the negative side, it is often done incorrectly and only adds more technical debt. Instead of solving systemic issues first, people are just slapping an AI band-aid in the hope to stop a gushing artery.

Using an agent (and essentially an LLM) as the primary orchestrator for deterministic workflows sacrifices many of the properties that decades of good software engineering have optimized for. We lose predictability, reproducibility, testability, debuggability, and the ability to reason formally about system behavior. More subtly, we often sacrifice encapsulation: instead of components exposing explicit, versioned interfaces, the LLM needs to understand implementation details and prompt conventions. This increases coupling between components, makes the overall system less coherent, and turns what should be local implementation changes into workflow-wide prompt adjustments. One should not forget that natural language is imprecise and ambiguous so it is in principle impossible to define complex workflows accurately.

Humans often lack understanding of how the agent interoperates with different components and it is hard to debug or predict how LLM will behave. For workflows where an execution sequence can be well defined (which includes quite a few workflows), the agent's role should be much narrower. It should primarily act as a thin interface layer: gather user intent, fill slots, perform lightweight validation, and invoke a well-defined sequence of deterministic scripts or services. The workflow itself should be explicit, documented, version-controlled, and independently executable. Mixing code-snippets and natural language commands in a single SKILLS.md file feels like an engineering antipattern. It tightly couples deterministic scripts with a non-deterministic sequence of steps defined by the agents and makes debugging extremely difficult. What makes debugging difficult is not just the nondeterminism. It's that the control flow becomes implicit rather than explicit.

Agent-based orchestration has an important operational requirement. If something goes wrong (whether due to an LLM hallucination, a provider outage, a rate limit from Codex or Claude Code, or simply an unexpected edge case) a human engineer should be able to resume or rerun the workflow manually by invoking the same documented scripts. The system should remain inspectable and recoverable because the deterministic backbone exists independently of the agent. In my experience, this requirement is frequently overlooked.

Agents still have valuable roles outside the critical execution path where their use is more flexible. They can assist with debugging failures, investigating unexpected outcomes, suggesting fixes, summarizing logs, or helping operators understand complex system state. In those cases, occasional mistakes are acceptable because a human is already in the loop. Moreover, when something goes wrong, it is better to obtain a quick (though possibly not 100% precise) assessment of the situation rather than waste days to obtain a correct one. But the production workflow itself should remain deterministic whenever the problem domain permits it.