Thinking About AI Orchestration

Goal - Develop a clear mental model about AI Orchestration

Bird’s Eye View

Let’s take a step back and look at what really happens in an agentic system or workflow.

The user gives some input, a set of actions are taken on the basis of it and we get an output. That’s just how any software system works. In fact, I just described functions.

• If there is an AI model in there somewhere, we can call it an AI workflow or an AI Agent
• If the set of actions are fixed, it is an AI workflow
• If we let the model decide the set of actions dynamically, it is an AI Agent.

If we were dealing with just AI workflows, the abstraction of composite functions fit perfectly. However, agents make things a little bit more diffcult.

The Agent Problem : Indeterminate Paths

In traditional programming, we build complex logic by composing functions. The output is derived from applying a series of functions sequentially to the input.

Output=f(g(h(Input)))

This model works flawlessly for fixed-action AI Workflows. If you define a sequence of fixed actions, such as

 \text{ValidateInput} \rightarrow \text{CallLLM} \rightarrow \text{FormatOutput}

you simply chain these functions together. The structure is linear and known upfront. This does not work with AI Agents because we do not know in advance what the steps may be.

The Problems

• Dynamic Branching - An agent may choose to call a tool, delegate or ask the user for clarifications
• Self Correction - An agent may choose to retry and fix errors
• Parallelism - There may be multiple parallel tool calls or sub agent delegations

Therefore, we need an abstraction that can handle these uncertainities. Something that is more like an interconnected network of steps rather than straight lines.

The Solution : DAGs

We need an abstraction where

• The next step can be dynamically decided by the previous step
• There can be multiple “next steps” running in parallel
• Information is preserved robustly across steps

Directed Acyclic Graphs have three components

• State : Think of this as the information store of the graph
• Nodes : These are the steps that we can take
• Edges : These define the flow or arrows between nodes

DAGs work well for the use case of AI orchestration. If you understand states, nodes and edges, you are halfway there. In the next step, we will dive into the core abstractions of Cascaide.