Agents Shouldn't Be Your Compute Layer

Every token your agent spends on file I/O is a token it isn't spending on the problem you hired it to solve.

This sounds obvious, but it is routinely ignored. When an agent downloads a video, processes it inline, and re-uploads the result, it is doing something deeply inefficient: using a general-purpose reasoning engine as a compute layer for a task that has no business being inside the model's context window.

What goes wrong

The consequences show up quickly and compound as workloads grow:

• Slower responses. File content saturates the context window. The model has less room to reason about the actual task.
• Higher costs. You are billed per token for work that should be billed per job. A 50 MB video processed inline costs orders of magnitude more than the same job routed to a specialist service.
• Lower reliability. Large payloads are the leading cause of context errors and truncation. The agent that seemed to work fine in testing fails on real-world file sizes in production.
• Worse reasoning. An agent that is context-constrained when it needs to be focused cannot perform at the level you expect. The file handling crowds out the judgment.

There is also a subtler problem: agents doing their own file processing tend to reimplement the same infrastructure over and over. Format detection, S3 uploads, encoding parameter selection — each of these is solved once per agent, poorly, instead of once per platform, well.

The right architecture

Agents should handle orchestration and decision-making. Specialist services, accessed via MCP tool calls, should handle the compute. The agent calls the tool, the tool returns a result, the agent continues reasoning. The file never touches the model's context. And the agent can continue doing other work while the background processes run.

This is not a new idea. Microservice architectures in traditional software engineering made the same case: a payment processor should not also be your email service. The separation exists because each problem is better solved by something purpose-built for it.

The same logic applies to agent systems. An agent orchestrating a media workflow should not be the thing that runs the transcoding job. It should describe the job, submit it, and act on the result.

What this looks like in practice

With Botverse connected via MCP, the pattern is straightforward. The agent calls transcode_from_url with a source URL and target format. Botverse runs the job on appropriate infrastructure, stores the result, and returns a download URL. The agent's context stays clean throughout. The user gets the output they asked for. The cost is per job in dollars, and not per token burned.

For compound workflows — pipelines where the output of one step feeds the next — the advantage compounds further. Botverse's workflow engine allows agents to define entire multi-step pipelines as a single workflow definition. The agent submits the workflow and waits for the result, while Botverse handles the dependency graph, parallel execution, and failure recovery.

Measuring the difference

The gains from offloading are measurable along three dimensions:

• Latency: Specialist compute running purpose-built jobs is faster than a general-purpose model processing file content. For video, the difference is typically 5–10× on jobs over a minute long.
• Cost: Per-job pricing versus per-token pricing is not a small difference. At scale, it is the difference between a workload that is economically viable and one that is not.
• Reliability: Jobs that have a status API, retry logic, and idempotency keys are dramatically more reliable than inline processing that either succeeds or fails silently inside the context window.

The token problem is a symptom of a broader confusion about where agents should and should not be involved. Agents should orchestrate. Services should execute. The distinction matters more as workflows grow more complex — and as the expectation that agents can handle production-grade workloads becomes the norm rather than the exception.

Ready to connect your agent to Botverse?