Adding agents to software you already run

Every few weeks someone asks me the same question in a slightly different shape. We have a product. It has users, an API, a database and an auth layer that took two years to get right. Now leadership wants "agents" in it. What do we build on?

The question usually arrives with a twist. The team doesn't want to be tied to one model vendor's managed agent platform. They want to self-host, and they want the option to run open models, usually through OpenRouter, so they can swap a model per task without a rewrite.

That rules out the managed harnesses straight away. Those are built to run their own vendor's models on their own infrastructure. If open models are on your list, you're picking an open-source framework and running it yourself.

So here's the map I now draw for people. It's opinionated, and it starts from a premise most comparison posts skip: you are not building an agent product. You are adding agents to a product that already exists. That changes which trade-offs matter.

The one-page version: view the infographic

Why "existing platform" changes the answer

When you build a standalone agent demo, the framework is the whole world. Visual builders, quick multi-agent setups and batteries-included platforms look great, because there is nothing else to fit around.

Inside an existing product, the framework is a guest. It has to live with:

• Your auth and permission model, not its own
• Your database, for state that has to survive a restart
• Your API gateway, rate limits and audit logs
• Your deploy pipeline, code review and observability stack
• Your latency budget, because users are waiting on the other end

So the questions that decide it are boring and specific. Can the agent pause for a human and resume tomorrow? Can I trust the shape of what it returns? Does it run in the language my platform already runs in? How heavy is it to operate?

GitHub stars don't answer any of those.

The question that decides it

Here's the short version I give people before we get into any detail:

If your agent must...                          Start with
Pause for approval, then resume later          LangGraph
Serve fast, lightweight agent endpoints        Agno
Return outputs your backend can trust          Pydantic AI
Live inside a Node / TypeScript platform       Mastra
Coordinate specialist agents on one task       CrewAI
Be prototyped visually, or edited by non-devs  Langflow or Dify

The first four are the ones I'd actually embed in a production platform. The last three earn their place too, but in different roles.

The four I'd embed

LangGraph: when the agent needs memory and a brake pedal

LangGraph models an agent as a graph of steps with explicit state. Its real selling point for platform teams is the checkpointer. State is saved at every step, into a store like Postgres, so a run can stop, wait for a human to approve something, and carry on hours later from exactly where it left off.

That matters the moment an agent touches anything consequential. "Approve this refund." "Confirm this vendor change." "Review this contract clause before it goes out." Loops, retries and routing back to an earlier step are all first-class too.

The cost is a steeper learning curve and more code than the lighter options. You'll write more structure up front. For workflows with real stakes, that structure is the point.

Pick it when: your agent runs multi-step business workflows with approvals, compliance or long-running state.

Agno: when speed and footprint matter most

Agno (formerly Phidata) goes the other way. It's lightweight Python with very little abstraction between you and the model call. Agents start fast and use little memory, which makes it a good fit for serverless runtimes and high-traffic endpoints.

It also maps neatly onto an existing microservice estate. Turning your current APIs into agent tools takes little ceremony.

The trade-off: long, branching workflows with durable state mean more of your own code. Agno gives you a fast engine, not a workflow system.

Pick it when: you're adding many small, fast agent capabilities behind existing APIs.

Pydantic AI: when your backend can't accept surprises

Open models are getting very good, but the smaller ones still slip on structured output. A missing field, a wrong type, a JSON blob wrapped in chatty prose. In a demo that's mildly annoying. In a backend that writes to your database, it's an incident.

Pydantic AI treats the schema as the contract. Tool inputs and model outputs are validated against Pydantic models, and the framework retries when the model gets it wrong. It feels like writing normal Python business logic rather than learning a new agent vocabulary, and it ships with OpenTelemetry support, so traces flow into whatever observability stack you already run.

It's lighter on multi-agent orchestration than LangGraph or CrewAI. For most "add an intelligent step to this service" use cases, that's fine.

Pick it when: you run a Python backend (FastAPI, Django, Flask) and correctness of outputs matters more than orchestration.

Mastra: when your platform is TypeScript

A lot of the advice in this space quietly assumes Python. If your product is Node.js, Next.js or a TypeScript service estate, bolting on a Python sidecar just to run agents adds a language, a runtime, a deployment target and a team skill gap.

Mastra is TypeScript-native, with workflows, agents, state and streaming built for the Node event loop. It deploys wherever your Node code already goes.

It's younger than the Python frameworks, and its ecosystem of integrations is smaller. That gap is closing, but check that the connectors you need exist before you commit.

Pick it when: your whole platform is TypeScript and you'd rather not run Python just for agents.

The three with different jobs

CrewAI: great for multi-agent experiments

CrewAI makes "a researcher agent hands off to a writer agent, who hands off to a reviewer" easy to express. For exploring multi-agent patterns, it's quick and readable.

In production, role-based crews tend to be less deterministic and harder to debug than an explicit graph. When something goes wrong at step four of a five-agent hand-off, you want to see exactly which state led there. I'd prototype with CrewAI and consider moving the proven flow to something more explicit.

Langflow: a visual canvas with a REST endpoint attached

Langflow lets you wire models, tools and agents together on a canvas, and every flow can be served as an API endpoint. It's excellent for prototyping and for getting non-developers involved early.

The catch for a platform team: the logic lives in a canvas, not in your codebase. That makes code review, diffing and testing harder. Use it to find the right flow, then decide whether that flow belongs in code.

Dify: a full platform in a box, with licence fine print

Dify bundles a visual builder, RAG, prompt management and an admin UI. For an internal tool, that's a lot of value out of one Docker Compose file.

Two cautions if you're embedding it in a product. It's heavy to run compared with a library. And its licence isn't plain Apache 2.0. The Dify Open Source License adds conditions, including that you can't use the source to run a multi-tenant SaaS similar to Dify's own service without written permission, and you can't remove the branding from its console. If your product would expose Dify to customers, have someone read the licence before you build on it.

Wiring open models through OpenRouter

The good news is that the model layer is the easy part. OpenRouter exposes an OpenAI-compatible API, so most of these frameworks connect to it through their OpenAI client with two settings:

OPENAI_API_KEY="sk-or-..."
OPENAI_BASE_URL="https://openrouter.ai/api/v1"
model="<provider>/<model>"

Some frameworks name the variables differently or route through LiteLLM, but the idea is the same. Swap the model string, keep the agent code.

That flexibility is the reason to do this at all. Use a strong model for planning, a cheap fast one for classification, and change either when something better ships, without touching the workflow.

Three checks before you ship

Test tool calling per model, not per framework. Function-calling reliability varies a lot between open models, and between sizes of the same model. Run your actual tool schemas against every model you plan to route to. Validate outputs and retry on failure, whichever framework you use.

Keep agent state in your own database. Durable state, conversation history and checkpoints belong in your Postgres, under your backups and access controls, not in a framework vendor's hosted cloud. This is also what lets you reconstruct what an agent did when someone asks.

Put agents behind your API gateway. An agent is just another service. Reuse your existing auth, rate limiting and audit logging instead of letting agents get their own side door into your systems.

The short version

If you remember one thing: choose by your stack and how much state the agent must hold.

Durable approvals and long-running workflows point to LangGraph. Fast, lightweight endpoints point to Agno. Strict output contracts point to Pydantic AI. A TypeScript platform points to Mastra. Keep CrewAI, Langflow and Dify for exploration, internal tools, or roles where their trade-offs don't bite.

And whichever you pick, the hard part won't be the framework. It'll be the same things it always is in a real platform: identity, state, observability and knowing exactly what ran on your behalf.

Sources: framework documentation for LangGraph, Agno, Pydantic AI, Mastra, CrewAI, Langflow and Dify, the Dify Open Source License, and OpenRouter's API docs. Related: When the regulator asks what your agent did, your prompt isn't the answer.