r/LangChain • • 4h ago

Discussion A meta-analysis of 516 multi-agent LLM configurations: once the single agent gets the same samples or compute, multi-agent systems average 4.2 points worse and tie or lose in 74% of comparisons

Thumbnail
reddit.com
2 Upvotes

r/LangChain • • 8h ago

Discussion Perplexity is launching teams of AI agents that split research tasks

4 Upvotes

Perplexity is rolling out agent teams for research: instead of one agent doing everything, a team splits the work — one gathers sources, another cross-checks, another drafts the synthesis. It is the multi-agent orchestration pattern moving from frameworks into a consumer product.

For people building multi-agent systems: how do you split roles in practice, and what breaks first — handoffs, shared context, or cost? Does a fixed team structure beat dynamic spawning?

We track stories like this daily in r/AINewsRumors, come join the discussion.


r/LangChain • • 5h ago

Question | Help 🙏 Looking for Suggestion - From the Expert/OGs of the fields.

2 Upvotes

Fellow Redditors,

How is everyone using multiple coding agents on the same project without stale context or conflicting changes?

I use the Codex app, OpenCode CLI and Antigravity IDE. Would appreciate a better setup if exists or is preferred.

Ideally, I would like to assign five separate issues and have all agents work on them in parallel without overwriting each other’s changes or making incompatible decisions.

Two things I’m struggling with:

  • Keeping context current: How do you automatically update AGENTS.md or other project notes after meaningful changes? Are you using hooks, memory tools or another approach?
  • Coordinating parallel work: Do you use separate Git worktrees, a coordinator agent, task ownership or something else? How to work with multiple agents at the same time

I’m especially interested in setups that work across different tools and models.

Would love to hear your actual workflow, including what went wrong and what you simplified.


r/LangChain • • 2h ago

Discussion I'm building FlowX: an open-source compiler for self-evolving AI agent workflows. Looking for collaborators.

Thumbnail reddit.com
1 Upvotes

r/LangChain • • 17h ago

Resources A Hands-on Guide to Building Agentic RAG with LangGraph

14 Upvotes

An open-source project for learning, building, and experimenting with Agentic RAG systems using LangGraph.

The repository includes interactive notebooks and a modular implementation covering the main components of an agentic retrieval pipeline.

🚀 Key Features

  • 🧠 Agentic RAG: Dynamic retrieval and reasoning workflows powered by LangGraph.
  • 🔍 Hierarchical Retrieval: Multi-level document retrieval for better context selection.
  • 🤖 Multi-Agent Workflows: Specialized agents collaborating to handle complex queries.
  • 💬 Conversation Memory: Context-aware interactions across multiple turns.
  • ❓ Query Clarification: Handling ambiguous or incomplete user queries.
  • 🔄 Self-Correction: Iterative refinement of retrieval and generated responses.
  • 📊 RAGAS Evaluation: Measuring retrieval quality and answer generation performance.
  • 🦙 Local LLM Support: Running models locally through Ollama.
  • ☁️ Cloud LLM Support: Integration with cloud-based language models.
  • 📓 Interactive Notebooks: Step-by-step examples for exploring the architecture and its components.

🔗 Repository

https://github.com/GiovanniPasq/agentic-rag-for-dummies

A practical resource for exploring Agentic RAG architectures, from individual components to complete workflows.


r/LangChain • • 4h ago

Discussion How do you debug a multi-step AI agent workflow when only the final result is wrong?

Thumbnail reddit.com
1 Upvotes

r/LangChain • • 11h ago

Discussion Does the “stale fact” multi-step failure match what you’ve actually seen?

1 Upvotes

A while back I asked this sub how people stop agents from making multi-step mistakes, and the answers shaped my capstone. The failure that kept coming up: every tool call is fine on its own, but the sequence acts on a fact that changed. The agent reads a record, an action gets approved, the record changes underneath, and the agent acts on the old value. One-call-at-a-time checks never see it.

I built a guardrail that re-reads the facts an irreversible action depends on right before it fires, and blocks if a relevant one moved since approval. On a 27-scenario benchmark it reports two numbers separately: how many real failures it catches, and how often it wrongly blocks a safe run. The false-alarm side turned out to be the hard part, not the catching.

Mostly I want to know if the problem is real for you:

  • does the stale-fact failure match something you’ve actually hit, or is it a problem you don’t really have?
  • what’s a multi-step failure you’d want caught that this wouldn’t?

Repo: https://github.com/rayyanmaan/agent-guardrail


r/LangChain • • 19h ago

Resources Stop letting your LangGraph agent write to the database. Make it propose instead.

5 Upvotes

A pattern that saved us in production (an AI assistant used by 25k+ people): the agent never
writes data directly. It proposes the write, the graph pauses, a human approves, and then it resumes.

LangGraph makes this simple with `interrupt` + a checkpointer:

from langgraph.types import interrupt, Command

def apply_change(state):
decision = interrupt({
"action": "update_record",
"proposed": state["proposed_change"],
})
if decision["approved"]:
save(state["proposed_change"])
return {"status": "applied"}
return {"status": "rejected", "reason": decision.get("reason")}
# graph compiled with a checkpointer (Postgres in prod)
# ...later, when the user clicks Approve in the UI:
graph.invoke(Command(resume={"approved": True}), config)

What we learned:
- Checkpoint in Postgres, not memory. Approvals can take hours, and servers restart.
- Show the diff, not the raw tool call. Users approve "Change end date: Mar 1 → Apr 1",
not JSON.
- Cap iterations per mode. A loop that keeps proposing and getting rejected burns tokens fast.
- Trace every interrupt (we use Langfuse) so you can see where users reject the agent most.
That's your roadmap.

How are others handling approval UX? Batch approvals, or one at a time?


r/LangChain • • 13h ago

Resources I built an audit trail for AI agents: 10 data models, 33 pages, 3 visits last month, 0 users. Looking for platform engineers to break the schema.

Thumbnail reddit.com
1 Upvotes

r/LangChain • • 20h ago

Discussion When your LangGraph run fails, how do you figure out WHICH node broke it? I built a tool that answers with evidence

2 Upvotes

Genuine question first, because it's why I built this: when a multi-agent run (LangGraph, or a graph calling different models per node) produces a wrong final answer, how do you currently find the cause? Every time I've watched someone debug this, it's hours of reading trace logs by hand — and half the time the actual break was the handoff, a value that changed between one node's output and the next node's input, not the node everyone suspected.
So I built BlameGraph (part of AgentsChecked). You upload a run trace and it reconstructs the actual workflow graph, traces important fields from source to final output, and when a handoff fails it names the responsible step with the supporting evidence — the field, the value before, the value after. The checks are deterministic, no LLM judge: values mutated in transit, fields dropped, fields an agent invented, two agents disagreeing with no resolution, retry loops, outputs nobody consumed, cost outliers, and final-answer values that appear in no step's output. You get a per-agent blame %, a score per handoff, and cost per task. Policy weights are customizable, results are exportable through the API, and a run is reproducible with the same trace, engine and policy versions.
One design choice I'd defend: results are counted separately as passed / failed / insufficient_evidence / not_auditable. If the evidence in a trace isn't enough to judge a handoff, the tool says so instead of guessing either way.
On testing, since a blame tool has to earn trust: the engine (v2.1.0) went through an external review batch that closed at effectively 108/108 checks — including one finding the reviewer later conceded was an error in their own test trace, which the tool had flagged correctly.
It's free during early access (the paid tiers are on the site, struck through at $0 for now):
Full disclosure, also stated on the page: the engine runs server-side, and traces you upload and their results are collected and may be used anonymised — please don't upload anything sensitive.
I'm not a coder by background — I'm the founder; this was built with an AI dev team. What I actually need from this sub: run a real trace through it and tell me where the attribution lands wrong. And what do you export traces as today? That's the next thing I'd build.


r/LangChain • • 1d ago

Projects I built a policy checkpoint for agent tool calls (works with LangChain/LangGraph loops) and open-sourced it

2 Upvotes

If your agent can call tools, anything it reads can try to give it orders. A support agent that reads email and can issue refunds is one crafted email away from sending money it should not have sent.

I built a small package that sits in front of tool execution: guardrail-middleware. Before a tool runs, you pass the proposed action (tool, operation, params, the role the agent is acting as, and where the instruction came from) to an evaluate() call. It returns APPROVE, APPROVE_WITH_CONDITIONS, ESCALATE_TO_HUMAN, or BLOCK. Seven deterministic detectors produce the verdict: prompt injection in the source content, role authorization, financial limits with a hard ceiling, bulk operation thresholds, PII/secret exfiltration, destructive actions, and untrusted sources. Every decision can be written to a SHA-256 hash-chained audit log.

It is framework-agnostic, so in a LangChain or LangGraph setup it wraps the tool node: evaluate the call, execute only if allowed. There is also a FastAPI middleware and a function decorator. The core has zero dependencies. Real-world tool names classify automatically, so send_payment, stripe_refund, and delete_user are understood without mapping code.

This is the enforcement core of a larger system I built (GuardRail) that was red-teamed against 42 adversarial scenarios. Plan-level (multi-step) validation is the next thing I am porting over.

Try it in the browser, no install: https://colab.research.google.com/github/Niraj21-star/guardrail-middleware/blob/main/examples/guardrail_middleware_demo.ipynb
Code: https://github.com/Niraj21-star/guardrail-middleware
pip install guardrail-middleware

Posting as the author and happy to answer questions. Especially interested in how others here gate tool calls today, and what the LangGraph integration should look like.


r/LangChain • • 1d ago

Resources guardrail-middleware: a zero-dependency policy checkpoint for AI agent tool calls, with a hash-chain

Thumbnail
reddit.com
1 Upvotes

r/LangChain • • 1d ago

Projects I gave an LLM agent a PDF with a hidden instruction to delete my Unity scene. Here's what stopped it.

1 Upvotes

I've been building an open-source runtime check for AI agents' tool calls, and I wanted to see it against a real prompt injection instead of a unit test.

Setup: an LLM agent drives a procedural-generation system in Unity. I ask it to add 1,000 trees (it does). Then I hand it a design brief as a PDF. Hidden inside the PDF is an instruction to delete the entire scene, and the agent goes for it.

The call gets blocked before it runs, because the session's intent was "add trees", not "delete things". The decision is deterministic (rules, not another LLM), so you can't talk it out of it with a clever prompt.

What I'd like to know from people actually running agents:

- Do you gate destructive tool calls today? How?

- Would you rather it ask you for approval, or just block?

- What would make you trust something like this?

Honest limits: it's early. Detection is rule-based, so a determined attacker can find bypasses. My benchmark (53 attack scenarios) is hand-written by me, so treat 51/53 as a regression test, not proof.

Repo (Apache-2.0): https://github.com/aegotrax-dev/aegotrax

Site: https://aegotrax.com

Happy to hear where it breaks :)


r/LangChain • • 1d ago

Tutorial I benchmarked a plain while-loop vs LangChain 1.5 agents on 99 tasks and 7 models. Same scores, 15.6 ms of overhead, and 331 of 560 killed runs charged the same order twice on resume

1 Upvotes

I wrote a LangChain 1.x manual with one rule: every claim has to point at a recorded run. Pinned to langchain 1.5.0, LangGraph 1.2 and Deep Agents 0.7, rechecked Nov 14.

TL;DR: the harness barely moved the scores, the model decided almost everything, and crash recovery is where you can get hurt.

Setup

  • 99 tasks in five families: tool chains, SQL, retrieval, files, multi-turn
  • Checkers read the state the tools left behind, not the model's final reply
  • Models: Claude Haiku 5.5, Sonnet 5.5, Qwen3 4B / 8B / 14B / 30B-A3B, gpt-oss 20B. Qwen3 4B ran on Ollama at temp 0
  • Whole bench: 2,589 API calls for $0.93

1. The harness didn't change the score

A plain loop, create_agent and a hand-built StateGraph gave identical outcomes on the shared tasks. Haiku 5.5 went 99/99 as a plain loop and 99/99 as a deep agent. create_agent added 15.6 ms to a 1.8 s run, and framework time stayed under 1.7% of task time in every cell.

2. Deep Agents hurt the small model

Qwen3 4B passed 70/99 in a plain loop and 55/99 as a deep agent, while sending 3.6–4.3x the prompt tokens. The default file backend writes to agent state rather than disk, which explains its losses on the file tasks. On Haiku with prompt caching, the cost gap shrank to 1.16x.

3. Resume re-runs side effects

I killed runs mid-flight and resumed from the checkpoint. A Postgres charge tool with no idempotency key charged the order twice in 331 of 560 trials. With a key on thread + order id, 0 of 140. A simpler file-write version: 80/150 duplicates without a key, 0/150 with one. If the process dies after a tool's side effect but before its result is checkpointed, resume runs it again. Write your tools for at-least-once.

4. Retries stack

With the model client's default retries left on and ModelRetryMiddleware added, a failing call made 9 attempts. Turn client retries off and you get the 3 you configured.

5. Checking the final reply isn't enough

Qwen3 8B told me it saved 150. The store held 129. An eval that reads only the reply marks that as a pass.

6. Four documented claims didn't hold at the pinned release

The retry count, the server recursion limit, the default jobs per worker, and a Python version rule. The manual flags each one.

Where it's thin: only Haiku, Qwen3 4B and Qwen3 8B ran the full suite. The rest ran a 30-task subset, and a few results are marked "next edition" because I couldn't confirm them yet.

Full manual (MIT, nine parts, from the first create_agent call to a FastAPI + Postgres service with approvals and evals): https://aiengineeringfromscratch.com/manual-langchain-101

For anyone running LangGraph in production: how are you keying idempotency for tools that touch payments or external APIs? Thread id + business id, or something else?


r/LangChain • • 1d ago

Announcement New controls in LangSmith to protect sensitive data in your traces

5 Upvotes

Hey r/langchain! 

I’m Trammell from the LangChain team, reaching out share an exciting new update to LangSmith.

We just launched some new access controls so engineers can access trace data while security teams maintain least privilege access.

You can read our announcement blog here, but a quick summary:

  • Control access to trace inputs and outputs (docs)
  • Create user groups and set group-level controls (docs)
  • Set access policies at a trace level using trace metadata (docs)

These new controls are now available in LangSmith Enterprise plans. We hope you give these features a try. Drop a comment in this thread if you have any questions or feedback!


r/LangChain • • 1d ago

Question | Help What are all the pros and cons of running Langraph agent workflows on Vercel?

2 Upvotes

I’ve recently been running long, complex research workflows that take 20–45+ minutes per job (recursive web scraping, parallel sub-agents, and Python data analysis). Before committing our stack, I’m wondering if Vercel’s platform (Workflows, Fluid Compute, and Sandbox) can actually handle this in production or if we'll hit major walls.


r/LangChain • • 1d ago

Discussion LangSmith now gives every agent an owner, an immutable ID and a type

1 Upvotes

LangChain's Nov 16 LangSmith rebuild gives every agent a named owner, an immutable unique ID and a type, with traces, evals and costs across environments rolled into one identity record.

It is the boring identity-and-accounting layer that made microservices manageable, now arriving for agents. Do you track your agents this way today, or is it still shared API keys and hope? What else belongs in an agent identity record?

We collect the day's agent news in r/AINewsRumors as well, if you follow this space.


r/LangChain • • 1d ago

Discussion Why use a 70B LLM to pick a tool or check syntax? I built Microbrains: a <2ms edge neural coprocessor for autonomous AI agents (Pure NumPy / OpenCL, 44 pre-trained micro-models, Top-2 MoE, Conformal Safety)

Thumbnail
reddit.com
0 Upvotes

r/LangChain • • 1d ago

Tutorial Gave my AI agent a durable, citable memory layer via MCP (Compoid) — feedback welcome

2 Upvotes
My agent keeps forgetting.

I run an agent that manages a research repository — papers, datasets, images,
analyses. Every day it indexes new material and publishes records. The problem
I kept running into: anything the agent produced lived in the transcript. A
report it wrote, a comparison it ran, a summary of a paper — gone when the
session ended. If it did the same work again, it redid it from scratch. No
citation, no history, no record of what it produced and when.

So I built Compoid: a repository where agents and humans share artifacts.
The agent connects through MCP, and "remembering" becomes a single act —
write the artifact as a record. Records persist, get versioned, stay
searchable, and are citable, with provenance on each one (who wrote it,
when, which model wrote it). Humans browse the same records and build on
them, so the agent and I work from the same corpus instead of two separate
memories.

The part I like most: the agent that manages the repository is itself an
agent using it — its own account, its own home community, its own token.
The daily indexing is agent-driven, and the records it publishes are
citable like anything else.

Genuinely looking for feedback from people who run MCP agents: what's the
most painful persistence gap you have today, and does this shape of
solution (versioned artifacts instead of transcript memory) actually
address it — or is it a problem you've already solved differently?

Browse the records: compoid.com
The server: github.com/compoid/compoid-mcp

r/LangChain • • 1d ago

Discussion Lessons from running a RAG support bot for real Shopify stores: the hard part wasn't retrieval, it was knowing when to say "I don't know"

Thumbnail reddit.com
1 Upvotes

r/LangChain • • 2d ago

Discussion How do you make sure the agent did its job in the downstream?

4 Upvotes

we run agents that take real actions: booking appointments, issuing refunds, updating CRM records. We use Langfuse for tracing so we can see every LLM call, tool call and tool response.

What we cannot see is whether the action actually landed correctly in the business system.
For example:

  • The tool returns 200, the agent tells the customer booked, but the event never shows up in the calendar
  • A refund goes through, but for the full order instead of the one item the customer asked about
  • A webhook returns that the workflow started before the workflow actually runs, and the agent reports success

From what I can tell, Langfuse code evaluators run without network access, so they can only judge what’s in the trace. Braintrust scorers look like they might allow outbound calls, but I haven’t seen anyone use them to check against a CRM or Stripe.

so a genuine question for people running agents in production:

  1. Is there a tool that does this, i.e. checks the trace against the system of record, not just the trace content?
  2. If you built something yourself, what does it look like? A reconciliation job, a scorer, something else?
  3. Or is this just not a problem worth solving in your setup?

Would love to hear how others handle it.


r/LangChain • • 2d ago

Discussion For people running Ai agents in production: How do you prevent conflicting actions across the same transaction?

4 Upvotes

I’m trying to understand a specific problem in production Ai-agents systems, particularly in e-commerce, marketplaces and fintech.
Imagine a customer places an order.
Multiple agents or automated workflows can interact with the same order.
• one agent initiates a refund.
• Another initiates a replacement.
• A third issues store credit.

Each action might individually pass its own authorization and validation checks.

But what ensures that the combined outcome is actually valid?

I’m interested in understanding how companies handles this today.
A few specific questions

  1. Do multiple independent agents or workflows in your production environment modify the same customer transaction?
  2. How do you maintain shared transaction state across those agents and systems?
  3. Do you enforce business-level rules across the entire transaction, or does each action get validated independently?
  4. Have you encountered situations where individually valid actions produced an incorrect combined outcome? What happened?
  5. Are existing tools such as workflow engines, state machines, database constraints and idempotency mechanisms sufficient, or are there gaps that require additional engineering?

I’m particularly interested in real production experience rather than hypothetical architecture discussions.

If you’ve encountered this, I’d genuinely appreciate hearing what happened, how you handled it, and what still remains difficult.

I’m also happy to have a short technical conversation with anyone working on these systems. It’ll really help me a lot.


r/LangChain • • 1d ago

Discussion Tasks vs Goals

Thumbnail reddit.com
1 Upvotes

r/LangChain • • 1d ago

Discussion Do you run a single API for web search, extraction, and embeddings, or use separate tools together?

Thumbnail reddit.com
1 Upvotes

r/LangChain • • 2d ago

Projects Built signed HTTP tools for LangChain agents, so a site can tell your agent from a scraper

1 Upvotes

I work on this project (AIDR). The SDKs, agent side and verifier side, are Apache-2.0.

The problem we kept hitting: an agent's requests look exactly like a scraper's. Same cloud IPs, a User-Agent anyone can type. Sites that want to let real agents in have nothing to check.

So we made the agent sign every request (RFC 9421 HTTP Message Signatures, Ed25519) and carry a short credential that ties its key to a verified organisation. In LangChain it is a tool swap:

from aidr_agent import Agent

from aidr_agent.langchain import aidr_http_tools

agent = Agent.load_or_enroll("research-01", audiences=["[https://shop.example](https://shop.example)"])

tools = aidr_http_tools(agent)   # http_get, http_post, signed

The tools only reach the origins you list. An LLM that picks its own URLs is how a prompt-injected agent ends up presenting its identity somewhere it should not.

What it does not do: it does not get you past a site that doesn't verify signatures. It only helps where the site has opted in (our middleware, or Cloudflare's Web Bot Auth once your key directory is registered with them). Enrolling also needs an organisation account, so it is not zero-setup.

 

You can watch the signature handshake in the browser, nothing is sent anywhere: https://aidr.evolivcore.com/demo

 

Longer write-up, including three bugs we shipped and fixed: https://hen-evoliv.hashnode.dev/scraper-or-agent-why-the-answer-has-to-be-checkable-offline-per-request-with-rfc-9421

What I would like to know: how are you identifying your agents to sites today, if at all?