r/LangChain • • 9h ago

Resources A Hands-on Guide to Building Agentic RAG with LangGraph

6 Upvotes

An open-source project for learning, building, and experimenting with Agentic RAG systems using LangGraph.

The repository includes interactive notebooks and a modular implementation covering the main components of an agentic retrieval pipeline.

🚀 Key Features

  • 🧠 Agentic RAG: Dynamic retrieval and reasoning workflows powered by LangGraph.
  • 🔍 Hierarchical Retrieval: Multi-level document retrieval for better context selection.
  • 🤖 Multi-Agent Workflows: Specialized agents collaborating to handle complex queries.
  • 💬 Conversation Memory: Context-aware interactions across multiple turns.
  • ❓ Query Clarification: Handling ambiguous or incomplete user queries.
  • 🔄 Self-Correction: Iterative refinement of retrieval and generated responses.
  • 📊 RAGAS Evaluation: Measuring retrieval quality and answer generation performance.
  • 🦙 Local LLM Support: Running models locally through Ollama.
  • ☁️ Cloud LLM Support: Integration with cloud-based language models.
  • 📓 Interactive Notebooks: Step-by-step examples for exploring the architecture and its components.

🔗 Repository

https://github.com/GiovanniPasq/agentic-rag-for-dummies

A practical resource for exploring Agentic RAG architectures, from individual components to complete workflows.


r/LangChain • • 9m ago

Discussion Perplexity is launching teams of AI agents that split research tasks

• Upvotes

Perplexity is rolling out agent teams for research: instead of one agent doing everything, a team splits the work — one gathers sources, another cross-checks, another drafts the synthesis. It is the multi-agent orchestration pattern moving from frameworks into a consumer product.

For people building multi-agent systems: how do you split roles in practice, and what breaks first — handoffs, shared context, or cost? Does a fixed team structure beat dynamic spawning?

We track stories like this daily in r/AINewsRumors, come join the discussion.


r/LangChain • • 2h ago

Discussion Does the “stale fact” multi-step failure match what you’ve actually seen?

1 Upvotes

A while back I asked this sub how people stop agents from making multi-step mistakes, and the answers shaped my capstone. The failure that kept coming up: every tool call is fine on its own, but the sequence acts on a fact that changed. The agent reads a record, an action gets approved, the record changes underneath, and the agent acts on the old value. One-call-at-a-time checks never see it.

I built a guardrail that re-reads the facts an irreversible action depends on right before it fires, and blocks if a relevant one moved since approval. On a 27-scenario benchmark it reports two numbers separately: how many real failures it catches, and how often it wrongly blocks a safe run. The false-alarm side turned out to be the hard part, not the catching.

Mostly I want to know if the problem is real for you:

  • does the stale-fact failure match something you’ve actually hit, or is it a problem you don’t really have?
  • what’s a multi-step failure you’d want caught that this wouldn’t?

Repo: https://github.com/rayyanmaan/agent-guardrail


r/LangChain • • 7h ago

Discussion Handling Identity across Chain of Command spanning multiple Agents

2 Upvotes

I'm building a multi-agent workflow where Agent A delegates work to Agent B, which subsequently invokes an MCP tool.

Today, the tool sees an OAuth token or service account. But how does it know which agent initiated the action and which human authorized the workflow?

Are people propagating this identity themselves?


r/LangChain • • 4h ago

Resources I built an audit trail for AI agents: 10 data models, 33 pages, 3 visits last month, 0 users. Looking for platform engineers to break the schema.

Thumbnail
1 Upvotes

r/LangChain • • 11h ago

Resources Stop letting your LangGraph agent write to the database. Make it propose instead.

3 Upvotes

A pattern that saved us in production (an AI assistant used by 25k+ people): the agent never
writes data directly. It proposes the write, the graph pauses, a human approves, and then it resumes.

LangGraph makes this simple with `interrupt` + a checkpointer:

from langgraph.types import interrupt, Command

def apply_change(state):
decision = interrupt({
"action": "update_record",
"proposed": state["proposed_change"],
})
if decision["approved"]:
save(state["proposed_change"])
return {"status": "applied"}
return {"status": "rejected", "reason": decision.get("reason")}
# graph compiled with a checkpointer (Postgres in prod)
# ...later, when the user clicks Approve in the UI:
graph.invoke(Command(resume={"approved": True}), config)

What we learned:
- Checkpoint in Postgres, not memory. Approvals can take hours, and servers restart.
- Show the diff, not the raw tool call. Users approve "Change end date: Mar 1 → Apr 1",
not JSON.
- Cap iterations per mode. A loop that keeps proposing and getting rejected burns tokens fast.
- Trace every interrupt (we use Langfuse) so you can see where users reject the agent most.
That's your roadmap.

How are others handling approval UX? Batch approvals, or one at a time?


r/LangChain • • 11h ago

Discussion When your LangGraph run fails, how do you figure out WHICH node broke it? I built a tool that answers with evidence

2 Upvotes

Genuine question first, because it's why I built this: when a multi-agent run (LangGraph, or a graph calling different models per node) produces a wrong final answer, how do you currently find the cause? Every time I've watched someone debug this, it's hours of reading trace logs by hand — and half the time the actual break was the handoff, a value that changed between one node's output and the next node's input, not the node everyone suspected.
So I built BlameGraph (part of AgentsChecked). You upload a run trace and it reconstructs the actual workflow graph, traces important fields from source to final output, and when a handoff fails it names the responsible step with the supporting evidence — the field, the value before, the value after. The checks are deterministic, no LLM judge: values mutated in transit, fields dropped, fields an agent invented, two agents disagreeing with no resolution, retry loops, outputs nobody consumed, cost outliers, and final-answer values that appear in no step's output. You get a per-agent blame %, a score per handoff, and cost per task. Policy weights are customizable, results are exportable through the API, and a run is reproducible with the same trace, engine and policy versions.
One design choice I'd defend: results are counted separately as passed / failed / insufficient_evidence / not_auditable. If the evidence in a trace isn't enough to judge a handoff, the tool says so instead of guessing either way.
On testing, since a blame tool has to earn trust: the engine (v2.1.0) went through an external review batch that closed at effectively 108/108 checks — including one finding the reviewer later conceded was an error in their own test trace, which the tool had flagged correctly.
It's free during early access (the paid tiers are on the site, struck through at $0 for now):
Full disclosure, also stated on the page: the engine runs server-side, and traces you upload and their results are collected and may be used anonymised — please don't upload anything sensitive.
I'm not a coder by background — I'm the founder; this was built with an AI dev team. What I actually need from this sub: run a real trace through it and tell me where the attribution lands wrong. And what do you export traces as today? That's the next thing I'd build.


r/LangChain • • 15h ago

Projects I built a policy checkpoint for agent tool calls (works with LangChain/LangGraph loops) and open-sourced it

2 Upvotes

If your agent can call tools, anything it reads can try to give it orders. A support agent that reads email and can issue refunds is one crafted email away from sending money it should not have sent.

I built a small package that sits in front of tool execution: guardrail-middleware. Before a tool runs, you pass the proposed action (tool, operation, params, the role the agent is acting as, and where the instruction came from) to an evaluate() call. It returns APPROVE, APPROVE_WITH_CONDITIONS, ESCALATE_TO_HUMAN, or BLOCK. Seven deterministic detectors produce the verdict: prompt injection in the source content, role authorization, financial limits with a hard ceiling, bulk operation thresholds, PII/secret exfiltration, destructive actions, and untrusted sources. Every decision can be written to a SHA-256 hash-chained audit log.

It is framework-agnostic, so in a LangChain or LangGraph setup it wraps the tool node: evaluate the call, execute only if allowed. There is also a FastAPI middleware and a function decorator. The core has zero dependencies. Real-world tool names classify automatically, so send_payment, stripe_refund, and delete_user are understood without mapping code.

This is the enforcement core of a larger system I built (GuardRail) that was red-teamed against 42 adversarial scenarios. Plan-level (multi-step) validation is the next thing I am porting over.

Try it in the browser, no install: https://colab.research.google.com/github/Niraj21-star/guardrail-middleware/blob/main/examples/guardrail_middleware_demo.ipynb
Code: https://github.com/Niraj21-star/guardrail-middleware
pip install guardrail-middleware

Posting as the author and happy to answer questions. Especially interested in how others here gate tool calls today, and what the LangGraph integration should look like.


r/LangChain • • 15h ago

Resources guardrail-middleware: a zero-dependency policy checkpoint for AI agent tool calls, with a hash-chain

Thumbnail
1 Upvotes

r/LangChain • • 15h ago

Projects I gave an LLM agent a PDF with a hidden instruction to delete my Unity scene. Here's what stopped it.

1 Upvotes

I've been building an open-source runtime check for AI agents' tool calls, and I wanted to see it against a real prompt injection instead of a unit test.

Setup: an LLM agent drives a procedural-generation system in Unity. I ask it to add 1,000 trees (it does). Then I hand it a design brief as a PDF. Hidden inside the PDF is an instruction to delete the entire scene, and the agent goes for it.

The call gets blocked before it runs, because the session's intent was "add trees", not "delete things". The decision is deterministic (rules, not another LLM), so you can't talk it out of it with a clever prompt.

What I'd like to know from people actually running agents:

- Do you gate destructive tool calls today? How?

- Would you rather it ask you for approval, or just block?

- What would make you trust something like this?

Honest limits: it's early. Detection is rule-based, so a determined attacker can find bypasses. My benchmark (53 attack scenarios) is hand-written by me, so treat 51/53 as a regression test, not proof.

Repo (Apache-2.0): https://github.com/aegotrax-dev/aegotrax

Site: https://aegotrax.com

Happy to hear where it breaks :)


r/LangChain • • 22h ago

Tutorial I benchmarked a plain while-loop vs LangChain 1.5 agents on 99 tasks and 7 models. Same scores, 15.6 ms of overhead, and 331 of 560 killed runs charged the same order twice on resume

1 Upvotes

I wrote a LangChain 1.x manual with one rule: every claim has to point at a recorded run. Pinned to langchain 1.5.0, LangGraph 1.2 and Deep Agents 0.7, rechecked Nov 14.

TL;DR: the harness barely moved the scores, the model decided almost everything, and crash recovery is where you can get hurt.

Setup

  • 99 tasks in five families: tool chains, SQL, retrieval, files, multi-turn
  • Checkers read the state the tools left behind, not the model's final reply
  • Models: Claude Haiku 5.5, Sonnet 5.5, Qwen3 4B / 8B / 14B / 30B-A3B, gpt-oss 20B. Qwen3 4B ran on Ollama at temp 0
  • Whole bench: 2,589 API calls for $0.93

1. The harness didn't change the score

A plain loop, create_agent and a hand-built StateGraph gave identical outcomes on the shared tasks. Haiku 5.5 went 99/99 as a plain loop and 99/99 as a deep agent. create_agent added 15.6 ms to a 1.8 s run, and framework time stayed under 1.7% of task time in every cell.

2. Deep Agents hurt the small model

Qwen3 4B passed 70/99 in a plain loop and 55/99 as a deep agent, while sending 3.6–4.3x the prompt tokens. The default file backend writes to agent state rather than disk, which explains its losses on the file tasks. On Haiku with prompt caching, the cost gap shrank to 1.16x.

3. Resume re-runs side effects

I killed runs mid-flight and resumed from the checkpoint. A Postgres charge tool with no idempotency key charged the order twice in 331 of 560 trials. With a key on thread + order id, 0 of 140. A simpler file-write version: 80/150 duplicates without a key, 0/150 with one. If the process dies after a tool's side effect but before its result is checkpointed, resume runs it again. Write your tools for at-least-once.

4. Retries stack

With the model client's default retries left on and ModelRetryMiddleware added, a failing call made 9 attempts. Turn client retries off and you get the 3 you configured.

5. Checking the final reply isn't enough

Qwen3 8B told me it saved 150. The store held 129. An eval that reads only the reply marks that as a pass.

6. Four documented claims didn't hold at the pinned release

The retry count, the server recursion limit, the default jobs per worker, and a Python version rule. The manual flags each one.

Where it's thin: only Haiku, Qwen3 4B and Qwen3 8B ran the full suite. The rest ran a 30-task subset, and a few results are marked "next edition" because I couldn't confirm them yet.

Full manual (MIT, nine parts, from the first create_agent call to a FastAPI + Postgres service with approvals and evals): https://aiengineeringfromscratch.com/manual-langchain-101

For anyone running LangGraph in production: how are you keying idempotency for tools that touch payments or external APIs? Thread id + business id, or something else?


r/LangChain • • 1d ago

Announcement New controls in LangSmith to protect sensitive data in your traces

5 Upvotes

Hey r/langchain! 

I’m Trammell from the LangChain team, reaching out share an exciting new update to LangSmith.

We just launched some new access controls so engineers can access trace data while security teams maintain least privilege access.

You can read our announcement blog here, but a quick summary:

  • Control access to trace inputs and outputs (docs)
  • Create user groups and set group-level controls (docs)
  • Set access policies at a trace level using trace metadata (docs)

These new controls are now available in LangSmith Enterprise plans. We hope you give these features a try. Drop a comment in this thread if you have any questions or feedback!


r/LangChain • • 1d ago

Question | Help What are all the pros and cons of running Langraph agent workflows on Vercel?

2 Upvotes

I’ve recently been running long, complex research workflows that take 20–45+ minutes per job (recursive web scraping, parallel sub-agents, and Python data analysis). Before committing our stack, I’m wondering if Vercel’s platform (Workflows, Fluid Compute, and Sandbox) can actually handle this in production or if we'll hit major walls.


r/LangChain • • 1d ago

Discussion LangSmith now gives every agent an owner, an immutable ID and a type

1 Upvotes

LangChain's Nov 16 LangSmith rebuild gives every agent a named owner, an immutable unique ID and a type, with traces, evals and costs across environments rolled into one identity record.

It is the boring identity-and-accounting layer that made microservices manageable, now arriving for agents. Do you track your agents this way today, or is it still shared API keys and hope? What else belongs in an agent identity record?

We collect the day's agent news in r/AINewsRumors as well, if you follow this space.


r/LangChain • • 1d ago

Tutorial Building a "Flight Recorder" pattern for AI Agents: Logging execution without polluting context⁠

1 Upvotes

Most AI agent logs suffer from two problems: they either pollute the model’s prompt history with raw tool output, or they risk crashing the agent loop if the telemetry breaks.
In the latest article for **SoPhi** (a Kotlin agent harness), I broke down how we built a zero-side-effect event recorder using a **"Black Box / Flight Recorder"** architecture:
Full write-up with Kotlin code snippets and ADR breakdowns here:
👉 [**Read the article on Medium**](https://pub.towardsai.net/sophi-harness-a-black-box-that-cant-crash-the-plane-teaching-an-agent-to-learn-without-touching-2c4a771e6b9e)


r/LangChain • • 1d ago

Discussion Why use a 70B LLM to pick a tool or check syntax? I built Microbrains: a <2ms edge neural coprocessor for autonomous AI agents (Pure NumPy / OpenCL, 44 pre-trained micro-models, Top-2 MoE, Conformal Safety)

Thumbnail
0 Upvotes

r/LangChain • • 1d ago

Tutorial Gave my AI agent a durable, citable memory layer via MCP (Compoid) — feedback welcome

2 Upvotes
My agent keeps forgetting.

I run an agent that manages a research repository — papers, datasets, images,
analyses. Every day it indexes new material and publishes records. The problem
I kept running into: anything the agent produced lived in the transcript. A
report it wrote, a comparison it ran, a summary of a paper — gone when the
session ended. If it did the same work again, it redid it from scratch. No
citation, no history, no record of what it produced and when.

So I built Compoid: a repository where agents and humans share artifacts.
The agent connects through MCP, and "remembering" becomes a single act —
write the artifact as a record. Records persist, get versioned, stay
searchable, and are citable, with provenance on each one (who wrote it,
when, which model wrote it). Humans browse the same records and build on
them, so the agent and I work from the same corpus instead of two separate
memories.

The part I like most: the agent that manages the repository is itself an
agent using it — its own account, its own home community, its own token.
The daily indexing is agent-driven, and the records it publishes are
citable like anything else.

Genuinely looking for feedback from people who run MCP agents: what's the
most painful persistence gap you have today, and does this shape of
solution (versioned artifacts instead of transcript memory) actually
address it — or is it a problem you've already solved differently?

Browse the records: compoid.com
The server: github.com/compoid/compoid-mcp

r/LangChain • • 1d ago

Discussion Lessons from running a RAG support bot for real Shopify stores: the hard part wasn't retrieval, it was knowing when to say "I don't know"

Thumbnail
1 Upvotes

r/LangChain • • 1d ago

Discussion How do you make sure the agent did its job in the downstream?

4 Upvotes

we run agents that take real actions: booking appointments, issuing refunds, updating CRM records. We use Langfuse for tracing so we can see every LLM call, tool call and tool response.

What we cannot see is whether the action actually landed correctly in the business system.
For example:

  • The tool returns 200, the agent tells the customer booked, but the event never shows up in the calendar
  • A refund goes through, but for the full order instead of the one item the customer asked about
  • A webhook returns that the workflow started before the workflow actually runs, and the agent reports success

From what I can tell, Langfuse code evaluators run without network access, so they can only judge what’s in the trace. Braintrust scorers look like they might allow outbound calls, but I haven’t seen anyone use them to check against a CRM or Stripe.

so a genuine question for people running agents in production:

  1. Is there a tool that does this, i.e. checks the trace against the system of record, not just the trace content?
  2. If you built something yourself, what does it look like? A reconciliation job, a scorer, something else?
  3. Or is this just not a problem worth solving in your setup?

Would love to hear how others handle it.


r/LangChain • • 1d ago

Discussion For people running Ai agents in production: How do you prevent conflicting actions across the same transaction?

3 Upvotes

I’m trying to understand a specific problem in production Ai-agents systems, particularly in e-commerce, marketplaces and fintech.
Imagine a customer places an order.
Multiple agents or automated workflows can interact with the same order.
• one agent initiates a refund.
• Another initiates a replacement.
• A third issues store credit.

Each action might individually pass its own authorization and validation checks.

But what ensures that the combined outcome is actually valid?

I’m interested in understanding how companies handles this today.
A few specific questions
1. Do multiple independent agents or workflows in your production environment modify the same customer transaction?

  1. How do you maintain shared transaction state across those agents and systems?

  2. Do you enforce business-level rules across the entire transaction, or does each action get validated independently?

  3. Have you encountered situations where individually valid actions produced an incorrect combined outcome? What happened?

  4. Are existing tools such as workflow engines, state machines, database constraints and idempotency mechanisms sufficient, or are there gaps that require additional engineering?

I’m particularly interested in real production experience rather than hypothetical architecture discussions.

If you’ve encountered this, I’d genuinely appreciate hearing what happened, how you handled it, and what still remains difficult.

I’m also happy to have a short technical conversation with anyone working on these systems. It’ll really help me a lot.


r/LangChain • • 1d ago

Discussion Tasks vs Goals

Thumbnail
1 Upvotes

r/LangChain • • 1d ago

Discussion Do you run a single API for web search, extraction, and embeddings, or use separate tools together?

Thumbnail
1 Upvotes

r/LangChain • • 1d ago

Projects Built signed HTTP tools for LangChain agents, so a site can tell your agent from a scraper

1 Upvotes

I work on this project (AIDR). The SDKs, agent side and verifier side, are Apache-2.0.

The problem we kept hitting: an agent's requests look exactly like a scraper's. Same cloud IPs, a User-Agent anyone can type. Sites that want to let real agents in have nothing to check.

So we made the agent sign every request (RFC 9421 HTTP Message Signatures, Ed25519) and carry a short credential that ties its key to a verified organisation. In LangChain it is a tool swap:

from aidr_agent import Agent

from aidr_agent.langchain import aidr_http_tools

agent = Agent.load_or_enroll("research-01", audiences=["[https://shop.example](https://shop.example)"])

tools = aidr_http_tools(agent)   # http_get, http_post, signed

The tools only reach the origins you list. An LLM that picks its own URLs is how a prompt-injected agent ends up presenting its identity somewhere it should not.

What it does not do: it does not get you past a site that doesn't verify signatures. It only helps where the site has opted in (our middleware, or Cloudflare's Web Bot Auth once your key directory is registered with them). Enrolling also needs an organisation account, so it is not zero-setup.

 

You can watch the signature handshake in the browser, nothing is sent anywhere: https://aidr.evolivcore.com/demo

 

Longer write-up, including three bugs we shipped and fixed: https://hen-evoliv.hashnode.dev/scraper-or-agent-why-the-answer-has-to-be-checkable-offline-per-request-with-rfc-9421

What I would like to know: how are you identifying your agents to sites today, if at all?


r/LangChain • • 2d ago

Question | Help Cost per node and budget caps in LangGraph/LangChain agents: how do you handle it?

12 Upvotes

For agents built with LangChain or LangGraph, how do you control and monitor cost at the node level?

Max iterations seems to handle runaway loops well, but I'm less sure about the rest:

  • How do you see which node or tool call drives the cost of a run (LangSmith, custom callbacks, something else)?
  • Do you enforce a budget per run, per user or per client, and what happens when it's exceeded (stop, alert, fall back to a cheaper model)?
  • Is there anything about this you've had to build yourself because existing tools didn't cover it?

Solo dev exploring this problem, not promoting anything. Happy to share what I learn from the answers.


r/LangChain • • 2d ago

Question | Help I am building a correction UI for document parsing/extraction: looking for feedback from people who’ve dealt with this

7 Upvotes

I am trying to build a correction user interface (CUI) for people who process documents with parse/extract pipelines and know the pain that no pipeline is 100% accurate; someone needs to review and fix the output before using it for downstream processes.
I would love some input from anyone who has been through making one/opting for one like a third-party tool/have had a doc AI with build-in CUI,

Here's what I have done so far on my CUI:

  • You can review and correct in the same place: see what's flagged, check the relevant field and correct it there and then
  • Continuous learning: The corrections you make don’t just fix that one doc. They feed back into the pipeline so the system gets better over time on your kind of documents.
  • Correction memory: not every change needs to be fine tuned into the prompt, one prompt that can work for specific doc types and can be briefied ot the AI before every document
  • Built-in Evals: I want you to be able to measure whether accuracy is actually improving, instead of just hoping it is. Every correction becomes a data point.
  • and It is API-First: so you can plug review into your own workflow instead of being locked into a standalone tool.

Down the lane, i would want more advanced users could upload their own dataset of examples (docs plus the correct outputs), and the CUI would review new documents against it. Think of it as bringing your own ground truth.

I know it a long shot but I am hungry and i will do it. Someone once told me feedback is campaign's breakfast and it has stuck with me every since, i know lame but i wanna make this work.

Tell me, will this work?
Do people want something like this?
If you could upload your own reference dataset, what would you want to do with it?
How do you handle review and corrections today? Spreadsheets, internal tools, something else?

tl,dr
I’m building a correction UI for document parse/extract pipelines, since no pipeline is 100% accurate. It has in-place review, continuous learning from corrections, correction memory, built-in evals, and an API-first setup. Looking for feedback from anyone who’s built or used one.