r/LangChain • • 5h ago

Resources A Hands-on Guide to Building Agentic RAG with LangGraph

6 Upvotes

An open-source project for learning, building, and experimenting with Agentic RAG systems using LangGraph.

The repository includes interactive notebooks and a modular implementation covering the main components of an agentic retrieval pipeline.

πŸš€ Key Features

  • 🧠 Agentic RAG: Dynamic retrieval and reasoning workflows powered by LangGraph.
  • πŸ” Hierarchical Retrieval: Multi-level document retrieval for better context selection.
  • πŸ€– Multi-Agent Workflows: Specialized agents collaborating to handle complex queries.
  • πŸ’¬ Conversation Memory: Context-aware interactions across multiple turns.
  • ❓ Query Clarification: Handling ambiguous or incomplete user queries.
  • πŸ”„ Self-Correction: Iterative refinement of retrieval and generated responses.
  • πŸ“Š RAGAS Evaluation: Measuring retrieval quality and answer generation performance.
  • πŸ¦™ Local LLM Support: Running models locally through Ollama.
  • ☁️ Cloud LLM Support: Integration with cloud-based language models.
  • πŸ““ Interactive Notebooks: Step-by-step examples for exploring the architecture and its components.

πŸ”— Repository

https://github.com/GiovanniPasq/agentic-rag-for-dummies

A practical resource for exploring Agentic RAG architectures, from individual components to complete workflows.


r/LangChain • • 7h ago

Resources Stop letting your LangGraph agent write to the database. Make it propose instead.

4 Upvotes

A pattern that saved us in production (an AI assistant used by 25k+ people): the agent never
writes data directly. It proposes the write, the graph pauses, a human approves, and then it resumes.

LangGraph makes this simple with `interrupt` + a checkpointer:

from langgraph.types import interrupt, Command

def apply_change(state):
decision = interrupt({
"action": "update_record",
"proposed": state["proposed_change"],
})
if decision["approved"]:
save(state["proposed_change"])
return {"status": "applied"}
return {"status": "rejected", "reason": decision.get("reason")}
# graph compiled with a checkpointer (Postgres in prod)
# ...later, when the user clicks Approve in the UI:
graph.invoke(Command(resume={"approved": True}), config)

What we learned:
- Checkpoint in Postgres, not memory. Approvals can take hours, and servers restart.
- Show the diff, not the raw tool call. Users approve "Change end date: Mar 1 β†’ Apr 1",
not JSON.
- Cap iterations per mode. A loop that keeps proposing and getting rejected burns tokens fast.
- Trace every interrupt (we use Langfuse) so you can see where users reject the agent most.
That's your roadmap.

How are others handling approval UX? Batch approvals, or one at a time?


r/LangChain • • 4h ago

Discussion Handling Identity across Chain of Command spanning multiple Agents

2 Upvotes

I'm building a multi-agent workflow where Agent A delegates work to Agent B, which subsequently invokes an MCP tool.

Today, the tool sees an OAuth token or service account. But how does it know which agent initiated the action and which human authorized the workflow?

Are people propagating this identity themselves?


r/LangChain • • 7h ago

Discussion When your LangGraph run fails, how do you figure out WHICH node broke it? I built a tool that answers with evidence

2 Upvotes

Genuine question first, because it's why I built this: when a multi-agent run (LangGraph, or a graph calling different models per node) produces a wrong final answer, how do you currently find the cause? Every time I've watched someone debug this, it's hours of reading trace logs by hand β€” and half the time the actual break was the handoff, a value that changed between one node's output and the next node's input, not the node everyone suspected.
So I built BlameGraph (part of AgentsChecked). You upload a run trace and it reconstructs the actual workflow graph, traces important fields from source to final output, and when a handoff fails it names the responsible step with the supporting evidence β€” the field, the value before, the value after. The checks are deterministic, no LLM judge: values mutated in transit, fields dropped, fields an agent invented, two agents disagreeing with no resolution, retry loops, outputs nobody consumed, cost outliers, and final-answer values that appear in no step's output. You get a per-agent blame %, a score per handoff, and cost per task. Policy weights are customizable, results are exportable through the API, and a run is reproducible with the same trace, engine and policy versions.
One design choice I'd defend: results are counted separately as passed / failed / insufficient_evidence / not_auditable. If the evidence in a trace isn't enough to judge a handoff, the tool says so instead of guessing either way.
On testing, since a blame tool has to earn trust: the engine (v2.1.0) went through an external review batch that closed at effectively 108/108 checks β€” including one finding the reviewer later conceded was an error in their own test trace, which the tool had flagged correctly.
It's free during early access (the paid tiers are on the site, struck through at $0 for now):
Full disclosure, also stated on the page: the engine runs server-side, and traces you upload and their results are collected and may be used anonymised β€” please don't upload anything sensitive.
I'm not a coder by background β€” I'm the founder; this was built with an AI dev team. What I actually need from this sub: run a real trace through it and tell me where the attribution lands wrong. And what do you export traces as today? That's the next thing I'd build.


r/LangChain • • 12h ago

Projects I built a policy checkpoint for agent tool calls (works with LangChain/LangGraph loops) and open-sourced it

2 Upvotes

If your agent can call tools, anything it reads can try to give it orders. A support agent that reads email and can issue refunds is one crafted email away from sending money it should not have sent.

I built a small package that sits in front of tool execution: guardrail-middleware. Before a tool runs, you pass the proposed action (tool, operation, params, the role the agent is acting as, and where the instruction came from) to an evaluate() call. It returns APPROVE, APPROVE_WITH_CONDITIONS, ESCALATE_TO_HUMAN, or BLOCK. Seven deterministic detectors produce the verdict: prompt injection in the source content, role authorization, financial limits with a hard ceiling, bulk operation thresholds, PII/secret exfiltration, destructive actions, and untrusted sources. Every decision can be written to a SHA-256 hash-chained audit log.

It is framework-agnostic, so in a LangChain or LangGraph setup it wraps the tool node: evaluate the call, execute only if allowed. There is also a FastAPI middleware and a function decorator. The core has zero dependencies. Real-world tool names classify automatically, so send_payment, stripe_refund, and delete_user are understood without mapping code.

This is the enforcement core of a larger system I built (GuardRail) that was red-teamed against 42 adversarial scenarios. Plan-level (multi-step) validation is the next thing I am porting over.

Try it in the browser, no install: https://colab.research.google.com/github/Niraj21-star/guardrail-middleware/blob/main/examples/guardrail_middleware_demo.ipynb
Code: https://github.com/Niraj21-star/guardrail-middleware
pip install guardrail-middleware

Posting as the author and happy to answer questions. Especially interested in how others here gate tool calls today, and what the LangGraph integration should look like.


r/LangChain • • 12h ago

Resources guardrail-middleware: a zero-dependency policy checkpoint for AI agent tool calls, with a hash-chain

Thumbnail
1 Upvotes

r/LangChain • • 12h ago

Projects I gave an LLM agent a PDF with a hidden instruction to delete my Unity scene. Here's what stopped it.

1 Upvotes

I've been building an open-source runtime check for AI agents' tool calls, and I wanted to see it against a real prompt injection instead of a unit test.

Setup: an LLM agent drives a procedural-generation system in Unity. I ask it to add 1,000 trees (it does). Then I hand it a design brief as a PDF. Hidden inside the PDF is an instruction to delete the entire scene, and the agent goes for it.

The call gets blocked before it runs, because the session's intent was "add trees", not "delete things". The decision is deterministic (rules, not another LLM), so you can't talk it out of it with a clever prompt.

What I'd like to know from people actually running agents:

- Do you gate destructive tool calls today? How?

- Would you rather it ask you for approval, or just block?

- What would make you trust something like this?

Honest limits: it's early. Detection is rule-based, so a determined attacker can find bypasses. My benchmark (53 attack scenarios) is hand-written by me, so treat 51/53 as a regression test, not proof.

Repo (Apache-2.0): https://github.com/aegotrax-dev/aegotrax

Site: https://aegotrax.com

Happy to hear where it breaks :)


r/LangChain • • 18h ago

Tutorial I benchmarked a plain while-loop vs LangChain 1.5 agents on 99 tasks and 7 models. Same scores, 15.6 ms of overhead, and 331 of 560 killed runs charged the same order twice on resume

1 Upvotes

I wrote a LangChain 1.x manual with one rule: every claim has to point at a recorded run. Pinned to langchain 1.5.0, LangGraph 1.2 and Deep Agents 0.7, rechecked Nov 14.

TL;DR: the harness barely moved the scores, the model decided almost everything, and crash recovery is where you can get hurt.

Setup

  • 99 tasks in five families: tool chains, SQL, retrieval, files, multi-turn
  • Checkers read the state the tools left behind, not the model's final reply
  • Models: Claude Haiku 5.5, Sonnet 5.5, Qwen3 4B / 8B / 14B / 30B-A3B, gpt-oss 20B. Qwen3 4B ran on Ollama at temp 0
  • Whole bench: 2,589 API calls for $0.93

1. The harness didn't change the score

A plain loop, create_agent and a hand-built StateGraph gave identical outcomes on the shared tasks. Haiku 5.5 went 99/99 as a plain loop and 99/99 as a deep agent. create_agent added 15.6 ms to a 1.8 s run, and framework time stayed under 1.7% of task time in every cell.

2. Deep Agents hurt the small model

Qwen3 4B passed 70/99 in a plain loop and 55/99 as a deep agent, while sending 3.6–4.3x the prompt tokens. The default file backend writes to agent state rather than disk, which explains its losses on the file tasks. On Haiku with prompt caching, the cost gap shrank to 1.16x.

3. Resume re-runs side effects

I killed runs mid-flight and resumed from the checkpoint. A Postgres charge tool with no idempotency key charged the order twice in 331 of 560 trials. With a key on thread + order id, 0 of 140. A simpler file-write version: 80/150 duplicates without a key, 0/150 with one. If the process dies after a tool's side effect but before its result is checkpointed, resume runs it again. Write your tools for at-least-once.

4. Retries stack

With the model client's default retries left on and ModelRetryMiddleware added, a failing call made 9 attempts. Turn client retries off and you get the 3 you configured.

5. Checking the final reply isn't enough

Qwen3 8B told me it saved 150. The store held 129. An eval that reads only the reply marks that as a pass.

6. Four documented claims didn't hold at the pinned release

The retry count, the server recursion limit, the default jobs per worker, and a Python version rule. The manual flags each one.

Where it's thin: only Haiku, Qwen3 4B and Qwen3 8B ran the full suite. The rest ran a 30-task subset, and a few results are marked "next edition" because I couldn't confirm them yet.

Full manual (MIT, nine parts, from the first create_agent call to a FastAPI + Postgres service with approvals and evals): https://aiengineeringfromscratch.com/manual-langchain-101

For anyone running LangGraph in production: how are you keying idempotency for tools that touch payments or external APIs? Thread id + business id, or something else?


r/LangChain • • 21h ago

Discussion LangSmith now gives every agent an owner, an immutable ID and a type

1 Upvotes

LangChain's Nov 16 LangSmith rebuild gives every agent a named owner, an immutable unique ID and a type, with traces, evals and costs across environments rolled into one identity record.

It is the boring identity-and-accounting layer that made microservices manageable, now arriving for agents. Do you track your agents this way today, or is it still shared API keys and hope? What else belongs in an agent identity record?

We collect the day's agent news in r/AINewsRumors as well, if you follow this space.


r/LangChain • • 23h ago

Tutorial Building a "Flight Recorder" pattern for AI Agents: Logging execution without polluting context⁠

1 Upvotes

Most AI agent logs suffer from two problems: they either pollute the model’s prompt history with raw tool output, or they risk crashing the agent loop if the telemetry breaks.
In the latest article for **SoPhi** (a Kotlin agent harness), I broke down how we built a zero-side-effect event recorder using a **"Black Box / Flight Recorder"** architecture:
Full write-up with Kotlin code snippets and ADR breakdowns here:
πŸ‘‰ [**Read the article on Medium**](https://pub.towardsai.net/sophi-harness-a-black-box-that-cant-crash-the-plane-teaching-an-agent-to-learn-without-touching-2c4a771e6b9e)