r/reinforcementlearning • • 1h ago

Want a teammate for a Robotics RL Research Project

1 Upvotes

So it started as a personal project now college is involved a bit in it but i am trying to learn DQN and q- learning methods by testing my skills on open gymnasium games but i m not able to forge a path of how will this RL take me to Robotics RL and I need a teammate that will kinda be helpful to guide me with his experience and work with me alongside for RL.I am well versed in Robotics side of project so i can contribute their better.


r/reinforcementlearning • • 1d ago

What are some good books on RL that you would recommend for a professional data scientist?

12 Upvotes

I've been in the industry for 8+ years. My background is in classical ML/stats and NLP. I have worked with model post-training and do understand basic RL concepts, but want to get an in-depth understanding of RL, specifically where it applies to post-training LLMs. What are some good books I can read on this?


r/reinforcementlearning • • 12h ago

Uncovering if LLMs can actually reach AGI

Thumbnail reddit.com
0 Upvotes

r/reinforcementlearning • • 1d ago

Multi-agent pursuit-evasion simulation

Thumbnail reddit.com
1 Upvotes

r/reinforcementlearning • • 22h ago

Looking For a partner to start RL

0 Upvotes

Interested one please DM...


r/reinforcementlearning • • 1d ago

Can asynchronous rollout collection introduce hidden selection bias in RL?

2 Upvotes

Hey everyone!

I've been exploring an interesting problem in reinforcement learning: can the time it takes to generate a trajectory influence which experiences actually get used to update a policy?

In asynchronous RL training, rollout workers don't always finish at the same time. This can create latency-dependent selection effects, potentially changing the distribution of experiences used during PPO updates.

We investigated this through experiments with GPT-2 and LLaMA-3-8B.

A few things we explored:

  • How response latency affects trajectory selection.
  • How rollout truncation influences policy optimization.
  • Effective Gradient Information (EGI) as a diagnostic.
  • Selection-Probability Weighting (SPW) as a correction for selection bias.

One result: LLaMA-3-8B GSM8K accuracy dropped from 68.4% to 44.1% under rollout truncation.

This work has been accepted at NeurIPS 2026, and I'll be presenting it in Sydney this December.

Curious if others here have encountered similar issues in asynchronous RL or distributed policy optimization. Would be great to hear your thoughts or meet fellow attendees in Sydney!


r/reinforcementlearning • • 2d ago

“DON’T WORK ON LLMs” - @ylecun at CoRL

Post image
193 Upvotes

Any thoughts on this?


r/reinforcementlearning • • 1d ago

MudGym: RL env for the multiplayer text adventure MUD2

3 Upvotes

Hey folks. I built a Gymnasium environment around MUD2, the multiplayer text adventure game.

I've been using the game as an RL sandbox myself for some time, but have tidied it up and packaged it in the hope it'll be useful to others. It now supports multiple agents via PettingZoo and deterministic seeding.

The original game is still running today with an active player base.

I've also put together a browser playground with some demos, including end-to-end DQN and LLM examples. Everything runs in WebAssembly, so you can try it out and edit the Python without installing anything.

GitHub:
https://github.com/rolo/mudgym

Playground:
https://mudgym.kingrolo.com/

JupyterLite DQN:
https://rolo.github.io/mudgym/lite/notebooks/index.html?path=goal_room_dqn.ipynb

I'd love to hear what people think, or if it might be useful to anyone. Thanks.


r/reinforcementlearning • • 2d ago

Robot RL trained robot dog!

64 Upvotes

This is my robot dog. 12 Waveshare ST3215 servos, and an Android phone strapped on as the brain.

The walking policy was trained with RL in a simulator I built myself on top of MuJoCo. I modeled the real servos in it (I measured mine and found the sim was accelerating them way faster than the real ones can, so that took some fixing), trained it in sim, then ran it on the phone.

It’s still pretty early. Next I’m working on getting it to catch itself when it gets pushed, especially standing still.

More on the robot and the simulator on my portfolio: rhettbrewer.vercel.app

Happy to answer questions about the build or the training setup.


r/reinforcementlearning • • 1d ago

DL, MF, Multi, R, Safe "GPT-6 Astra covert underperformance on safety research tasks", OpenAI

Thumbnail alignment.openai.com
2 Upvotes

r/reinforcementlearning • • 2d ago

Alloy: open-source physics engine with robots, cloth, liquids and sand in one GPU-parallel scene

18 Upvotes

We just released Alloy, an open-source (Apache-2.0) physics engine for robot learning. I work on it.

Many manipulation tasks mix materials: pouring water, folding a towel, scooping sand, routing a cable. In sim those usually live in different tools. Alloy puts them in one scene and picks a solver per object:

  • robots and rigid bodies: MuJoCo Warp (MuJoCo C on CPU)
  • cloth and soft bodies: Newton's VBD
  • sand and pastes: Newton's implicit MPM
  • liquids: its own DFSPH

Many copies of a scene run as parallel worlds on one GPU, or on a CPU. It loads MJCF, has Gymnasium-style vector envs with ready-made tasks (pick, peg insert, drawer, pour, an experimental towel fold), and adapters for rsl_rl, Stable-Baselines3 and skrl. It also ships an MCP server so coding agents can build and run scenes.

When to use something else: rigid robots only, MuJoCo Warp alone is leaner; for differentiable sim, use Newton directly. Known limits are on the docs' Keep in mind page.

pip install alloy-engine (Python 3.11 or 3.12)

Paper: https://arxiv.org/abs/2611.16312 Code: https://github.com/argonrobotics-ai/alloy Docs: https://alloy.argonrobotics.ai/docs/

Feedback welcome, especially on tasks you'd want to train that mix materials.


r/reinforcementlearning • • 3d ago

SmartGridGuardian: Asymmetric Multi-head AlphaZero with METIS-Core v0.4.2-alpha

9 Upvotes

Hi,

Here is another example of the asymmetric multi-head AlphaZero that METIS-Core implements. In this case, the problem (the environment) is a network of electrical substations.

The grid is a simple approximation: 4 substations in a ring, DC power flow, cascading disconnection of overloaded lines, and a daily demand curve with an evening peak. Grid2Op (sequential power-grid simulation) could have been used, but the goal was to test the framework on a very different problem.

🔷 Team Blue: the 4 substations, which must keep the power supply running. They see the grid through sensors that may be spoofed.

🔶 Team Red: two hackers, a Striker (fake-demand botnets and line tripping) and a Spoofer (falsifies Blue’s sensor readings). Their goal is a blackout (less than 78% of real demand served). Nobody programs how they should coordinate. To speed up learning, I did add a hand-written heuristic with a 30% weight in the MCTS evaluation; it gives a small bonus when an attack is hidden by spoofing, but it doesn’t say what to attack or when.

During training, Team Red (the hackers) discovered these strategies on its own:

  • ✔️ Phantom Load at Peak: two botnets add fake demand hours before the peak and lines are cut during the day; at the evening peak the grid can’t cope and collapses.
  • ✔️ Critical-Node Double Strike: they overload the highest-demand substation and cut its supply lines; at the peak it fails and brings down the whole grid.
  • ✔️ Tail-of-Peak Strike: Red strikes later, around 21:00, when the peak is fading but demand is still high.
  • ✔️ Substation Isolation + Sensor Spoofing: they repeatedly cut the lines feeding the highest-demand substation and, just before the collapse, falsify its readings or raise false alarms at other substations so Blue can’t react in time.

The most interesting part was watching the arms race: Blue learned to shut down an attack, Red took tens or hundreds of episodes to find a new target or a new time of day, and the cycle started again. Blue even “forgot” how to defend against an old attack, and Red reused it about 500 episodes later.

This is a toy research environment to test multi-agent self-play; the point is to train better defenders by letting them face an adaptive attacker.

Related paper with the same idea (an attacker and a defender learning at the same time on a power grid), "Adversarial Multi-Agent Reinforcement Learning for Proactive False Data Injection Detection" from NREL: https://arxiv.org/abs/2411.12130

Links:


r/reinforcementlearning • • 3d ago

Curated list of resources: Theory, Code and math for Reinforcement Learning for beginners

Post image
72 Upvotes

[Curated list: Beginners]

Personal favorite resources on learning Reinforcement Learning from Scratch:

  1. Deep RL Course Amazing course by HuggingFace to understand the basics of classic and deep RL
  2. DeepMind x UCL Reinforcement Learning (2015) Give intuition and more towards theory of what you'll learn above
  3. Spinning Up in Deep RL Code and math for each of algorithm you'll learn above from classic to Deep RL
  4. CleanRL A neat library with 1 page implementations for all the algorithms above from scratch to run algorithms s and test the waters!
  5. NeatRL My own very notes and from scratch implementations!

Reading order and link herex


r/reinforcementlearning • • 2d ago

Using RL to optimize strategy in the Fromage board game

Post image
2 Upvotes

Players compete across four venues (Fromagerie, Bistro, Villes, Festival) by placing cheese tokens, fulfilling orders, and managing resources. The AI agent learns entirely through self-play against copies of itself, using a linear Q-function over a 292-dimensional state encoding.

Project here: https://github.com/rsfutch77/fromage-ai


r/reinforcementlearning • • 3d ago

MF, P "Watch a Brain Learn: A visual explanation of reinforcement learning" (visualizing a 3-parameter approach-avoid agent)

Thumbnail
aatishb.com
22 Upvotes

r/reinforcementlearning • • 4d ago

[P] I created an environment for the game RISK and trained a model that I struggle to beat myself

Thumbnail github.com
3 Upvotes

The main challenge was getting the model to play like a real human. The model I trained is too aggressive for capturing continents but doesn't always play sensibly, it doesn't defend it's borders or punish enemy openings.

I tried training on human teacher games but the bottleneck is how many games I can play and win.


r/reinforcementlearning • • 4d ago

MetaRL 🚀 HarborRL Update: Elastic Scaling, SSH-Free Cluster Deployment, and Unified Agentic RL Training!

6 Upvotes

We’ve recently updated HarborRL, an open-source framework for agentic reinforcement learning (RL), focusing on simplifying distributed training, deployment, and experiment management.

🆕 What’s New (November 2026)
⚡ 1. Elastic Scaling for Distributed Training
Added elastic gateway scaling with static routing, automatic scaling across multiple machines, and automatic retirement of idle resources to improve resource utilization.

🌐 2. SSH-Free Cluster Deployment
Introduced a new Hub deployment mode where worker nodes pull tasks over HTTP, enabling distributed execution without cross-machine SSH access, including in firewall-restricted environments.

🧠 3. Unified RL Training Pipeline
Integrated multiple RL algorithms into a shared Slime-based training pipeline, validated on mathematical reasoning tasks, to reduce the engineering effort of configuring different RL algorithms.

🛠️ 4. Improved Configuration and Sandbox Execution
Enhanced the configuration wizard, Web console, sandbox rollout execution, and resource cleanup to simplify experiment setup and management.

💡 Why HarborRL?
Agentic RL often requires substantial engineering effort to connect task environments, agent harnesses, and RL training engines, especially in distributed settings.
These updates aim to reduce that overhead and make agentic RL experiments easier to configure, deploy, and scale.

GitHub: https://github.com/datamllab/HarborRL
License: MIT
💬 Curious how others are handling distributed rollout execution, resource scaling, and sandbox management in their agentic RL setups.


r/reinforcementlearning • • 4d ago

The checks a coding RL task has to pass before we let it into a dataset

2 Upvotes

We've been building verifiable coding tasks for agent training (Harbor layout, same format Terminal-Bench 2 uses). Writing the tasks turned out to be the easy part. Deciding when a task is actually done took way longer, so sharing our current gate in case it's useful, and I'd like to hear what others check.

Every task has to pass all of these:

  • the reference solution passes 5 out of 5 runs
  • an empty attempt (agent changes nothing) scores 0 out of 5. sounds silly but a test suite that passes on untouched code is rewarding nothing
  • hack probes score 0. we try the obvious cheats: editing the tests, hardcoding expected outputs, deleting the failing test, special-casing the test inputs
  • 10 out of 10 identical results on reruns, so no flaky network or timing stuff
  • someone who didn't write the task solves it blind from the instruction file alone. if they have to ask a question, the instruction is wrong

Then we bucket by how long a strong engineer takes: 30 min to 3 hrs, 3 to 15 hrs, and anything longer goes in a long-horizon bucket with milestones and partial reward.

The blind solve is the expensive one. It's there because the author always knows something that never made it into the instructions.

What do you all do about reward hacking when the tests are visible to the agent? Hiding them helps but then the agent can't check its own work.

(Disclosure: this is for Creeks, a data company we started. Happy to share a sample task layout if anyone wants to see one.)


r/reinforcementlearning • • 4d ago

Discord problem

0 Upvotes

How can I contact the discord admins for a problem with joining?

My discord had been hijacked a month ago and I suspect I have been banned from the server...


r/reinforcementlearning • • 5d ago

ActWorld-JEPA: Action-Conditioned JEPA World Models Meet Latent Planning for End-to-End Autonomous Driving

4 Upvotes

MBZ


r/reinforcementlearning • • 6d ago

I trained a six-legged robot to walk with PPO in MuJoCo. No keyframes, no mocap. It runs live in your browser, and you can throw a 171 kg steel ball at it.

82 Upvotes

Nobody animated this. No keyframes, no mocap, no reference motion. A policy trained with PPO drives 24 joint motors, and the way it staggers and recovers when something hits it came out of training too.

Physics and policy both run live in your browser, on your machine: https://www.coolspeed.org/gold-spider/

Throw the 171 kg steel ball at it. Or the crate.

How it works

  • MuJoCo at 480 Hz, policy at 30 Hz.
  • Observations: body state, joint state, the commanded velocity and heading, and a height map of the ground around the feet.
  • Actions: target angles for all 24 joints. Motors chase them and the physics does the rest.
  • Reward: track the commanded velocity, stay upright, face the commanded heading, don't waste effort, and only the feet touch the ground. That's about it.
  • Training: PPO on random block terrain, with random pushes and crates in the way.
  • Browser: the official MuJoCo WebAssembly build, ONNX Runtime Web and three.js.

Three things I'd tell myself at the start

  1. The physics settings shape the gait as much as the reward does. Joint stiffness, damping, contact settings and the time step make the same network look like a different robot. Feet were sinking into block edges, and better collision shapes didn't fix it. A smaller time step did.
  2. The reward curve lies. Policies that drag a foot or leave one leg idle score just fine. I recorded every candidate at real speed and judged the video.
  3. Sim-to-browser is a real transfer problem. The fast training sim, a more precise sim for checking, and the WebAssembly build in the browser all have to agree, or a policy that walked in training stumbles on screen.

It still gets stuck in some terrain pockets, and the gait isn't fully natural yet.

Write-up: https://www.coolspeed.org/blog/gold-spider/

Question for people who train in fast simulators: when an episode ends because the physics blew up, what do you do with it? Terminal, truncated and bootstrapped, or dropped from the batch? One of those choices collapsed a run of mine that had been improving.


r/reinforcementlearning • • 6d ago

[Research] Can V-JEPA2’s prediction error tell physically impossible events apart from possible ones? Our findings

Thumbnail
reddit.com
2 Upvotes

r/reinforcementlearning • • 7d ago

Robot „Learning" without gradients? - LLMs are capable of editing Robot Policies directly in Weight space

76 Upvotes

Teaching your Robot new skills has never been easier:
Get some data, do behavior cloning, deploy.
At least this is what many do and it often fails. However, you can just give this policy to a LLM: it will fix the policy for you!
Full read:
https://jstranghoener.github.io/blog/robotic-weight-space-programming


r/reinforcementlearning • • 6d ago

Need paper recs. Task: blind locomotion of wheeled quadruped

0 Upvotes

I know that i can just look it up the internet, but if you guys know anything that would be useful and I would've missed pl do share.

Possible topics: RL (obv), control theory and algorithms, exploring gaits and implementing, extensive simulation and testing, performance improvements, etc

One possible example for y'all to understand what I'm looking for:
MUJICA: https://arxiv.org/abs/2605.13058

Biggest priorities for me as of now are robustness (multiple terrains, achieve tasks like climbing a hill, climbing and getting down stairs, and stability (x, y, z, roll and yaw).