r/IsaacSim • • 2h ago

Showcase Shepherding controller on a differential-drive robot in Isaac Sim 6.0: herding 6 cats into a pen (3x speed)

4 Upvotes

Disclosure: this is a robot my team is building, imported from our prototype CAD. Everything here is simulation, not real footage. Mostly posting because the shepherding part turned out to be a fun controls problem.

Setup:

  • Isaac Sim 6.0, headless, PhysX at 250 Hz, rendered at 25 fps. The video is sped up 3x and has music added.
  • Robot: a full articulation with a two-wheel differential-drive base and a rear ball caster (about 26 kg). The controller outputs (vx, wz), which go straight to the two wheel velocity drives, so there’s no lateral motion. It has to turn toward its path before it drives.
  • Controller: based on Strömbom et al. 2014, “Solving the shepherding problem”. It has two modes. COLLECT gets behind a straggler and pushes it back to the group. DRIVE gets behind the group and pushes it toward the gate through a staging point. The approach path bends around the flock so the robot doesn’t scatter it. Max speed is 1.2 m/s; the real prototype runs much slower today.
  • Cats: kinematic agents running a small flocking model. They wander, flee when the robot is close, keep apart from each other with weak cohesion, avoid walls, and settle at the food bowl once inside the pen. The meshes are 4 CC-BY models from Sketchfab, used for 6 cats. There’s no rig, so they slide, with a small bob added.
  • Scene: CC0 grass from Poly Haven, a picket fence built from one PointInstancer with box colliders, and the stock Isaac HDRI and vegetation.

r/IsaacSim • • 4d ago

Showcase 一個開源應用程式,運行於Quest/PICO耳機上,用於ROS2(Cyclone DDS)機器人遙控操作

Post image
1 Upvotes

r/IsaacSim • • 9d ago

Showcase NVIDIA and RLWRLD announce DexBench, a simulation benchmark for robot dexterity. Hosted runs available

Thumbnail dexbench.ai
10 Upvotes

r/IsaacSim • • 23d ago

Showcase Can AI 3D generators actually make sim-ready objects for Isaac Sim or MuJoCo? Compared 4 tools

Post image
11 Upvotes

I went down a rabbit hole on this after the NVIDIA and World Labs tutorial last year, where they brought Marble-generated rooms into Isaac Sim using splats plus a collider mesh. The rooms looked great, but it got me wondering about the objects a robot actually has to pick up or push around. Most comparisons I could find were about how nice the renders look, which doesn’t tell you much if you need a mug to sit on a table without falling through it.

For an object to be usable in sim you need collision geometry, mass, friction and the right scale, and a mesh from a generator usually gives you none of that. RoboCasa made the case that object variety helps policies generalize, so if generators could hand you all this directly it would save a lot of manual work. Four tools are trying, from pretty different starting points.

  1. Rigyd: Robotics-first, best for generating a lot of objects fast

You give it text, one or four images, or a GLB/FBX/OBJ you already have, and it gives back USD or MJCF with collision, mass and friction already set, so it loads straight into Isaac Sim or MuJoCo. There’s a Python SDK and CLI too if you want it in a pipeline. Articulated and soft objects aren’t supported yet, they’re on the roadmap.

  1. EmbodiedGen: Open-source, best if you want full control

This one’s from Horizon Robotics. It turns text or images into URDF with physical properties and real-world scale, and there’s a converter for USD and MJCF if you need those. V2 came out in July and goes further into whole task environments and multi-room scenes. You’d be running and maintaining it yourself though.

  1. Meshy: Visual quality first, physics set when you import

Meshy has an Isaac Sim bridge where you pick body type, collision, material and real-world height as you send the model over, and mass can be auto-calculated. The catch is that dynamic bodies only get convex or bounding-box collision, so a mug handle or the inside of a bowl basically gets filled in. It also needs a Pro plan, and the bridge is Windows-only right now.

  1. Tripo: Lots of export formats, no physics

Tripo exports USD, FBX, OBJ, STL, GLB and 3MF, but as far as I can tell the USD is just geometry and materials. So you’d be adding collision, mass and friction yourself, which makes it more of a mesh source you feed into something else.

Full disclosure, I work at Rigyd, so take what I’ve said about it with that in mind. I’ve tried my best to compare all four fairly and stick to what each one’s own docs say. But if anyone reading this feels I haven’t done justice to one of them, or got something wrong, please call it out. I’d rather go back, research it properly and put up a better post than leave a lopsided one here.

What I think actually decides whether a generated object is usable for training, roughly in order:

  1. Collision that matches where contact actually happens, like handles, rims and edges. If that’s wrong the grasp just doesn’t work, however good the textures are.
  2. Mass and friction that land inside whatever range you’re randomizing over anyway.
  3. Real-world scale, which seems to be the one that quietly breaks things since a lot of generated meshes come in at arbitrary sizes and nothing warns you.
  4. Visual quality, which mostly matters if your policy is camera-only.

How I’d pick between them:

  • If you need lots of sim-ready objects in USD or MJCF, Rigyd
  • If you want an open-source pipeline you can change, EmbodiedGen
  • If you only need a few good-looking objects and don’t mind setting physics on import, Meshy
  • If you just want raw meshes for your own pipeline, Tripo

Honestly for most setups I’d guess a mix works best, something sim-ready for the bulk of objects and then a visual generator plus some manual tuning for the few objects your task really depends on. They all end up as USD or MJCF anyway so they sit fine in the same scene.

So if you’ve trained on generated objects, how did you check the physics before trusting them? And is there a tool I’ve missed that should be on here?


r/IsaacSim • • 26d ago

Showcase How to Build a Robot With NVIDIA

Thumbnail
blackscarab.ai
8 Upvotes

NVIDIA has built a pretty impressive robotics ecosystem, but figuring out how Isaac Sim, Isaac Lab, GR00T, Cosmos, Omniverse, and everything else actually fit together can be confusing.

So we put together an illustrated walkthrough following a fictional startup building a warehouse humanoid.

We go through creating its virtual environment, teaching it through demonstrations, training and testing its policy, dealing with the Sim2Real gap, and eventually deploying the AI onto physical hardware.

We also wanted to explain what actually happens between an AI model deciding to move an arm and the motors physically executing that movement.

The goal was to connect the dots rather than explain each NVIDIA product in isolation.

Would love feedback from anyone who's actually worked with these tools, especially on where the real-world workflow differs from our example.


r/IsaacSim • • 29d ago

Help Newton vs PhysX in Isaac Lab: has anyone switched yet?

9 Upvotes

I was reading up on Newton for something I'm working on and realised nearly everything I could find was launch coverage or NVIDIA's own blog.

Barely anyone talking about actually using it. So I ended up going through the release notes myself.

If you haven't been following, the Newton physics engine hit 1.0 at GTC in March. It's the open-source engine NVIDIA built with DeepMind and Disney Research, and Isaac Lab 3.0 lets you swap it in as the physics backend instead of PhysX. MuJoCo Warp for rigid bodies, a separate solver for cables and cloth, and hydroelastic contact from Drake, which in theory should help a lot with insertion.

NVIDIA's number was 475x faster than MJX on manipulation. That was on their newest Blackwell workstation card though, so I'm not reading too much into it. The one independent benchmark I found (GPUSimBench, from July) didn't have a clear winner, it changed depending on the scenario.

What I actually found more interesting was the 1.1.0 changelog. A big chunk of it is import fixes. Mass and inertia, fixed joints, joint properties not parsing right between USD, URDF and MJCF.

Which kind of matters if the whole point is running one environment on different physics backends? If the inertia comes in slightly different on each one you've got a slightly different robot, and you probably won't notice until something doesn't transfer.

This is a problem we deal with at work. I'm at Rigyd, where we convert 3D models, Images and Text descriptions into simulation-ready USD and MJCF assets for Isaac Sim and MuJoCo, with the collision meshes, mass and inertia set up so the same asset behaves the same across engines.

So I'm biased toward thinking import is the bigger story than the speedup.

Skild is apparently using Newton for connector insertion and Samsung for cable handling with Lightwheel, but those are announced partnerships. I haven't seen anyone else write up switching over from PhysX.

So if you've tried it, did contact-heavy stuff actually transfer better or was it mostly faster? And how much did you have to retune assets that were already set up for PhysX?

Also if anyone's seen the same task run on PhysX and Newton with real robot results, please link it, would love to go through it


r/IsaacSim • • Oct 18 '26

Help Is an NVIDIA A10 gpu enough to run an ISAAC sim?

2 Upvotes

In summary: My goal is to simulate an environment where I can run and training a robot with RGB-D sensors, to autonomously navigate an environment.
I only have access to an A10 and want to know if it is enough to run NVIDIA ISAAC sim. It is fine if it runs slowly (within reason), I just want to know if it is possible.

Edit: Specifically, I am having trouble determining if the RT cores the A10 has is enough because I am seeing conflicting answers online.


r/IsaacSim • • Oct 07 '26

Help Collision Detection Series

2 Upvotes

Hi Someone in this group said that dealing with collision geometry was often still a manual part of your workflow.

I have a series on Collision Detection that might be useful:

https://www.youtube.com/watch?v=KZJXb5AF6NM&list=PLYuY7qeQZcWaofUHmLQy2q7xG9eiSxk1W&pp=sAgC

I also have a developing series in Physics Programming:

https://www.youtube.com/watch?v=5iuOYFrjfXs&list=PLYuY7qeQZcWZrK1RD-3-0Se4irqU5tBAA&pp=sAgC

Any feedback is welcome.


r/IsaacSim • • Oct 04 '26

Showcase 🚀 Free Tool To Build Custom OmniGraph / Action Graphs in Isaac Sim

3 Upvotes

🎥 YouTube Video:

https://youtu.be/MLXvw-wREh0

Creating custom OmniGraph / Action Graphs in NVIDIA Isaac Sim can be confusing and time-consuming. This video explains a simple tool that makes the process much easier.

🛠️ Try the free tool:

https://hrithik-verma.github.io/isaac-sim-omnigraph-node-builder/

The tool lets you visually build your custom OmniGraph and simplify the process of setting up nodes and connections.


r/IsaacSim • • Sep 30 '26

Showcase Introducing RobotGym : a public MCP for Robotics Simulation

22 Upvotes

r/IsaacSim • • Sep 26 '26

Help Issac Sim Learning Tutorials or Resources

5 Upvotes

I have a project in mind that I want to simulate first and make it work in simulation before I do any hardware stuff. The project will use AI concepts in order to make the robot learn stuff and perform it on its own. I have already learned ros2 and gazebo before and I have never experienced working with isaac sim or AI related stuff. So I would like you to recommend me some tutorials for learning how to work with isaac sim and like spawn a robot and move it, as well as some resources to start with the AI stuff on isaac sim and how to incorporate it with the robot. I do love learning the concept and theory together while applying things, so a tutorial that combines that would be perfect.


r/IsaacSim • • Sep 16 '26

Showcase Trained a robotic hand to walk in Isaac Lab, then deployed it on a Raspberry Pi

14 Upvotes

r/IsaacSim • • Sep 09 '26

Help I need help connecting Isaac Sim Camera and ROS2 Humble

2 Upvotes

I follow Nvidia's IsaacSim docs ROS2 Cameras

I run 2 containers: ROS2 Humble and Isaac Sim 6.0.1 on Linux machine.

As a result I do get topic listed, but i don't get any data from camera. Im terrible 😞

I don't get any warnings while playing the simulation in Isaac Sim.

Since a triple checked every checkbox and param and topic name, DomainId etc, I start to think there's some common mistake I make but can't see it.

Any suggestions, please, thank you!


r/IsaacSim • • Sep 09 '26

Help Isaac Lab vs pybullet for a 4 person undergrad team (15 weeks) worth the learning curve? +hardware implementation worth it?

Thumbnail reddit.com
1 Upvotes

r/IsaacSim • • Sep 08 '26

Help Looking for a sanity check on a high-end TRX50 workstation I am buying for a robotics lab

1 Upvotes

Looking for a sanity check on a high-end TRX50 workstation I am buying for a robotics lab.

The primary workload is large-scale GPU robotics simulation and RL training with Isaac Lab / mjlab. Sustained compute throughput, thermals, and reliability matter much more to us than gaming performance or benchmark scores.

Current vendor quote:

  • CPU: AMD Ryzen Threadripper 9980X — 64C/128T
  • Motherboard: ASUS Pro WS TRX50-SAGE WIFI
  • Cooling: 360mm liquid cooler with sTR5 mount
  • Memory: 64GB DDR5 ECC RDIMM — 4×16GB, all four channels populated
  • Storage: 4TB PCIe Gen5 NVMe
  • GPUs: 2× NVIDIA RTX PRO 6000 Blackwell Workstation Edition, 96GB each, full 600W versions (not Max-Q)
  • PSU: Super Flower Leadex Titanium 2200W, native 12V-2x6 to each GPU
  • Chassis: SilverStone RM51 5U rackmount with rails
  • Fans: Noctua
  • OS: Ubuntu 24.04 LTS
  • Warranty: 3 years, plus lifetime support
  • Price: $42,690 shipped

The GPUs are essentially fixed — I want the 2×96GB configuration. I'm mainly deciding how to balance the rest of the platform.

My biggest concern is the RAM. 64GB feels very low for a ~$43k workstation, especially with all four DIMM slots already occupied. One alternative would be to drop the CPU to a 9970X and put the savings toward 128GB or possibly 256GB of ECC RDIMM.

A few questions:

  1. For this kind of dual-GPU compute workload, would you take 9980X + 64GB or 9970X + 128/256GB?
  2. Any reason to avoid running two full-power 600W RTX PRO 6000s on TRX50?
  3. Are there any PCIe lane or topology concerns with dual x16 GPUs plus a Gen5 NVMe drive?
  4. Has anyone run two very high-power GPUs in the SilverStone RM51? How does it handle sustained airflow and thermals?
  5. The 2200W Super Flower PSU appears to require 200–240V input. Beyond having the appropriate circuit/receptacle, what else should I verify before deployment?
  6. The RM51 spec lists a 190mm PSU length limit, while this PSU appears to be 200mm long. Has anyone used this exact combination, or is that a hard incompatibility?
  7. Would you trust a 360mm AIO on a 9980X under sustained all-core loads, or would you choose a different cooling setup?
  8. Does anything else about this build look obviously unbalanced or questionable?

This system will likely spend long periods under heavy CPU + dual-GPU load, so stability, cooling, and sustained performance are the priorities.

I'd especially appreciate feedback from anyone running similar Threadripper + multi-GPU compute workstations.


r/IsaacSim • • Sep 02 '26

Help Did anyone run Issac sim on a low VRAM machine? I am trying to run it on a 4 GB VRAM machine. What are your thoughts?

4 Upvotes

I have a 3050 mobile with 4 GB VRAM. It's a laptop coupled with Ryzen 7 Hs CPU also, and I have Ubuntu 24.04. I'm not trying to download. I know the minimum requirement for the isaac sim is 16 GB VRAM. I definitely don't have that, so I'm not going to be using multiple robots or spawning multiple robots. I just need to spawn one robot, see if it works on my laptop, and understand what issac sim is.

I need to learn issac sim because I need that particular skill. Does anyone have any thoughts on how I get this thing to run on my machine, or is there any kind of online platform that can simulate the issac sim? I don't want alternative. I'm using mojoco right now for reinforcement learning. I understand how that works, but I'm curious to see how issac sim works. If you have any kind of suggestion or anything, can you please share it with me?

Thanks for reading my post. It means a lot to me.


r/IsaacSim • • Aug 31 '26

Showcase My Custom Reinforcement Learning Script for Isaac Sim

26 Upvotes

I have been working on different robotics task mainly in Pybullet. Complexity of Isaac Sim has kept me from experimenting with it but I finally finished my first successful reinforcement learning script using it.

Getting the settings right so I could observe the trainings in windowed mode with my laptop 3070 gpu took I while but I managed to get it working well enough to troubleshoot some early issues I noticed through visual inspection.

I gave the PPO full control of my robot's controller, not the joint angle outputs, meaning it had to figure out a way to climb the ramps using the directional controller inputs and body adjustments like pitch and height. The training took about 45 minutes with my RTX 3070 mobile GPU vs 1 hour and 45 minutes using cpu with Pybullet.

I am now working on full locomotion simulation, again with my custom robots, its a bit more involved than I expected but I am hoping in the end I can come up with my own full locomotion training script and load the model to my actual robot to control it.

I also share my tutorial scripts with my videos on youtube, if you are interested in watching the video for this one you can find it with the link below:

https://youtu.be/0x5BBosrq-E

You can also download the simulation script from my github repository with the link below.

https://github.com/serdarselimys/HexaDogZBD-IsaacSim-RL

If you are interested in the real 3D printed robot, you can fine info about it from the video link below.

https://youtu.be/qflyEQOJObM


r/IsaacSim • • Aug 21 '26

Help How much can realistic camera simulation reduce the pain of robotics development?

1 Upvotes

I've been thinking about how much of robotics development can realistically be done before the physical sensors arrive.

With depth cameras, there are a lot of things that are difficult to validate purely from specifications:

  • Camera placement
  • Field of view and blind spots
  • Depth range
  • Multi-camera coverage
  • Robot-camera transforms
  • ROS 2 integration
  • Navigation and perception behavior

We recently worked on an interesting example of this with DepthVista Helix, an iToF 3D camera that has been integrated into NVIDIA Isaac Sim.

The simulated camera reproduces the physical camera's FOV, depth modes, resolutions, calibration, transforms and ROS 2 outputs. There are also ready-to-run examples for robotic manipulation and a four-camera AMR navigation setup.

What I found interesting is that this lets you evaluate the camera configuration and perception pipeline before the physical hardware is available.

For example, the simulated camera has a high-resolution mode reaching 2 m and a long-range mode reaching 6 m, so you can evaluate which mode makes more sense depending on whether you're working on manipulation or navigation.

Here's the technical write-up if anyone is interested:

Robotics Development Just Got Faster & Smarter: e-con Systems’ iToF 3D Camera Comes to NVIDIA Isaac Sim

Curious how others handle this today.

Do you normally simulate the camera first, use a generic depth sensor in simulation and switch later, or wait until the physical camera is available?


r/IsaacSim • • Aug 18 '26

Showcase Designing & training a robot from start to finish for beginners

Thumbnail
youtu.be
38 Upvotes

For people looking to design and train their own robot using isaacsim and isaaclab:

I use fusion, Isaacsim and isaaclab in this video, which I have an installation guide for on my channel, it can be installed easily.

The video is aimed at beginners who want to just start training a robot quickly. I've had positive feedback from people so far so I thought I would share it here also.

Any issues (big or small) with any part of your project please feel free to message me here, on youtube or email.


r/IsaacSim • • Aug 18 '26

Showcase Got 6/7-DoF robot arms in Isaac Lab running reinforcement learning

Post image
25 Upvotes

We've been trying RL in Isaac Lab and adapted a SO-ARM100-based setup to two robot arms from AgileX Robotics: PiPER (6-DoF) and NERO (7-DoF). The current setup includes:

  • PiPER: end-effector reaching
  • NERO: reaching + cube manipulation
  • PPO-based training
  • 64 parallel simulation environments

The main goal was to get a quick RL workflow up and running for PiPER and NERO in simulation. Will be exploring sim-to-real deployment on the physical arms. Still a work in progress, but sharing it here in case it's useful for anyone. Feel free to drop your questions, glad to communicate.


r/IsaacSim • • Aug 09 '26

Help Any good resources for training an unusual robot to move in Isaac Lab?

34 Upvotes

Hi! I’ve recently started experimenting with reinforcement learning in Isaac Lab. My goal is to teach this robotic lamp different ways of moving, including balancing, locomotion and eventually jumping.

I already have a basic training setup running, but getting useful behaviour from such an unusual mechanism is proving harder than I expected. Most of the materials I’ve found are built around humanoids or quadrupeds, and I’m still fairly new to RL.

Are there any guides, papers, repositories or general approaches you’d recommend for training locomotion on a custom non-humanoid robot?

The video shows one of my very early jumping experiments.


r/IsaacSim • • Jul 30 '26

Help Setting Up Environment using Isaac Sim vs World model

1 Upvotes
  1. How to do it? I am using AMD and i am struggling for last few days
  2. Can World model entirely repalce Isaac sim world models?

r/IsaacSim • • Jul 24 '26

Help how to build accelerator kit for robotics using Isaac sim and isaac lab

0 Upvotes

Need to build accelerator kit for robotics using Isaac sim and isaac lab. Using this accelerator we can easily build robotic project that’s my aim. Can you please provide the information, is it possible to do it


r/IsaacSim • • Jul 22 '26

Help Best LLM for IsaacSim and IsaacLab?

6 Upvotes

I have personally used codex & claude to help me write isaacsim and also help in doing experimentation with isaaclab. However what I have seen is it misses a lot and I need to spend a lot of time debugging or just prompting.

I am curious if anyone has better workflows to use an LLM as a trusted assistant just to get a job done?