rmagine_gazebo_plugins

Hardware-accelerated range sensors for Gazebo

Code • rmagine • Wiki

Range sensor plugins for Gazebo, built on the ray tracing sensor simulation library rmagine. With rmagine’s OptiX or Vulkan backends you can simulate depth sensor data directly on a GPU (OptiX requires an RTX card and CUDA; Vulkan runs on any Vulkan-ray-tracing-capable GPU); with the Embree backend you can simulate any provided sensor on the CPU. All backends build an acceleration structure over the scene once and refit it in place as objects move, so simulating dense depth sensors stays fast even in large Gazebo worlds.

Conceptually, two kinds of plugins work together, one pair per backend (Embree/CPU, OptiX/GPU, Vulkan/GPU):

Map plugin syncs rmagine’s scenes with Gazebo’s world, as geometry moves, appears, or disappears.
Sensor plugins use the synced acceleration structure to do raycasting with rmagine’s sensor models: Spherical, Pinhole, O1Dn and OnDn

Currently supported rmagine backends are

embree: Intel Embree-based. Runs smoothly even on low-end hardware
optix: Ultra-fast hardware-accelerated lidar simulation. Requires your system to be compatible with Nvidia CUDA and OptiX
vulkan: Hardware-accelerated lidar simulation on any Vulkan-ray-tracing-capable GPU (not limited to Nvidia/CUDA). No noise model support yet (only OptiX has that currently).

See Architecture for the technical details.

Installation

[!IMPORTANT] Tested with

ROS 2 Gazebo

Jazzy Harmonic (gz-sim8)

Humble Fortress (ignition-gazebo6)

For older versions checkout the branch noetic or humble-gazebo-classic

ROS 2	Gazebo
Jazzy	Harmonic (`gz-sim8`)
Humble	Fortress (`ignition-gazebo6`)

Clone both packages into your ROS 2 workspace’s src folder and build:

user@pc:~/ros_ws/src$ git clone git@github.com:uos/rmagine.git
user@pc:~/ros_ws/src$ git clone git@github.com:uos/rmagine_gazebo_plugins.git
user@pc:~/ros_ws/src$ cd ..
user@pc:~/ros_ws$ colcon build --packages-select rmagine rmagine_gazebo_plugins

rmagine needs a few common system libraries (TBB, Boost, Eigen, Assimp, CMake); for the OptiX backend, CUDA; for the Vulkan backend, a Vulkan loader/SDK (libvulkan-dev, glslang-tools). If colcon build complains about a missing dependency, see rmagine’s own installation instructions.

Built targets depend on which rmagine components were found:

rmagine::embree found: rmagine_embree_map_system, rmagine_embree_sensor_system
rmagine::optix found: rmagine_optix_map_system, rmagine_optix_sensor_system
rmagine::vulkan found: rmagine_vulkan_map_system, rmagine_vulkan_sensor_system

Quickstart

launch/robot_demo.launch.py spawns a small differential-drive robot (urdf/example_robot.urdf.xacro) carrying one rmagine Embree spherical lidar into worlds/gz_embree_robot_demo.sdf (a few static boxes/a cylinder to drive around and scan), and bridges everything to ROS via config/ros_gz_bridge_robot_demo.yaml:

ros2 launch rmagine_gazebo_plugins robot_demo.launch.py rmagine:=embree

Will launch a Gazebo server and GUI with embree backend enabled (you can switch it to optix or vulkan). Additionally it launches a preconfigured RViz with the standard gpu_lidar (Ogre2) colored in white and the rmagine version colored by object id:

image info

You can drive it with any keyboard teleop of choice. Switch the fixed frame to see how the scans look like with or without localization.

Integration

Once you’ve run the quickstart, these are the building blocks for wiring rmagine sensors into your own world/robot.

Map system (one per world, per backend)

```xml my_robot my_robot::lidar_link 200 0.001 0.001 false ... ``` `ignore_model`/`ignore_link` exclude a model (or a single link of one, `model::link`) from the raytracing scene entirely; useful for excluding a sensor's own housing, or a robot the sensor is mounted on. `update/rate_limit` caps how often the scene is re-synced (Hz); `delta_trans`/`delta_rot` are the minimum pose change (meters/radians) before a moved entity's transform is pushed into the scene.

Sensor system (one plugin per world, one <sensor> per actual sensor)

```xml ... default lidar_link /model/my_robot/scan /model/my_robot/points 20 spherical -3.14159 0.01745 360 -0.2618 0.008727 60 0.2 100.0 </link> ``` `type="custom"` is required (sdformat validates the standard `type` attribute against its own known sensor type names); `gz:type` is the free-form identifier this package's factory plugin looks for (`rmagine_embree`, `rmagine_optix`, or `rmagine_vulkan`), gz-sim's own documented convention for third-party sensor types, the same one used by its shipped `environmental_sensor.sdf` example. ``/`` are plain gz-transport topic names (this plugin does no automatic `/model//...` namespacing the way gz-sim's built-in sensors do): write the full path you want if you're bridging into a namespaced robot, or a bare name like `scan` if not. `LaserScan` is only published for a single-ring (`` omitted) Spherical model; a multi-ring 3D scan (or any other model type) publishes `PointCloudPacked` only. Extra output topics can be added with repeated `...scan|points` elements. </details>

Bridging to ROS

The published topics use standard `gz.msgs` types, so bridge them to ROS with [`ros_gz_bridge`](https://github.com/gazebosim/ros_gz) like any other Gazebo sensor (see `config/ros_gz_bridge_robot_demo.yaml` for a complete example).

Migrating from gpu_lidar

Switching an existing robot from gz-sim's built-in `gpu_lidar` sensor to rmagine is a same-shape edit to the `` block, not a rewrite. The only thing that never needs to change is your `ros_gz_bridge` config: keep `` set to `/points`, matching `gpu_lidar`'s own convention of auto-publishing `PointCloudPacked` on `/points`, and the same bridge entries that worked for `gpu_lidar` keep working unchanged. **Before** (`gpu_lidar`): ```xml scan 10 lidar_link 360 -3.14159 3.14159 16 -0.261799 0.261799 0.2 30.0 ``` **After** (rmagine): ```xml scan scan/points 10 lidar_link -3.14159 0.0175019 360 -0.261799 0.0349065 16 0.2 30.0 ``` Everything else (``, ``, ``, Pinhole/O1Dn/OnDn-specific tags, ``) is a rmagine-only addition with no `gpu_lidar` equivalent, and all of it is optional. </details>

Non-spherical sensor models

Set `` to `pinhole`, `o1dn`, or `ondn` (default `spherical`). Every model type follows the same shape as Spherical's `` above: a type-named wrapper containing a `` (how that model scans) and a `/</range>`. **Pinhole** (depth camera-style): ```xml pinhole 640 480 1.0472 0.2 100.0 ``` **O1Dn** (one shared ray origin, arbitrary ray directions), reads its rays from a `` YAML file: ```yaml width: 8 height: 4 rays: orig: [0, 0, 0] dirs: - [1, 0, 0] - [0.99, 0.01, 0] # ... width * height entries, row-major ``` ```xml o1dn /path/to/rays.yaml 0.2 100.0 ``` ...or inline, for small hand-authored ray sets that don't warrant a separate file (same `orig`/`dirs` shape as the YAML, just as nested SDF elements, one `` per ray): ```xml 2 1 0 0 0 1 0 0 0.99 0.01 0 0.2 100.0 ``` **OnDn** (arbitrary ray origins and directions), same ``-or-inline choice, with a per-ray `origs` list instead of a single shared `orig`: ```yaml width: 8 height: 4 rays: origs: - [0, 0, 0] - [0, 0, 0] dirs: - [1, 0, 0] - [0.99, 0.01, 0] # ... width * height entries, row-major ``` ```xml ondn /path/to/rays.yaml 0.2 100.0 ``` ```xml 2 1 0 0 0 0 0 0 1 0 0 0.99 0.01 0 0.2 100.0 ``` </details>

Noise (OptiX/GPU sensor system only -- not yet available on Vulkan)

Repeatable `` elements, applied in order to the simulated ranges (in VRAM, before download): ```xml gaussian 0.0 0.01 rel_gaussian 0.002 1.0 uniform_dust 0.0000001 0.5 ``` </details> ## Architecture Two plugin roles per backend, mirroring the classic split between a scene-sync world plugin and a raycasting sensor plugin (gz-sim only has one plugin base type, `System`, so both are `System` plugins, but the responsibilities stay separate): - **Map system** (`rmagine_embree_map_system` / `rmagine_optix_map_system` / `rmagine_vulkan_map_system`, attached to ``): builds and incrementally maintains one persistent Embree/OptiX/Vulkan scene from the world's `` geometry. Publishes the current map through an in-process registry keyed by `map_key` (default `"default"`). Adding/removing geometry triggers a full acceleration-structure rebuild; moving already-tracked geometry refits the existing one in place instead (all three backends). - **Sensor system** (`rmagine_embree_sensor_system` / `rmagine_optix_sensor_system` / `rmagine_vulkan_sensor_system`, attached once per ``, like the map system): auto-discovers every `` anywhere in the world via gz-sim's `components::CustomSensor` (the closest available analogue to Gazebo Classic's `GZ_REGISTER_STATIC_SENSOR`; gz-sensors' own plugin-loading mechanism for custom sensor types was removed upstream). For each discovered sensor it looks up the map by `map_key`, raycasts against it (Spherical/Pinhole/O1Dn/OnDn models), and publishes `gz.msgs.LaserScan` (Spherical, single-ring only) and `gz.msgs.PointCloudPacked` over plain **gz-transport**; all sensors of one backend share a single `gz::transport::Node` owned by the factory system. This plugins are ROS-agnostic (map and sensor systems alike). If you want the data in ROS, bridge it with [`ros_gz_bridge`](https://github.com/gazebosim/ros_gz), see [Bridging to ROS](#usage) above. TF isn't published by this plugin either: attach gz-sim's own `gz::sim::systems::PosePublisher` to your robot and bridge its `gz.msgs.Pose_V` output to `tf2_msgs/msg/TFMessage`, exactly as shown in the quickstart. ## Citation This package is a Gazebo front-end for [rmagine](https://github.com/uos/rmagine). Please reference the following paper when using it in your scientific work: ```bib @inproceedings{mock2023rmagine, title = {Rmagine: 3D Range Sensor Simulation in Polygonal Maps via Ray Tracing for Embedded Hardware on Mobile Robots}, author = {Mock, Alexander and Wiemann, Thomas and Hertzberg, Joachim}, booktitle = {IEEE International Conference on Robotics and Automation (ICRA)}, year = {2023}, doi = {10.1109/ICRA48891.2023.10161388} } ``` The paper is available on [IEEE Xplore](https://ieeexplore.ieee.org/document/10161388) and as a preprint on [arXiv](https://arxiv.org/abs/2209.13397).

This site is open source. Improve this page.