How I set up a bimanual manipulation simulation, created a teleoperation and data collection pipeline, and trained an Action-Chunking Transformer model to use two arms to perform a household task.
See Demo at end
https://github.com/AlmondGod/aloha-bigym
The ALOHA two-arm system is an inexpensive open-source manipulation robot consisting of two single-gripper 7 DOF arms opposing each other on top of a wooden table, with a metal frame and multiple cameras supporting the whole setup
BiGym is a demonstration based two-arm manipulation benchmark (for evaluating a robots’ ability to manipulate common household objects, for example closing a Dishwasher or moving blocks to a target)
Objective: The goal in this first step was to combine the ALOHA bimanual system simulated assets with the bigym assets in the same Mujoco physics simulation
I first obtained the ALOHA XML assets and tried to directly integrate them with the Bigym system in one Mujoco simulation. However, there were two obstacles:
Bigym requires the manipulator to be a member of the robot class, which comes with a large amount of specific requirements and attributes. Thus, I first had to go into the ALOHA XML assets and add/adjust/split structures to fit with the Bigym robot class.
I then had to create new robot-inheriting configuration and base classes for ALOHA (see bigym.envs.xmls.alohafor the modified folder of assets from ALOHA (particularly scene.xml, target.xml, and aloha.xml) and bigym.robots.configs.aloha.py for the config) and resolved the many little problems that arose in integrating two different systems
Now the remaining problem was how to adjust these Bigym tasks, meant for humanoid/mobile robots, to the stationary table-based ALOHA.
I created a new file for each Bigym task and edited the xmls, configs, and other files to adjust each task to be on a table
Some adjustments were conceptually straightforward, like moving the StackBlocks task from its table to the ALOHA table