1 Environment Setup

1.1 Prerequisites

GPU: Recommend GeForce RTX 3080 (12GB) or above
Operating System: Ubuntu 20.04 LTS
Network: GitHub access required
Ensure TRON1 systemsoftware is higher then 3.0.17

1.2 One-click RL Training Environment Installation

Run the following command in a terminal:

mkdir -p ~/limx_rl && cd ~/limx_rl && \
sudo apt update && sudo apt install -y git && \
if [ ! -d "pointfoot-legged-gym" ]; then \
  git clone https://github.com/limxdynamics/pointfoot-legged-gym.git; \
fi && \
cd pointfoot-legged-gym && bash install.sh && source ~/.bashrc

After executing these commands, please wait till you see the following output in the terminal:

At this point, hold down the Enter key to scroll through and read the Anaconda license agreement. You will then see:

Type yes and press Enter. When prompted for the Anaconda installation path (usually no need to change), simply press Enter:

Type no and press Enter to complete the Anaconda installation.

The script will continue installing pointfoot-legged-gym and its dependencies.

1.3 Verifying the Training Environment

1.3.1 Verify NVIDIA Driver

nvidia-smi

Driver has been successfully installed when NVIDIA driver version and GPU information are displayed

1.3.2 Verify Isaac Gym

Run the example blow to verify that Isaac Gym has been correctly installed and configured.

conda activate pointfoot_legged_gym
cd ~/limx_rl/isaacgym/python/examples
python 1080_balls_of_solitude.py

1.3.3 Verify pointfoot-legged-gym

After completing the environment setup, your ~/limx_rl directory should contain the following structure:

cd ~/limx_rl
.
├── isaacgym
└── pointfoot-legged-gym

1.4 One-click RL Deployment Environment Installation

Open a terminal and enter the following command

提示：
Note: The Conda environments of training and deployment are separate.

source ~/anaconda3/etc/profile.d/conda.sh && \
conda create -n pointfoot_deploy python=3.8 -y && \
conda activate pointfoot_deploy && \
mkdir -p ~/limx_ws && cd ~/limx_ws && \
git clone --recurse-submodules https://github.com/limxdynamics/pointfoot-mujoco-sim.git && \
git clone --recurse-submodules https://github.com/limxdynamics/rl-deploy-with-python.git && \
ARCH=$(uname -m) && \
if [ "$ARCH" = "x86_64" ]; then \
  pip install rl-deploy-with-python/pointfoot-sdk-lowlevel/python3/amd64/limxsdk-\*-py3-none-any.whl; \
elif [ "$ARCH" = "aarch64" ]; then \
  pip install rl-deploy-with-python/pointfoot-sdk-lowlevel/python3/aarch64/limxsdk-\*-py3-none-any.whl; \
else \
  echo "Unsupported architecture：$ARCH"; \
fi && \
pip install onnx && \
echo 'export ROBOT_TYPE=PF_TRON1A' >> ~/.bashrc && source ~/.bashrc

1.5 Validation of Deployment Environment

Open a bash terminal to run control policy:

# Activate the pointfoot_deploy Conda environment
conda activate pointfoot_deploy

# Run the control algorithm
cd ~/limx_ws && python rl-deploy-with-python/main.py

Open a bash terminal to run the MuJoCo simulator:

# Activate the pointfoot_deploy Conda environment
conda activate pointfoot_deploy

# Run the simulator
cd ~/limx_ws && python pointfoot-mujoco-sim/simulator.py

Open a bash terminal to launch the virtual joystick:

cd ~/limx_ws && ./pointfoot-mujoco-sim/robot-joystick/robot-joystick

During simulation, you can use the virtual joystick to control the robot's movement:

Left stick: Forward, backward, turning left, and turning right
Right stick: Lateral (side-to-side) movement

If the robot moves correctly in MuJoCo, the environment has been successfully configured.

If the robot appears to have fallen, click the Reset button in the left menu after launching the motion controller, the robot should then recover and begin walking.

2 TRON 1 Point-foot Mode Training

2.1 Launch Training

Switch to the correct branch:

cd ~/limx_rl/pointfoot-legged-gym && git checkout encoder-actor-critic

Commonly Used Training Command Parameters： - --task=pointfoot_flat: Specifies the task or environment type as pointfoot_flat.

--num_envs 1024: Specifies the number of environments to be created as 1024.
--max_iteration 10000:** Specifies the number of training iterations as 10,000.
--headless: No graphical interface is displayed. This is typically used when running on a server without a monitor or when high computational efficiency is needed without graphical rendering.

Training：
a. Recommended training command (headless mode):

conda activate pointfoot_legged_gym
cd ~/limx_rl/pointfoot-legged-gym
python legged_gym/scripts/train.py --task=pointfoot_flat --headless

In headless mode, the training will run without a graphical interface, which improves training speed.
b. GUI mode (for visualization):

conda activate pointfoot_legged_gym
cd ~/limx_rl/pointfoot-legged-gym
python legged_gym/scripts/train.py --task=pointfoot_flat

Running screen of Isaac Gym

You can press the V on your keyboard to pause rendering and speed up the training process.

2.2 Resume Training from a Checkpoint

You can resume from a specific checkpoint file
Make sure to replace the --load_run and --checkpoint arguments with the actual values corresponding to your training session.

cd ~/limx_rl/pointfoot-legged-gym
python legged_gym/scripts/train.py --task=pointfoot_flat --resume --headless --load_run Dec23_17-38-22_ --checkpoint 200

You can see an output line starting with Loading model from: , indicating that the checkpoint has been successfully loaded from the specified path to resume training.

2.3 Monitor Training with TensorBoard

Activate the Conda environment

conda activate pointfoot_legged_gym

Launch TensorBoard

cd ~/limx_rl/pointfoot-legged-gym
tensorboard --logdir=logs/pointfoot_flat

View training progress

Open a browser and navigate to [http://127.0.0.1:6006](http://127.0.0.1:6006/) to monitor training metrics

TensorBoard curves

2.4 Export Trained Models

View training results

By default, the most recent run and checkpoint will be loaded. If you want to specify a particular run and checkpoint, use the --load_run and --checkpoint arguments.

--load_run: Specifies the identifier of the training run to be loaded (e.g., the name or ID of the run). This identifier is typically associated with a training session and is used to locate the corresponding logs or configuration.
How to find it: Navigate to the logs directory and inspect its contents. The subdirectories are usually named using the run identifiers.
Example:

ls -l ~/limx_rl/pointfoot-legged-gym/logs/pointfoot_flat

You should see a directory named something like Dec23_17-38-22_. This is the value to be passed to --load_run.

--checkpoint: Specifies the checkpoint file to load. A checkpoint file contains the intermediate state of the model and can be used to resume training or perform inference.
- How to find it: Inside the logs directory, navigate to the subdirectory specified by --load_run. Checkpoint files are typically stored here with a .pt extension and often include the training iteration number or a timestamp in the filename.
- Example:

ls -l ~/limx_rl/pointfoot-legged-gym/logs/pointfoot_flat/Dec23_17-38-22_

You should see a file like model_200.pt, where 200 is the value to be used for the --checkpoint argument.

Example
- Please make sure to replace the parameters after --load_run and --checkpoint with your training logs directory.

conda activate pointfoot*legged_gym
cd ~/limx_rl/pointfoot-legged-gym
python legged_gym/scripts/play.py --task=pointfoot_flat --load_run Dec23_17-38-22* --checkpoint 200

Export training results

Export the Model as an ONNX Format File

After running the previous script, you can find the exported file in the directory:

ls -l ~/limx_rl/pointfoot-legged-gym/logs/pointfoot_flat/exported/policies

Exported files

2.5 Python based Sim-to-Sim Deployment

Using the PF_TRON1A robot model as an example, the RL policy and configuration files are located at: ~/limx_ws/rl-deploy-with-python/controllers/model/PF_TRON1A, as shown below.

tree ~/limx_ws/rl-deploy-with-python/controllers/model/PF_TRON1A
.
├── params.yaml
└── policy
└── policy.onnx
└── encoder.onnx

Move the exported training results, which are the ONNX files (policy.onnx and encoder.onnx) into the corresponding policy folder.
Launch the Simulation

Open a Bash terminal to run the control policy:

conda activate pointfoot_deploy
cd ~/limx_ws && python rl-deploy-with-python/main.py

Open a Bash terminal to launch the MuJoCo simulator:

conda activate pointfoot_deploy
cd ~/limx_ws && python pointfoot-mujoco-sim/simulator.py

Open a Bash terminal to start the virtual controller:

cd ~/limx_ws && ./pointfoot-mujoco-sim/robot-joystick/robot-joystick

You should then be able to observe the behavior of the trained policy deployed within the MuJoCo simulation environment.

3 Deployment On TRON 1

3.1 Real-robot Debugging

Set Developer PC IP Address: Ensure your development PC is connected to the robot via the external Ethernet port. Set IP address to 10.192.1.200, and verify the connection by running the following command in a shell:

Wired connection to the computer

Enter Developer Mode: After powering on the robot, press R1 + Left on the remote controller simultaneously. The robot will automatically reboot and switch into developer mode, which allows users to run custom motion control algorithms.This mode persists across reboots and does not reset after shutdown.
Joint Calibration：After the robot boots up and before running any control programs, perform a zeroing operation to return all joints to their initial positions. Use the combination L1 + R1 on the remote to trigger calibration.
Run On-Robot Deployment: Use the following shell command in a Bash terminal to start the control algorithm

Note: Ensure the robot is securely suspended during real-robot deployment for safety.

conda activate pointfoot_deploy
cd ~/limx_ws
python rl-deploy-with-python/main.py 10.192.1.2

Control the robot with:

L1 + △ to start walking
L1 + □ to stop
Left stick: forward/back/left/right
Right stick: lateral motion

3.2 Final Deployment on Hardware

After completing both simulation and real-robot debugging, you can proceed to deploy your control algorithm onto the robot.

Preparation
- Make sure the robot remains in developer mode to allow program deployment and debugging.
- Ensure your development PC is connected to the robot via the external Ethernet port, once deployment is complete, the network connection is no longer required.
Copy the Algorithm to the Robot
- Open a terminal and navigate to your working directory containing the algorithm, e.g.:~/limx_ws.
- Use the scp command to copy the algorithm directory to the robot, the default robot user is guest and the password is 123456:

cd ~/limx_ws
scp -r rl-deploy-with-python guest@10.192.1.2:/home/guest

Install SDK on the robot:

ssh guest@10.192.1.2 "pip install /home/guest/rl-deploy-with-python/pointfoot-sdk-lowlevel/python3/amd64/limxsdk-*-py3-none-any.whl"

Set up auto-launch:
Use the ssh command to remotely log into the robot's system. The default password is 123456:

ssh guest@10.192.1.2

Open the autolaunch script located at /home/guest/autolaunch/autolaunch.sh for editing:

busybox vi /home/guest/autolaunch/autolaunch.sh

In the script file, find the command that starts main.py: ensure that the line python3 /home/guest/rl-deploy-with-python/main.py 10.192.1.2 is not commented out (i.e., there is no # comment symbol at the beginning of the line). After editing is complete, please save the file and exit the editor.

#!/bin/bash

while true; do
# Launch the Python controller script for the Pointfoot robot
# The argument 10.192.1.2 specifies the IP address of the robot
# This line is currently commented out; uncomment it to enable automatic controller startup
# Modify this path according to the actual location of your controller script
  python3 /home/guest/rl-deploy-with-python/main.py 10.192.1.2

  # Use roslaunch to start the robot control algorithm
  # Uncomment the following lines to enable launching the robot control algorithm
  # Modify the paths according to your actual installation
  # source /home/guest/install/setup.bash
  # roslaunch robot_hw pointfoot_hw.launch

  # Restart after 3 seconds
  sleep 3
done

Reboot the robot to apply the configuration

Control the Robot's Movement
After the system has started, you can control the robot with:
- L1 + △: to start walking；
- L1 + □: to stop
- Left stick: forward/back/left/right
- Right stick: lateral motion

TRON1 RL Training & Deployment Guide（Isaac Gym）