中文

FluxVLA Engine Quick Start Guide

FluxVLA EngineMarch 26, 2026

👉👉👉 FluxVLA Engine Quick Start Guide

FluxVLA is a unified platform for training, evaluating, and deploying robotic Vision-Language-Action (VLA) models. Focused on the field of robot learning, this project is dedicated to providing modular, extensible, and efficient deep learning solutions for various VLA architectures (such as OpenVLA, LlaVA-VLA, GR00T, Pi0.5, etc.) and cutting-edge vision/language/multimodal backbones (LLaMA, QwenVL, Gemma, DinoSigLIP, etc.).

FluxVLA is compatible with mainstream dataset formats (e.g., Parquet, RLDS) and supports distributed training, closed-loop simulation, and integrated evaluation—enabling a seamless workflow from training and testing to deployment. Its All-in-One configuration and automation tools simplify model development, experiment management, and real-world robot deployment for researchers and engineers.

I. Core Features

Unified Configuration Management

Features an All-in-One configuration design where training, data, models, evaluation, and deployment settings are managed centrally to simplify workflows.

Low Migration Cost

Quickly switch between training pipelines, model architectures, datasets, simulation environments, and real-robot deployments by simply modifying specific modules in the configuration file.

Low Deployment Cost

In the same environment, real-robot deployment can be completed by simply downloading the saved work_dir directory, requiring no additional configuration or downloads.

Efficient Training & Testing

Supports FSDP multi-node distributed training and LIBERO multi-machine parallel closed-loop simulation, significantly boosting efficiency.

Integrated Training & Evaluation

Built-in LIBERO simulation evaluation allows for automatic closed-loop simulation testing upon training completion, achieving a seamless transition between training and assessment.

II. Project Overview

2.1 Supported VLA Model Architectures

  • OpenVLA: Open vision-language-action model based on the Transformer architecture.
  • LlavaVLA: A VLA variant based on the LLaVA architecture.
  • Gr00t: A VLA model supporting multiple vision and language backbones.
  • Pi0: An action prediction model based on Flow Matching.
  • Pi0.5: An improved version of Pi0.

2.2 Supported Backbone Networks

Language Model (LLM) Backbones

  • LLaMA Series
  • Gemma Series
  • Qwen Series

Vision Backbones

  • DinoSigLIP (Vision encoder combining DINO and SigLIP)

Vision-Language Model (VLM) Backbones

  • PaliGemma
  • QwenVL

2.3 Dataset Support

  • Parquet Datasets: Supports high-efficiency data loading in Parquet format.
  • RLDS Datasets: Supports datasets in TensorFlow RLDS format.
  • Multi-dataset Mixed Training: Supports simultaneous training on multiple datasets.

2.4 Training Features

  • Distributed Training: Supports FSDP (Fully Sharded Data Parallel) and DDP (Distributed Data Parallel).
  • Parameter-Efficient Fine-Tuning (PEFT): Supports LoRA (Low-Rank Adaptation) training modes.
  • Mixed Precision Training: Supports AMP (Automatic Mixed Precision) to improve training efficiency.
  • Checkpoint Recovery: Supports resuming training from specific checkpoints.
  • Post-training Evaluation: Supports automatic evaluation immediately after training.

2.5 Evaluation and Inference

  • Multi-GPU Evaluation: Supports distributed evaluation to accelerate the process.
  • LIBERO Benchmarking: Specialized support for the LIBERO robot learning benchmark, including evaluation on devices without hardware ray tracing (e.g., NVIDIA A100).
  • Real-Robot Inference: Provides specialized inference scripts for real-world robot environments.
  • Inference Mode: Supports skipping the loading of pre-trained weights during inference to reduce memory footprint.

III. Project Structure

fluxvla/
├── fluxvla/                  # Core package
│   ├── models/               # Model definitions
│   │   ├── vlas/             # VLA model implementations
│   │   ├── backbones/        # Backbone networks
│   │   ├── heads/            # Prediction heads
│   │   └── projectors/       # Projection layers
│   ├── datasets/             # Dataset loaders
│   ├── transforms/           # Data transformation and preprocessing
│   ├── tokenizers/           # Tokenizers and action encoders
│   ├── engines/              # Training engines and runners
│   ├── optimizers/           # Optimizers and schedulers
│   └── collators/            # Data collators
├── configs/                  # Configuration files
│   ├── openvla/              # OpenVLA configs
│   ├── llava/                # LlavaVLA configs
│   ├── gr00t/                # Gr00t configs
│   ├── pi0/                  # Pi0 configs
│   └── pi05/                 # Pi0.5 configs
├── scripts/                  # Training and evaluation scripts
│   ├── train.py              # Main training script
│   ├── eval.py               # Evaluation script
│   └── inference_real_robot.py  # Real-robot inference script
└── test/                     # Testing code

IV. Technical Architecture

4.1 Model Architecture

All VLA models inherit from the BaseVLA base class, utilizing a modular design:

  • Vision Encoder: Processes multi-view image inputs.
  • Language Encoder: Processes natural language instructions.
  • Projection Layer: Projects visual features into the language model space.
  • Action Prediction Head: Converts language model outputs into robotic actions.

4.2 Data Pipeline

  • Data Loading: Loads trajectory data from Parquet or RLDS formats.
  • Data Transformation: Applies image transformations, action normalization, etc.
  • Batching: Uses custom collators for batch processing.
  • Forward Pass: Propagates data through the VLA model.
  • Loss Calculation: Computes action prediction loss.
  • Backpropagation: Updates model parameters.

4.3 Training Workflow

  • Supports multiple training runners (FSDP/DDP).
  • Flexible optimizer and learning rate scheduling strategies.
  • Comprehensive logging and checkpointing mechanisms.

4.4 Use Cases

  • Robot Manipulation: Tasks such as grasping, placing, folding, etc.
  • Multi-task Learning: Training across multiple tasks to improve generalization.
  • Transfer Learning: Fine-tuning pre-trained models for specific downstream tasks.
  • Research & Experimentation: A unified platform for VLA architectural research.

4.5 Future Roadmap

  • Support for more vision and VLM backbones.
  • Training with VLM data or Chain-of-Thought (CoT) data.
  • Isaac Sim simulation environment support.
  • Full implementation of enhanced logging features.

4.6 Tech Stack

  • Deep Learning: PyTorch 2.6.0 / Transformers 4.53.2 / Flash Attention 2.5.5
  • Data/Simulation: TensorFlow (RLDS), Parquet, LIBERO

V. Conclusion

FluxVLA is a comprehensive and flexible VLA model training framework that provides powerful tool support for robot learning research. Through its modular architecture and rich configuration options, users can easily experiment with different model architectures, training strategies, and datasets, driving the advancement of Vision-Language-Action models in robotics.