wehub-resource-sync
232 lines
8.5 KiB
Markdown
232 lines
8.5 KiB
Markdown
<!-- WEHUB_ZH_README -->
|
||
> [!NOTE]
|
||
> 本文档由 WeHub 基于上游 README 翻译整理，属于社区翻译，非官方中文文档。
|
||
> [English](./README.en.md) · [原始项目](https://github.com/silverwingsbot/EasyCarla-RL) · [上游 README](https://github.com/silverwingsbot/EasyCarla-RL/blob/HEAD/README.md)
|
||
> 原作者、版权与许可证归属以原始项目及本仓库 LICENSE 文件为准。
|
||
|
||
🚀 [**新数据集已发布！**](#-download-dataset)
|
||
|
||
# EasyCarla-RL：基于 CARLA 模拟器构建的轻量级、新手友好的 OpenAI Gym 环境
|
||
|
||
## 概述
|
||
|
||
EasyCarla-RL 为 CARLA 模拟器提供了一个轻量、易用的 Gym 兼容接口，专为强化学习（RL）应用量身打造。它集成了激光雷达（LiDAR）扫描、自车状态、附近车辆信息和路点等核心观测组件。该环境支持带奖励与代价信号的安全感知学习、路点可视化，以及可自定义参数（包括交通设置、车辆数量和传感器范围）。EasyCarla-RL 旨在帮助研究人员和初学者在无需大量工程开销的情况下，高效训练与评估 RL 智能体。
|
||
|
||
<div align="center">
|
||
|
||
<table>
|
||
<tr>
|
||
<td><img src="assets/part1.gif" width="100%"/></td>
|
||
<td><img src="assets/part2.gif" width="100%"/></td>
|
||
<td><img src="assets/part3.gif" width="100%"/></td>
|
||
</tr>
|
||
</table>
|
||
|
||
</div>
|
||
|
||
## 安装
|
||
|
||
克隆仓库：
|
||
|
||
```bash
|
||
git clone https://github.com/silverwingsbot/EasyCarla-RL.git
|
||
cd EasyCarla-RL
|
||
```
|
||
|
||
安装所需依赖：
|
||
|
||
```bash
|
||
pip install -r requirements.txt
|
||
```
|
||
|
||
将 EasyCarla-RL 安装为本地 Python 包：
|
||
|
||
```bash
|
||
pip install -e .
|
||
```
|
||
|
||
请确保已运行与当前环境兼容的 [CARLA 模拟器](https://carla.org/) 服务端。
|
||
|
||
详细安装说明请参阅 [CARLA 官方文档](https://carla.readthedocs.io/en/0.9.13/start_quickstart/)
|
||
|
||
## 快速入门
|
||
|
||
运行一个简单演示，与环境交互：
|
||
|
||
```bash
|
||
python easycarla_demo.py
|
||
```
|
||
|
||
该脚本演示如何：
|
||
- 创建并重置环境
|
||
- 选择随机动作或自动驾驶（autopilot）动作
|
||
- 逐步推进环境并接收观测、奖励、代价与结束（done）信号
|
||
|
||
运行演示前，请确保 CARLA 服务端已启动。
|
||
|
||
## 进阶示例：使用 Diffusion Q-Learning 进行评估
|
||
|
||
进阶用法下，可在 EasyCarla-RL 环境中运行预训练的 [Diffusion Q-Learning](https://github.com/Zhendong-Wang/Diffusion-Policies-for-Offline-RL) 智能体：
|
||
|
||
```bash
|
||
cd example
|
||
python run_dql_in_carla.py
|
||
```
|
||
|
||
请确保已在 `example/params_dql/` 目录下下载或准备好训练好的模型检查点。
|
||
|
||
本示例演示：
|
||
- 加载预训练 RL 智能体
|
||
- 与 EasyCarla-RL 交互以进行评估
|
||
- 在模拟自动驾驶任务上评估真实 RL 模型的性能
|
||
|
||
## 📥 下载数据集
|
||
|
||
本仓库提供用于在 EasyCarla-RL 环境中训练与评估 RL 智能体的离线数据集。
|
||
|
||
该数据集包含超过 **7,000 条轨迹** 和 **110 万步时间步**，由专家策略与随机策略混合采集（专家与随机比例为 **8:2**），在 Town03 地图中记录。数据以 **HDF5 格式** 存储。
|
||
|
||
可从以下来源下载：
|
||
|
||
* [从 Hugging Face 下载（直链）](https://huggingface.co/datasets/silverwingsbot/easycarla/resolve/main/easycarla_offline_dataset.hdf5)
|
||
* [从百度网盘下载（提取码：2049）](https://pan.baidu.com/s/1yhCFzl4RFHzxfszebYnOIg?pwd=2049)
|
||
|
||
文件名：`easycarla_offline_dataset.hdf5` 大小：\~2.76 GB 格式：HDF5
|
||
|
||
### 数据集结构（HDF5）
|
||
|
||
数据集中每个样本包含以下字段：
|
||
|
||
```
|
||
/ (root)
|
||
├── observations → shape: [N, 307] # concatenated: ego_state + lane_info + lidar + nearby_vehicles + waypoints
|
||
├── actions → shape: [N, 3] # [throttle, steer, brake]
|
||
├── rewards → shape: [N] # scalar reward per step
|
||
├── costs → shape: [N] # safety-related cost signal per step
|
||
├── done → shape: [N] # 1 if episode ends
|
||
├── next_observations → shape: [N, 307] # next-step observations, same format as observations
|
||
├── info → dict containing:
|
||
│ ├── is_collision → shape: [N] # 1 if collision occurs
|
||
│ └── is_off_road → shape: [N] # 1 if vehicle leaves the road
|
||
```
|
||
|
||
* `N` 表示所有 episode 中的总时间步数（\~1.1 million）。
|
||
* `observations` 与 `next_observations` 是通过拼接得到的 307 维向量：
|
||
|
||
* `ego_state` (9) + `lane_info` (2) + `lidar` (240) + `nearby_vehicles` (20) + `waypoints` (36)
|
||
|
||
### 观测格式
|
||
|
||
数据集中每条观测以 **307 维扁平向量** 存储，按以下顺序拼接多个组件构成：
|
||
|
||
```python
|
||
# Flattening function used during data generation
|
||
|
||
def flatten_obs(obs_dict):
|
||
return np.concatenate([
|
||
obs_dict['ego_state'], # 9 dimensions
|
||
obs_dict['lane_info'], # 2 dimensions
|
||
obs_dict['lidar'], # 240 dimensions
|
||
obs_dict['nearby_vehicles'], # 20 dimensions
|
||
obs_dict['waypoints'] # 36 dimensions
|
||
]).astype(np.float32) # Total: 307 dimensions
|
||
```
|
||
|
||
该格式在保留关键空间与语义信息的同时，便于高效训练神经网络。
|
||
|
||
### 如何加载 HDF5 数据集并用于训练？
|
||
|
||
本示例展示如何加载离线数据集，并在典型的 RL 训练循环中使用。此处的模型仅为占位符——你可接入任意行为克隆（behavior cloning）、Q-learning 或 actor-critic 模型。
|
||
|
||
```python
|
||
import h5py
|
||
import torch
|
||
import numpy as np
|
||
|
||
# === Load dataset from HDF5 ===
|
||
with h5py.File('easycarla_offline_dataset.hdf5', 'r') as f:
|
||
observations = torch.tensor(f['observations'][:], dtype=torch.float32)
|
||
actions = torch.tensor(f['actions'][:], dtype=torch.float32)
|
||
rewards = torch.tensor(f['rewards'][:], dtype=torch.float32)
|
||
next_observations = torch.tensor(f['next_observations'][:], dtype=torch.float32)
|
||
dones = torch.tensor(f['done'][:], dtype=torch.float32)
|
||
|
||
# === (Optional) check shape info ===
|
||
print("observations:", observations.shape)
|
||
print("actions:", actions.shape)
|
||
|
||
# === Placeholder model example ===
|
||
class YourModel(torch.nn.Module):
|
||
def __init__(self, obs_dim, act_dim):
|
||
super().__init__()
|
||
# define your model here
|
||
pass
|
||
|
||
def forward(self, obs):
|
||
# define forward pass
|
||
return None
|
||
|
||
# === Training setup ===
|
||
model = YourModel(obs_dim=observations.shape[1], act_dim=actions.shape[1])
|
||
optimizer = torch.optim.Adam(model.parameters(), lr=3e-4)
|
||
loss_fn = torch.nn.MSELoss()
|
||
|
||
# === Offline RL training loop ===
|
||
for epoch in range(1, 11): # e.g. 10 epochs
|
||
for step in range(100): # e.g. 100 steps per epoch
|
||
# sample random batch
|
||
idx = np.random.randint(0, len(observations), size=256)
|
||
obs_batch = observations[idx]
|
||
act_batch = actions[idx]
|
||
rew_batch = rewards[idx]
|
||
next_obs_batch = next_observations[idx]
|
||
done_batch = dones[idx]
|
||
|
||
# forward, compute loss
|
||
pred = model(obs_batch) # e.g. predict action or Q-value
|
||
loss = loss_fn(pred, act_batch) # just an example
|
||
|
||
# backward and update
|
||
optimizer.zero_grad()
|
||
loss.backward()
|
||
optimizer.step()
|
||
|
||
print(f"[Epoch {epoch}] Loss: {loss.item():.4f} # Replace with your own logging or evaluation")
|
||
```
|
||
|
||
## 项目结构
|
||
|
||
```
|
||
EasyCarla-RL/
|
||
├── easycarla/ # Main environment module (Python package)
|
||
│ ├── envs/
|
||
│ │ ├── __init__.py
|
||
│ │ └── carla_env.py # Carla environment wrapper following the Gym API
|
||
│ └── __init__.py
|
||
├── example/ # Advanced example
|
||
│ ├── agents/
|
||
│ ├── params_dql/
|
||
│ ├── utils/
|
||
│ └── run_dql_in_carla.py # Script to run a pretrained RL model
|
||
├── easycarla_demo.py # Quick Start demo script (basic Gym-style environment interaction)
|
||
├── requirements.txt
|
||
├── setup.py
|
||
└── README.md
|
||
```
|
||
|
||
## 许可证
|
||
|
||
本项目采用 [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). 许可证授权。
|
||
|
||
## 作者
|
||
|
||
由 [SilverWings](https://github.com/silverwingsbot) 创建。
|
||
|
||
## 💓 致谢
|
||
|
||
本项目的实现离不开以下杰出的开源贡献：
|
||
|
||
- [CARLA](https://github.com/carla-simulator/carla)
|
||
- [gym-carla](https://github.com/cjy1992/gym-carla)
|
||
- [Diffusion Q-Learning](https://github.com/Zhendong-Wang/Diffusion-Policies-for-Offline-RL)
|