wehub-resource-sync
e06fe8e8c6
Secret Leaks / trufflehog (push) Failing after 1s
Build documentation / build (push) Failing after 1s
Build documentation / build_other_lang (push) Failing after 0s
CodeQL Security Analysis / CodeQL Analysis (push) Failing after 0s
PR CI / pr-ci (push) Failing after 1s
Slow tests on important models (on Push - A10) / Get all modified files (push) Failing after 1s
Slow tests on important models (on Push - A10) / Model CI (push) Has been skipped
Self-hosted runner (benchmark) / Benchmark (aws-g5-4xlarge-cache) (push) Has been cancelled
New model PR merged notification / Notify new model (push) Has been cancelled
Update Transformers metadata / build_and_package (push) Has been cancelled
2.0 KiB
2.0 KiB
TRL
TRL is a post-training framework for foundation models. It includes methods like SFT, GRPO, and DPO. Each method has a dedicated trainer that builds on the [Trainer] class and scales from a single GPU to multi-node clusters.
from datasets import load_dataset
from trl import GRPOTrainer
from trl.rewards import accuracy_reward
dataset = load_dataset("trl-lib/DeepMath-103K", split="train")
trainer = GRPOTrainer(
model="Qwen/Qwen2-0.5B-Instruct",
reward_funcs=accuracy_reward,
train_dataset=dataset,
)
trainer.train()
Transformers integration
TRL extends Transformers APIs and adds method-specific settings.
-
TRL trainers build on [
Trainer]. Method-specific trainers like [~trl.GRPOTrainer] add generation, reward scoring, and loss computation. Config classes extend [TrainingArguments] with method-specific fields. -
Model loading uses [
AutoConfig.from_pretrained], then instantiates the model class from the config with that class'from_pretrained.
Resources
- TRL docs
- Fine Tuning with TRL talk