Async GRPO with LoRA across HF Jobs: a bucket, a proxy, and no NCCL
LoRA support recently landed in TRL's AsyncGRPOTrainer with PR #7017 , and ships with TRL v1.14. The asynchronous trainer can now train an adapter instead of the full model, and it syncs only the LoRA adapter to vLLM. This post covers a real-world project built on top of it, where training and inference no longer shar…