praveenkrovvidi/rl-cas-trl-agent

RL-CAS TRL Agent

Fine-tuned Qwen2.5-3B-Instruct that selects enterprise tools to resolve customer service queries. Trained via SFT + GRPO on expert trajectories from an RL-trained SAC agent.

Project: huggingface.co/spaces/praveenkrovvidi/rl-cas

Training Pipeline

1. 1.SAC-Discrete (Round 1) -- MLP [256,256] trained for 3,000 episodes on a 14-signal dense reward environment with 13 actions (7 API tools + 5 terminal + 1 meta)
2. 2.SFT (Round 2a) -- QLoRA fine-tuning on SAC expert trajectories (reward >= 5.0), 3 epochs, lr=2e-4
3. 3.GRPO (Round 2b) -- RL with live environment rewards, 482 steps, lr=5e-5, 4 completions/prompt

Model Details

Input / Output

Input: System prompt + user query describing a customer service issue.

Output: JSON with chain-of-thought reasoning and action ID.

{"reasoning": "Customer needs order status. I should look up their order first.", "action_id": 0}

Action Space

Benchmark (100 Queries, 8 Categories)

Intended Use

Designed for the RL-CAS customer service environment. Selects tools from a fixed action space -- does not generate free-form customer responses.

Limitations

• —97% resolution rate (struggles with auth and refund multi-tool sequences)
• —Requires the RL-CAS mock backend for tool execution
• —Trained on synthetic customer service data, not real customer interactions

Citation

@misc{rlcas2026,
  title={RL-CAS: Reinforcement Learning for Customer Service Agent},
  author={Praveen Krovvidi},
  year={2026},
  url={https://huggingface.co/praveenkrovvidi/rl-cas-trl-agent}
}