ServiceNow/PrivacyAlign-Nemotron-3-Nano-4B-Annotation-Conditioned-Reward

PrivacyAlign-Nemotron-3-Nano-4B-Annotation-Conditioned-Reward

A privacy-aligned variant of [nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16](https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16), fine-tuned with RL against an annotation-conditioned reward: a pairwise LLM judge that scores candidate responses conditioned on per-example privacy annotations.

Training

• —Method: online policy optimization (RL), KL-regularized to the base policy.
• —Algorithm: SAPO (tau_pos = 1.0, tau_neg = 1.05).
• —Reward: a pairwise LLM judge conditioned on the example's privacy annotations
• —Judge model: nvidia/NVIDIA-Nemotron-3-Nano-4B-BF16 (thinking enabled).

Hyperparameters

Paper & citation

For full details on the method, data, and evaluation, please read our paper:

[PrivacyAlign: Contextual Privacy Alignment for LLM Agents](https://arxiv.org/abs/2606.21710)

@article{tamber2026privacyaligncontextualprivacyalignment,
      title={PrivacyAlign: Contextual Privacy Alignment for LLM Agents},
      author={Manveer Singh Tamber and Abhay Puri and Marc-Etienne Brunet and Perouz Taslakian and Jimmy Lin and Spandana Gella},
      year={2026},
      eprint={2606.21710},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2606.21710},
}