danielfein/raid-dual-neologism-tokens

RAID Dual Neologism Tokens

This repo contains two learned token embeddings trained on matched RAID GPT-4 vs human text pairs.

Tokens:

• —<neo_detect_ai_900001>
• —<neo_detect_human_900002>

Artifacts:

• —concept_900001_token.pt: AI-style token checkpoint
• —concept_900002_token.pt: human-style token checkpoint
• —raid_eval_summary.json: held-out RAID dual-token evaluation
• —beemo_eval_summary.json: cross-dataset transfer evaluation on Beemo model outputs

The checkpoints are small PyTorch payloads with keys:

• —concept_id
• —new_token
• —new_token_id
• —embedding

Loading

import torch
payload = torch.load('concept_900001_token.pt', map_location='cpu')
token = payload['new_token']
embedding = payload['embedding']

These are not full finetuned model weights. They are learned embedding rows intended to be inserted into the base tokenizer/model vocabulary used during training.

Repo: danielfein/raid-dual-neologism-tokens