ai4data/data-use-annotation

1# 📖 Data Use Annotation Tool — Annotator Guide2 3Welcome! This guide explains how to use the **Data Use Annotation Tool** to review and annotate data/dataset mentions in documents.4 5---6 7## 1. Getting Started8 9### Signing In101. Open the tool — you'll see a login screen112. Click **🤗 Sign in with HuggingFace**123. Authorize with your HuggingFace account134. You'll be redirected to the tool showing your assigned documents14 15> **Note:** Only accounts listed in the annotator configuration will see documents. If you see no documents after logging in, contact the admin.16 17---18 19## 2. Interface Overview20 21The tool has two main panels:22 23| Panel | Purpose |24|-------|---------|25| **Left — PDF Viewer** | Shows the original PDF for the current page |26| **Right — Markdown Annotation** | Shows extracted text with highlighted data mentions |27 28### Top Bar29- **Title** — "Data Use Annotation Tool"30- **Progress Bar** — Overall annotation progress across all corpora31- **User Badge** — Your HuggingFace username32- **📊 Leaderboard** — See annotation stats for all annotators33 34### Document Selector35- Dropdown at top-left showing your assigned documents36- Documents are labeled by corpus: **[World Bank]**, **[UNHCR]**, etc.37- Format: `[Corpus] Doc N (X pages)`38 39---40 41## 3. Page Navigation42 43At the bottom of the screen you'll find the page navigator:44 45```46⏮  ← Prev  |  Page 3 ●  (3 / 11)  |  Next →  ⏭47```48 49| Button | Action |50|--------|--------|51| **← Prev / Next →** | Move one page at a time |52| **⏮ / ⏭** | Jump to the previous/next page that has data mentions |53| **● (green dot)** | Indicates the current page has AI-detected data mentions |54 55All pages are shown, including those without mentions. Use the jump buttons to quickly navigate to pages of interest.56 57---58 59## 4. Understanding Data Mentions60 61The AI model pre-detects potential dataset mentions in the text. Each mention is highlighted with a color based on its **tag**:62 63| Color | Tag | Meaning |64|-------|-----|---------|65| 🟢 Green | **Named** | A specific, named dataset (e.g. "2022 National Census") |66| 🟡 Amber | **Descriptive** | A described but not formally named dataset (e.g. "a household survey") |67| 🟣 Purple | **Vague** | An unclear or ambiguous data reference |68| ⚪ Gray | **Non-Dataset** | Flagged by the model but not actually a dataset |69 70A **legend** above the text shows the count of each type on the current page.71 72---73 74## 5. Reviewing Existing Mentions (Validation)75 76Click the **toggle button (‹)** on the right edge to open the **Data Mentions** side panel. For each AI-detected mention you can:77 78### Validate791. Click **Validate** on a mention802. Optionally add notes explaining your decision813. Choose one of:82   - ✅ **Correct** — The mention is a real dataset83   - ❌ **Wrong** — The mention is not a dataset (false positive)84 85### Change Tag86- Click the **tag badge** (e.g. "Named") to edit it87- Select the correct tag from the dropdown88- Click **Save** to update89 90### Delete91- Click **🗑 Delete** to remove a false mention92- Click again to confirm (auto-cancels after 3 seconds)93 94### Status Indicators95- **"Needs review"** — Not yet validated by you96- **"✓ verified"** / **"✗ rejected"** — Your validation result97- A checkmark appears next to validated mentions98 99---100 101## 6. Adding New Annotations102 103If you spot a dataset mention that the AI missed:104 1051. **Select the text** — Click and drag to highlight the dataset name in the markdown preview1062. **Click "✍️ Annotate Selection"** — The annotation modal will appear1073. **Choose a Dataset Tag**:108   - **Named Dataset** — A specific named dataset109   - **Descriptive** — A described but unnamed dataset110   - **Vague** — An ambiguous reference1114. **Click "Save Annotation"** — Your annotation is saved112 113> **Tip:** If no text is selected when you click the button, it will shake to remind you to select text first.114 115---116 117## 7. Page Workflow118 119For each page, the recommended workflow is:120 1211. **Read** the markdown text on the right while referencing the PDF on the left1222. **Review** each highlighted mention — validate or reject in the side panel1233. **Add** any missed mentions using text selection1244. **Move** to the next page (a warning appears if you have unvalidated mentions)125 126### Unvalidated Mentions Warning127When moving to the next page with unvalidated mentions, you'll see:128> ⚠️ You have N unverified data mention(s) on this page. Do you want to proceed?129 130You can proceed or go back to finish validating.131 132---133 134## 8. Tips & Best Practices135 136- **Use the PDF** for context — the markdown is extracted text and may have formatting issues137- **Jump buttons (⏮/⏭)** let you skip pages without mentions quickly138- **Pages without mentions** may still contain datasets the AI missed — browse them when possible139- **Validate everything** on a page before moving on for the most efficient workflow140- **Be precise** when selecting text for new annotations — select just the dataset name, not surrounding context141 142---143 144## 9. FAQ145 146**Q: Can I undo a validation?**147A: Click "Validate" again to re-validate with a different verdict.148 149**Q: What if the markdown text doesn't match the PDF?**150A: This can happen with complex layouts (tables, figures). Annotate based on what you can read. The PDF is the source of truth.151 152**Q: Why are some pages empty?**153A: Some pages (like cover pages or blank pages) may have no extracted text. Use the jump buttons to skip them.154 155**Q: Who sees my annotations?**156A: Annotations are stored centrally. Admins and other annotators with access to the same documents may see your work.157 158---159 160## Need Help?161 162Contact the project admin if you encounter issues or have questions about specific annotation decisions.163