muffin2006/document-classification-env
1
1# 📊 VISUAL GUIDE - HOW THE OPENENV WORKS2 3## 🎬 Complete Agent Interaction Flow4 5```6┌──────────────────────────────────────────────────────────────────┐7│ OPENENV DOCUMENT CLASSIFICATION ENVIRONMENT │8└──────────────────────────────────────────────────────────────────┘9 10════════════════════ START EPISODE ════════════════════11 12 env.reset()13 │14 ▼15 ┌─────────────────────────────────────┐16 │ Generate Documents │17 │ ───────────────────────────────── │18 │ Easy: 100 documents │19 │ Medium: 500 documents │20 │ Hard: 1000 documents │21 │ │22 │ Categories: [5/10/20] │23 │ Features: TF-IDF (100-dim) │24 └─────────────────────────────────────┘25 │26 ▼27 ┌─────────────────────────────────────┐28 │ Return Initial Observation │29 │ ───────────────────────────────── │30 │ { │31 │ "document_id": "doc_000000", │32 │ "content": "My invoice...", │33 │ "word_count": [47], │34 │ "features": [0.12, -0.34, ...], │35 │ "document_index": [0], │36 │ "total_documents": [100] │37 │ } │38 └─────────────────────────────────────┘39 │40════════════════════ AGENT LOOP ════════════════════41 │42 ├─→ [ DOCUMENT 1 ]43 │ │44 │ ▼45 │ ┌────────────────────────────────┐46 │ │ Agent Analyzes Document │47 │ │ "My invoice shows error" │48 │ │ → Features: TF-IDF weights │49 │ │ → Decision: Category? │50 │ └────────────────────────────────┘51 │ │52 │ ▼53 │ ┌────────────────────────────────┐54 │ │ Agent Decision │55 │ │ ────────────────────────── │56 │ │ Possible Actions (Easy): │57 │ │ • 0 = General │58 │ │ • 1 = Billing ◄─ PREDICTED │59 │ │ • 2 = Support │60 │ │ • 3 = Technical │61 │ │ • 4 = HR │62 │ │ │63 │ │ Agent chooses: action = 1 │64 │ └────────────────────────────────┘65 │ │66 │ ▼67 │ env.step(action=1)68 │ │69 │ ▼70 │ ┌────────────────────────────────┐71 │ │ Evaluate Decision │72 │ │ ────────────────────────── │73 │ │ True Label: Billing (1) ✓ │74 │ │ Predicted: Billing (1) ✓ │75 │ │ Result: CORRECT! │76 │ └────────────────────────────────┘77 │ │78 │ ▼79 │ ┌────────────────────────────────┐80 │ │ Calculate Reward │81 │ │ ────────────────────────── │82 │ │ Correct: +1.0 ✓ │83 │ │ Speed: +0.1 (< 100ms) ✓ │84 │ │ ───────────────────────── │85 │ │ Total: +1.1 │86 │ └────────────────────────────────┘87 │ │88 │ ▼89 │ ┌────────────────────────────────┐90 │ │ Update Episode Stats │91 │ │ ────────────────────────── │92 │ │ Correct: 1/1 = 100% │93 │ │ Total Reward: 1.1 │94 │ │ Avg Reward: 1.1 │95 │ └────────────────────────────────┘96 │ │97 ├─→ [ DOCUMENT 2 ]98 │ │99 │ ▼100 │ ┌────────────────────────────────┐101 │ │ "I found a bug in system" │102 │ │ True Label: Technical (3) │103 │ │ Predicted: Support (2) ✗ │104 │ │ ────────────────────────── │105 │ │ Correct: -0.5 ✗ │106 │ │ Speed: +0.1 ✓ │107 │ │ Total: -0.4 │108 │ │ ────────────────────────── │109 │ │ Accuracy: 1/2 = 50% │110 │ └────────────────────────────────┘111 │ │112 │ ▼113 │ ... REPEATS FOR ALL DOCUMENTS ...114 │ │115 ├─→ [ DOCUMENT N ]116 │ │117 │ ▼118 │ ┌────────────────────────────────┐119 │ │ Last Document │120 │ │ All predictions made │121 │ │ Episode complete! │122 │ └────────────────────────────────┘123 │124════════════════════ END EPISODE ════════════════════125 │126 ▼127 ┌─────────────────────────────────────┐128 │ Episode Summary │129 │ ───────────────────────────────── │130 │ • Total Accuracy: 87/100 = 87% │131 │ • Total Reward: 87.3 │132 │ • Avg Time/Doc: 50ms │133 │ • Final Score: 0.87 (Easy) │134 │ ───────────────────────────────── │135 │ Returns: │136 │ - obs: empty (episode done) │137 │ - reward: 0.0 │138 │ - done: True │139 │ - info: │140 │ └─ episode_summary: {...} │141 └─────────────────────────────────────┘142 │143 ▼144 Agent has learned from 100 documents!145 Ready for next episode or deployment.146```147 148---149 150## 📈 Task Difficulty Progression151 152```153EASY (BASELINE: 0.78)154┌────────────────────────────────┐155│ Categories: 5 (Simple) │156│ ├─ General │157│ ├─ Billing │158│ ├─ Support │159│ ├─ Technical │160│ └─ HR │161│ │162│ Documents: 100 (Pre-processed) │163│ Time Limit: None │164│ Challenge Level: ⭐ │165│ Training Time: 1-2 seconds │166└────────────────────────────────┘167 │168 │ Difficulty increases169 ▼170MEDIUM (BASELINE: 0.65)171┌────────────────────────────────┐172│ Categories: 10 (Detailed) │173│ ├─ General │174│ ├─ Billing │175│ ├─ Billing-Dispute ◄ NEW │176│ ├─ Support │177│ ├─ Technical │178│ ├─ Technical-Bug ◄ NEW │179│ ├─ HR-Payroll │180│ ├─ HR-Benefits │181│ ├─ Legal │182│ └─ Executive │183│ │184│ Documents: 500 (Raw text) │185│ Time Limit: 2 seconds/doc │186│ Challenge Level: ⭐⭐ │187│ Training Time: 10-15 seconds │188└────────────────────────────────┘189 │190 │ Difficulty increases191 ▼192HARD (BASELINE: 0.52)193┌────────────────────────────────┐194│ Categories: 20 (Fine-grained) │195│ ├─ General │196│ ├─ Billing / Billing-Dispute │197│ ├─ Billing-Refund ◄ NEW │198│ ├─ Support-Urgent / Support- │199│ │ Normal ◄ NEW (split) │200│ ├─ Technical / Technical-Bug │201│ ├─ Technical-Feature ◄ NEW │202│ ├─ HR (3 variants) │203│ ├─ Legal (3 variants) │204│ ├─ Executive (2 variants) ◄NEW │205│ ├─ Finance ◄ NEW │206│ ├─ Marketing ◄ NEW │207│ └─ Operations ◄ NEW │208│ │209│ Documents: 1000 (Complex) │210│ Time Limit: 1 second/doc │211│ Challenge Level: ⭐⭐⭐ │212│ Training Time: 30-60 seconds │213│ Speed-Accuracy Tradeoff: YES │214└────────────────────────────────┘215```216 217---218 219## 🏆 Scoring System Visualization220 221```222┌─────────────────────────────────────────────────────────┐223│ EASY TASK SCORING │224├─────────────────────────────────────────────────────────┤225│ Calculation: SCORE = ACCURACY │226│ │227│ Example: 87 correct out of 100 │228│ ──────────────────────────────────────────────── │229│ Score = 87 / 100 = 0.87 │230│ │231│ Interpretation: │232│ 1.0 ████████████████████ Perfect │233│ 0.9 ██████████████████░░ Excellent │234│ 0.8 ████████████████░░░░ Good ◄─ 0.87 (HERE) │235│ 0.7 ██████████████░░░░░░ Fair │236│ 0.6 ████████████░░░░░░░░ Poor │237│ 0.5 ██████████░░░░░░░░░░ Baseline │238│ 0.0 ░░░░░░░░░░░░░░░░░░░░ Random │239└─────────────────────────────────────────────────────────┘240 241┌─────────────────────────────────────────────────────────┐242│ MEDIUM TASK SCORING │243├─────────────────────────────────────────────────────────┤244│ Calculation: SCORE = 0.8 × ACCURACY + 0.2 × SPEED │245│ │246│ Example: │247│ Accuracy = 68% │248│ Processing Time = 140ms (within 2s limit) → 0.15 │249│ ──────────────────────────────────────────────────── │250│ Score = 0.8 × 0.68 + 0.2 × 0.15 │251│ Score = 0.544 + 0.03 = 0.574 │252│ │253│ (Without speed bonus it would be 0.544) │254└─────────────────────────────────────────────────────────┘255 256┌─────────────────────────────────────────────────────────┐257│ HARD TASK SCORING │258├─────────────────────────────────────────────────────────┤259│ Calculation: SCORE = 0.75 × ACCURACY + 0.25 × SPEED │260│ │261│ Example: │262│ Accuracy = 73% (1000 docs, this is hard!) │263│ Processing Time = 95ms (< 100ms) → 0.20 bonus │264│ ──────────────────────────────────────────────────── │265│ Score = 0.75 × 0.73 + 0.25 × 0.20 │266│ Score = 0.5475 + 0.05 = 0.5975 │267│ │268│ Speed is MORE valued in hard task (tradeoff) │269└─────────────────────────────────────────────────────────┘270```271 272---273 274## 💰 Reward Mechanics275 276```277CORRECT CLASSIFICATION278┌─────────────────────┐279│ Accuracy: +1.0 │280└─────────────────────┘281 +282┌─────────────────────┐ ┌──────────────────────────┐283│ Speed Bonus: │────▶│ Easy: +0.1 (< 100ms) │284│ │ │ Medium: +0.15 (< 200ms) │285│ │ │ Hard: +0.2 (< 100ms) │286└─────────────────────┘ └──────────────────────────┘287 │288 ▼289 +1.1 to +1.2 Reward ✓290 291━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━292 293INCORRECT CLASSIFICATION294┌─────────────────────┐295│ Accuracy: -0.5 │296└─────────────────────┘297 +298┌─────────────────────┐ ┌──────────────────────────┐299│ Speed Bonus: │────▶│ Easy: +0.1 (< 100ms) │300│ │ │ Medium: +0.15 (< 200ms) │301│ │ │ Hard: +0.2 (< 100ms) │302└─────────────────────┘ └──────────────────────────┘303 │304 ▼305 -0.4 to -0.3 Reward ✗306 307━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━308 309EPISODE REWARD ACCUMULATION (100 documents, Easy)310┌──────────────────────────────────────────────┐311│ Doc 1: +1.1 → Total: 1.1 Acc: 100% │312│ Doc 2: -0.4 → Total: 0.7 Acc: 50% │313│ Doc 3: +1.1 → Total: 1.8 Acc: 67% │314│ Doc 4: +1.1 → Total: 2.9 Acc: 75% │315│ ... │316│ Doc 87: +1.1 → Total: 87.3 Acc: 87% │ ◄─ Final317│ Doc 88: -0.4 → Total: 86.9 Acc: 86.36% │318│ ... │319│ Doc100: +1.1 → Total: 87.3 Acc: 87% │320│ │321│ Average Reward = 87.3 / 100 = 0.873 │322│ Final Score = 0.87 │323└──────────────────────────────────────────────┘324```325 326---327 328## 📊 Document Feature Representation329 330```331Document Text332 │333 ▼334"I was overcharged on my last invoice"335 │336 │ (TF-IDF Vectorizer)337 │ Extract important words338 ▼339[word_weights: 100 dimensional vector]340 │341 │ Common words (low importance):342 │ "I", "on", "my" → near 0343 │344 │ Important words (high importance):345 │ "overcharged" → 0.45 (rare, specific)346 │ "invoice" → 0.52 (rare, specific)347 │ "billing" → 0.38 (related, important)348 │349 ▼350[-0.02, 0.45, 0.52, -0.01, ..., 0.38] ◄─ Fed to agent351 (100 dimensions total)352 │353 ▼354Agent Neural Net:355 Input: [100 features]356 ▼357 Hidden Layer 1: Learns feature combinations358 ▼359 Hidden Layer 2: Learns category patterns360 ▼361 Output: [5/10/20 category logits]362 ▼363 Softmax: Probability distribution364 ▼365 ArgMax: Best category366 ▼367 Action: Category index (0-4, 0-9, or 0-19)368 │369 ▼370Decision: "This is BILLING (category 1)"371```372 373---374 375## 🎯 Agent Learning Process376 377```378Episode 1: Random Guessing379┌──────────────┐380│ Accuracy: 20%│ ◄─ 1 in 5 correct (random)381│ Score: 0.20 │382└──────────────┘383 │384 ▼385Agent learns: Document contains "invoice" → likely Billing386 387 │388 ▼389Episode 2-5: Learning Keywords390┌──────────────┐391│ Accuracy: 45%│ ◄─ Better than random392│ Score: 0.45 │393└──────────────┘394 │395 ▼396Agent learns: Category combinations, priority397 398 │399 ▼400Episode 6-20: Pattern Recognition401┌──────────────┐402│ Accuracy: 75%│ ◄─ Strong signal learning403│ Score: 0.75 │404└──────────────┘405 │406 ▼407Agent learns: Subtle distinctions, edge cases408 409 │410 ▼411Episode 21+: Expert Level412┌──────────────┐413│ Accuracy: 87%│ ◄─ Near human performance414│ Score: 0.87 │415└──────────────┘416 │417 ▼418Agent learns: Rare cases, complex routing419 420 │421 ▼422Final: Deployed423```424 425---426 427## 🔄 Full Example: One Complete Classification428 429```430STEP 1: Document Arrives431┌──────────────────────────────────┐432│ Email: "I was charged twice for │433│ my order #5432. Please refund │434│ one charge immediately!" │435└──────────────────────────────────┘436 437STEP 2: Features Extracted438┌──────────────────────────────────┐439│ Tokenized: ["charged", "twice", │440│ "order", "refund", "immediately"]│441│ │442│ TF-IDF Vector: 100-dimensional │443│ [0.45, 0.38, 0.52, 0.41, ...] │444└──────────────────────────────────┘445 446STEP 3: Agent Processes447┌──────────────────────────────────┐448│ Input: Feature vector │449│ Process: Neural network inference│450│ Logits: [0.1, 0.9, 0.3, ...] │451│ Probabilities: │452│ General: 5% │453│ Billing-Dispute: 85% ◄─ MAX │454│ Support: 10% │455└──────────────────────────────────┘456 457STEP 4: Decision Made458┌──────────────────────────────────┐459│ Selected Category: │460│ action = 2 (Billing-Dispute) │461│ │462│ Reasoning: │463│ • "charged" + "twice" = dispute │464│ • "refund" = compensation claim │465│ • "immediately" = urgent tone │466└──────────────────────────────────┘467 468STEP 5: Evaluation469┌──────────────────────────────────┐470│ Ground Truth: Billing-Dispute │471│ Prediction: Billing-Dispute │472│ Result: ✓ CORRECT │473│ │474│ Reward: │475│ • Accuracy: +1.0 │476│ • Speed (45ms): +0.1 │477│ • Total: +1.1 │478└──────────────────────────────────┘479 480STEP 6: Routing481┌──────────────────────────────────┐482│ Route to Department: │483│ → Billing Team │484│ → Dispute Resolution Sub-team │485│ → Priority: High │486│ │487│ Action Taken: Email forwarded │488│ Learning: Agent improved score │489└──────────────────────────────────┘490```491 492---493 494## 📈 Performance Benchmarks495 496```497Baseline Agent Performance Across Tasks:498 499Easy Task (5 categories)500┌─────────────────────────────────┐501│ Accuracy: 78% ███████░░ │502│ Reward: 0.83 │503│ Time: 48ms │504│ Score: 0.78 │505└─────────────────────────────────┘506 ├─ Why 78%? Keyword matching507 │ works well for simple cases508 └─ Gap to perfect: 22%509 510Medium Task (10 categories)511┌─────────────────────────────────┐512│ Accuracy: 68% ██████░░░░ │513│ Reward: 0.71 │514│ Time: 150ms │515│ Score: 0.65 │516└─────────────────────────────────┘517 ├─ Why 68%? More categories,518 │ harder to distinguish519 └─ Gap to target (0.85): 20%520 521Hard Task (20 categories)522┌─────────────────────────────────┐523│ Accuracy: 55% █████░░░░░░░░ │524│ Reward: 0.57 │525│ Time: 100ms │526│ Score: 0.52 │527└─────────────────────────────────┘528 ├─ Why 55%? Complex routing,529 │ tight time constraints530 └─ Gap to target (0.75): 23%531 532Expected ML Model Performance:533┌─────────────────────────────────┐534│ Easy: 0.95+ (Expert) │535│ Medium: 0.85+ (Strong) │536│ Hard: 0.75+ (Capable) │537└─────────────────────────────────┘538```539 540---541 542## 🎓 Summary543 544```545┌─────────────────────────────────────────────────────┐546│ COMPLETE OPENENV DOCUMENT CLASSIFICATION SYSTEM │547├─────────────────────────────────────────────────────┤548│ │549│ INPUT: Documents + Content + Features │550│ ▼ │551│ PROCESS: TF-IDF → Agent Decision → Evaluation │552│ ▼ │553│ OUTPUT: Score (0.0-1.0) + Feedback │554│ │555│ RESULT: Agents learn to route documents! │556│ │557├─────────────────────────────────────────────────────┤558│ Ready to deploy and test! │559└─────────────────────────────────────────────────────┘560```561