suryanshchattree/npa-ppo-train

feat: durable training-log persistence for PPO Space

feat: PPO multi-GPU DDP trainer + Hugging Face Space (4x L4)

new commit

hello

feat: implement robust DQN training pipeline with observation normalization, AdamW optimization, and expanded testing scripts

feat: increase DQN training episodes and remove tick cap

feat: offline DQN training and traffic randomness

feat: upgrade DQN agent to support offline pre-training and fast inference

feat: added configurable traffic randomness and traffic_seed

docs: auto-update openapi.yaml with latest app.py endpoints and schemas

feat: added advantage normalization and entropy regularization to PPO

feat: upgrade PPO training with domain randomization and advantage baseline

fix: correct traffic arrival rate multiplier in environment

chore: upload PPO Lora weights via LFS

fix: PPO agent PEFT adapter injection and FastAPI return payload formatting

feat: add PPO agent to FastAPI backend endpoints

feat: integrate Unsloth LoRA adapter with Qwen2 configuration patching for PPO agent V1

feat: integrate Unsloth LoRA adapter with Qwen2 configuration patching for PPO agent V1

Disable KV cache in forward pass to prevent inplace Autograd error

Fix PyTorch Inference Mode Autograd tensor error

Switch to custom PyTorch REINFORCE algorithm to bypass TRL dependencies

Resolve merge conflict, fix PPO setup, add peft/trl requirements

changed requirements

updated LRU

reverted to previous commit

changed to tiny Lama

added a shared session for multiple agents

changed sdk to docker

chore: update SDK from gradio to Docker and refine package versions in requirements

fix: revert fastapi and httpx to specific versions in requirements

chore: add bitsandbytes package to requirements

chore: update requirements to include transformers and accelerate packages

feat: implement LLM-based agent with Qwen2.5-3B for dynamic memory management decisions

fix: handle reset failures by skipping tasks instead of exiting and reduce maximum tick limits for medium and hard configurations

refactor: implement multi-tier GPU utilization heuristic in LRUAgent and reduce max tick limits in constants

feat: add gpu_total threshold check and update deadlock logic in environment terminators

fix: flag environment as crashed on any early termination regardless of penalty type

feat: implement batch admission logic with proportional GPU resource allocation across agents and environment components

feat: track and log per-tick token statistics across environment and agents

feat: update inference dashboard metrics and remove episode limit from agent simulations

feat: update observation metrics to include request counts and add settings endpoint to API

feat: add dynamic environment settings API and support for custom tick limits in simulations

feat: add /api/agents endpoint to list supported simulation agents and usage instructions

feat: add Deep Q-Network (DQN) agent and integrate into the simulation API with pessimistic state initialization in Q-learning agent

feat: implement and integrate a Q-Learning agent for environment optimization

refactor: simplify OpenAPI specification by removing detailed documentation and redundant endpoints

chore: update app_file path in README.md to reflect project structure

feat: implement LRU agent and transition from server-based API to local execution scripts

changed git ignore

fix app config in hf