blind-assist/nvila-walk-50samples

nvila-walk-50samples

Fine-tuned NVILA-Lite-2B model for blind assistance navigation.

Model Details

• —Base Model: Efficient-Large-Model/NVILA-Lite-2B
• —Training Dataset Size: 50 samples
• —Train: 35
• —Validation: 7
• —Test: 8
• —Training Date: 2025-12-28
• —Run Name: NVILA-2B-Walk-50samples-20251228_211018

Task

Given visual input from a user's forward perspective, generate exactly one short sentence to guide a visually impaired user by:

• —Identifying critical obstacles or landmarks
• —Describing locations using clock directions (12 o'clock is straight ahead)
• —Including relevant details (size, material, distance)
• —Giving one clear action
• —Prioritizing immediate safety

Example Output

"At 1 o'clock direction there is a tree, be careful to avoid it."

Usage

from llava.model.builder import load_pretrained_model

model_path = "blind-assist/nvila-walk-50samples"
tokenizer, model, image_processor, context_len = load_pretrained_model(model_path)

Training Configuration

• —Batch Size: 8 (effective)
• —GPUs: 1
• —Precision: BF16