TsienDragon/flux-kontext-face-segmentation

Qwen-Image-Lora-Faceseg

<Gallery />

Model description

This is a LoRA fine-tuned face segmentation model based on Flux-Kontext architecture, specifically designed to transform facial images into precise segmentation masks. The model leverages the powerful multimodal capabilities of Flux-Kontext and enhances it through Parameter-Efficient Fine-Tuning (PEFT) using LoRA (Low-Rank Adaptation) technique.

Model Architecture

• —Base Model: Flux-Kontext-Dev
• —Fine-tuning Method: LoRA (Low-Rank Adaptation)
• —Task: Image-to-Image translation (Face → Segmentation Mask)
• —Input: RGB facial images
• —Output: Binary&#x2F;grayscale segmentation masks highlighting facial regions

Training Configuration

• —Dataset: 20 carefully curated face segmentation samples
• —Training Steps: 900-1000 steps
• —Prompt: &quot;change the image from the face to the face segmentation mask&quot;
• —Precision Options:
• —BF16 precision for high-quality results
• —FP4 quantization for memory-efficient deployment

Key Features

1. 1.High Precision Segmentation: Accurately identifies and segments facial boundaries with fine detail preservation
2. 2.Memory Efficient: FP4 quantized version maintains competitive quality while significantly reducing memory footprint
3. 3.Fast Inference: Optimized for real-time applications with 20 inference steps
4. 4.Robust Performance: Handles various lighting conditions and facial orientations
5. 5.Parameter Efficient: Only trains LoRA adapters (~18M parameters) while keeping base model frozen

Technical Specifications

• —Inference Steps: 20
• —CFG Scale: 2.5
• —Input Resolution: Configurable (typically 512x512)
• —Model Size: Base model + ~18M LoRA parameters
• —Memory Usage:
• —BF16 version: Higher memory, best quality
• —FP4 version: 75% memory reduction, competitive quality

Use Cases

• —Identity Verification: KYC (Know Your Customer) applications
• —Privacy Protection: Face anonymization while preserving facial structure
• —Medical Applications: Facial analysis and dermatological assessments
• —AR&#x2F;VR Applications: Real-time face tracking and segmentation
• —Content Creation: Automated face masking for video editing

Performance Highlights

• —Accuracy: Significantly improved boundary detection compared to base model
• —Detail Preservation: Maintains fine facial features in segmentation masks
• —Consistency: Stable segmentation quality across different input conditions
• —Efficiency: FP4 quantization achieves 4x memory savings with minimal quality loss

Deployment Options

• —High-Quality Mode: BF16 precision for maximum accuracy
• —Efficient Mode: FP4 quantization for resource-constrained environments
• —Real-time Applications: Optimized inference pipeline for low-latency requirements This model represents a practical solution for face segmentation tasks, offering an excellent balance between accuracy, efficiency, and deployability across various hardware configurations

Example:

Control Images

Edited Image with Qwen-Image-Edit by promot &#x60;change the face to face segmentation mask&#x60;

After Lora Finetune with same prompt

Code

Lora Finetune of Qwen-Image-Edit Code here: https://github.com/tsiendragon/qwen-image-finetune

Download model

Download them in the Files & versions tab.