hugging-science/inference-checkpoint-saver

inference-checkpoint-saver

A reference implementation of checkpointed save & resume inference for AI models that need more processing time than a single available GPU session provides. Instead of requiring one uninterrupted session long enough for the entire task, generation state is written out mid-process and can be reloaded later to resume from the exact point it left off, with no loss of progress and no discontinuity in the output.

This Space demonstrates the technique using LongCat-Video-Avatar-1.5-2nd, an audio-driven talking-head video model, as one example implementation: upload a reference image, a driving audio clip, and a short text prompt, and get back a lip-synced video up to 2 minutes long, generated at 480p using the INT8-quantized DiT with the DMD2-distilled 8-step LoRA.

While processing is underway, the full generation state — accumulated video latents, the VAE decoder's cache, the current diffusion step, and the reference inputs — is serialized to a portable .pt state file. That file can be loaded in any later session — here again, on your own local GPU, or anywhere else that runs this pipeline — and generation continues from the exact frame and diffusion step it left off at, with no skipped or discontinuous frames. This pattern generalizes to any autoregressive or chunked generation model that needs to produce long-form output under a limited amount of continuous compute time, not just this one.

• —Source: https://github.com/meituan-longcat/LongCat-Video
• —Weights: https://huggingface.co/meituan-longcat/LongCat-Video-Avatar-1.5