Skip to content
KoishiAI
ไทย
← Back to all articles
huggingface diffusion-models quantization ai-tools flux-1

Nunchaku Diffusers Integration: Fast 4-Bit Inference

Hugging Face integrates Nunchaku into Diffusers for fast 4-bit diffusion inference. Run Flux.1 on consumer GPUs with SVDQuant and reduced VRAM usage.

KoishiAI · Reviewed by: เกียรติดำรง ตรีครุธพันธ์ · · 4 min read
AI-drafted from cited sources, fact-checked and reviewed by a human editor. How we work · Standards · Report an error
Close-up of two NVIDIA RTX 2080 graphics cards with dual fans, high-performance hardware.
Photo by Nana Dua on Pexels

TL;DR: Hugging Face integrated Nunchaku’s SVDQuant technology into Diffusers, enabling 4-bit diffusion models like Flux.1 to run on consumer GPUs with under 30GB VRAM. This eliminates complex setup barriers and democratizes access to high-performance image generation for developers worldwide.

Key facts

  • Hugging Face integrated Nunchaku into its Diffusers library, enabling standard from_pretrained() loading for 4-bit diffusion models without separate CUDA compilation.
  • Nunchaku utilizes SVDQuant technology developed by MIT and NVIDIA researchers, supporting W4A4 (4-bit weights/activations) or W4A8 configurations to bypass memory bandwidth bottlenecks.
  • The integration allows running large models like Flux.1-dev, which normally requires over 30GB VRAM in BF16, on consumer hardware such as the RTX 4090.
  • Nunchaku v1.2.0 provides a 20–30% performance boost for Z-Image models and supports INT4 quantization on NVIDIA 20-series GPUs via pre-built wheels for Linux and Windows.
  • GPU support is architecture-specific: INT4 is used for non-Blackwell GPUs (pre-50-series), while NVFP4 is optimized for Blackwell GPUs (50-series).
  • Ready-to-use checkpoints are available on Hugging Face for SDXL, FLUX.1-dev, Qwen-Image, and Qwen-Image-Edit-2509, including lightning variants distilled to 4 or 8 steps.
  • The SVDQuant research paper titled ‘Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models’ is scheduled for publication at ICLR 2025.

Hugging Face brings Nunchaku to Diffusers

Hugging Face has integrated Nunchaku, a high-performance inference engine optimized for 4-bit diffusion models, directly into its popular Diffusers library [1]. This development allows developers to load and run quantized checkpoints using standard methods like from_pretrained(), eliminating the previous requirement for separate inference libraries or local CUDA compilation [1]. The integration leverages the Hugging Face kernels package to handle specialized operations, making advanced quantization techniques accessible without complex setup [1].

Nunchaku is built on SVDQuant (Singular Value Decomposition Quantization), a method developed by researchers from MIT and NVIDIA [2]. Unlike traditional weight-only quantization which stores weights in low precision but dequantizes them at compute time—often adding latency overhead—SVDQuant runs transformer layers with both 4-bit weights and activations (W4A4 or W4A8) [1, 2]. This approach significantly reduces memory usage while accelerating the denoising loop by breaking through memory bandwidth bottlenecks common in large-scale diffusion models like Flux.1 and Stable Video Diffusion [2].

For example, a model like Flux.1-dev typically requires over 30GB of VRAM in BF16 precision; Nunchaku enables efficient inference on consumer-grade hardware such as the RTX 4090 or mid-range data center cards [2]. Large diffusion models like Flux.1-dev require over 30GB of VRAM in BF16 precision, creating a barrier for consumer hardware [2]. Nunchaku was developed by researchers from MIT and NVIDIA [2]. The SVDQuant paper, ‘Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models,’ is scheduled for publication at ICLR 2025 [3-8].

How SVDQuant Works

SVDQuant improves on standard quantization by handling outliers more effectively. Traditional methods often struggle with large activation values that can distort the model’s output when compressed to low bit-widths. By using Singular Value Decomposition, Nunchaku absorbs these outliers into low-rank components, allowing the remaining weights and activations to be safely quantized to 4 bits [1]. This results in both lower memory consumption and faster inference speeds compared to previous 4-bit implementations.

The SVDQuant method uses 4-bit weights and activations (W4A4/W4A8), unlike weight-only quantization methods that add latency overhead [1,2]. Nunchaku v1.2.0 offers a 20–30% performance boost for Z-Image models and supports INT4 on 20-series GPUs [3]. Pre-built wheels are available for Nunchaku on Linux and Windows platforms [4].

GPU Architecture Support

The ecosystem supports various GPU architectures through specific quantization formats: INT4 for non-Blackwell GPUs (pre-50-series) and NVFP4 for Blackwell GPUs (50-series) [6, 7]. Users can choose between rank configurations, such as r32 for faster inference or r128 for higher quality at the cost of speed [7]. Pre-built wheels are available for Linux and Windows, and models can be used across multiple platforms including Diffusers, ComfyUI, Google Colab, Kaggle, and local apps like Draw Things and DiffusionBee [4].

Nunchaku models support INT4 quantization for non-Blackwell GPUs (pre-50-series) and NVFP4 for Blackwell GPUs (50-series) [6,7,8]. This flexibility ensures that developers can utilize the latest hardware optimizations while maintaining compatibility with older systems.

Ready-to-Use Checkpoints

Several ready-to-use checkpoints have been released on Hugging Face. These include quantized versions of Stable Diffusion XL (SDXL), FLUX.1-dev, Qwen-Image, and Qwen-Image-Edit-2509 [5, 6, 7, 8]. The Qwen-Image models also offer lightning variants distilled to 4 or 8 steps using LoRA fusion for rapid generation [3, 7, 8]. Additionally, a companion toolkit called diffuse-compressor allows users to quantize new architectures and publish them as standard Diffusers repositories [1].

Ready-to-use Nunchaku checkpoints are available on Hugging Face for SDXL, FLUX.1-dev, Qwen-Image, and Qwen-Image-Edit-2509 [5,6,7,8]. Qwen-Image models offer lightning variants distilled to 4 or 8 steps using LoRA fusion for faster inference [3,7,8]. The diffuse-compressor toolkit allows users to quantize new architectures and publish them as regular Diffusers repositories [1].

Practical Implications for Developers

This integration marks a significant step toward democratizing high-end diffusion model inference. By embedding Nunchaku directly into the Diffusers library, Hugging Face has removed many of the technical barriers that previously prevented developers from running large models on consumer hardware [1]. Thai developers building image-generation apps can now leverage these optimizations to create more efficient and accessible applications without needing deep expertise in low-level GPU programming or custom inference engines.

The availability of pre-built wheels for Linux and Windows further simplifies deployment, ensuring that users can quickly set up their environments and start generating images with minimal friction [4]. As the ecosystem continues to grow, with new models and optimizations being added regularly, the potential for innovative applications in creative fields, education, and enterprise workflows expands significantly.

Nunchaku v1.2.0 introduced further improvements, including a 20–30% performance boost for Z-Image models and native LoRA support in ComfyUI [3]. This ongoing development cycle ensures that users benefit from the latest advancements in quantization technology, keeping their applications at the cutting edge of AI-generated content capabilities.

The integration of Nunchaku into Diffusers represents a major milestone in making powerful diffusion models accessible to a broader audience. By combining high-performance inference with ease of use, Hugging Face is helping to bridge the gap between advanced research and practical application, empowering developers worldwide to create more sophisticated and efficient AI-driven experiences.

Sources

  1. Bringing Nunchaku 4-bit Diffusion Inference to Diffusers (huggingface.co) — 2026-07-23
  2. Optimizing Diffusion Models with Nunchaku 4-bit Inference in Diffusers (explore.n1n.ai) — 2026-07-23
  3. GitHub - nunchaku-ai/nunchaku: [ICLR2025 Spotlight] SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models (github.com) — 2024-11-06
  4. nunchaku-ai/nunchaku · Hugging Face (huggingface.co) — 2025-07-17
  5. nunchaku-ai/nunchaku-qwen-image · Hugging Face (huggingface.co) — 2024-11-07

Frequently asked questions

How do I use Nunchaku with Hugging Face Diffusers? ▼
Nunchaku integrates directly into the Diffusers library, allowing you to load quantized checkpoints using standard methods like `from_pretrained()`. This eliminates the need for separate inference libraries or local CUDA compilation. You can leverage the Hugging Face kernels package to handle specialized operations without complex setup.
What is SVDQuant and how does it improve performance? ▼
Nunchaku uses SVDQuant, which runs transformer layers with both 4-bit weights and activations (W4A4 or W4A8). Unlike traditional weight-only quantization that dequantizes at compute time, this approach reduces memory usage and accelerates the denoising loop. It breaks through memory bandwidth bottlenecks common in large-scale diffusion models.
Which GPU architectures are supported by Nunchaku? ▼
The ecosystem supports INT4 quantization for non-Blackwell GPUs (pre-50-series) and NVFP4 for Blackwell GPUs (50-series). You can also choose between rank configurations, such as r32 for faster inference or r128 for higher quality. This flexibility ensures compatibility with both older systems and the latest hardware optimizations.
What ready-to-use Nunchaku checkpoints are available? ▼
Ready-to-use checkpoints are available on Hugging Face for models like SDXL, FLUX.1-dev, Qwen-Image, and Qwen-Image-Edit-2509. The Qwen-Image models also offer lightning variants distilled to 4 or 8 steps using LoRA fusion for rapid generation. These pre-built options allow for immediate use without custom quantization.
Can I run large diffusion models like Flux.1 on a consumer GPU? ▼
Nunchaku enables efficient inference on consumer-grade hardware like the RTX 4090 by significantly reducing VRAM requirements. For example, Flux.1-dev typically requires over 30GB of VRAM in BF16 precision, but Nunchaku makes it accessible on mid-range cards. This removes the barrier for running large models on standard consumer GPUs.