📒Awesome-SD-Inference

📒A small curated list of Awesome SD/DiT/Diffusion Inference with Distributed/Caching/Sampling. For Awesome LLM Inference, please check 📖Awesome-LLM-Inference

🤖Contents

• 📙Awesome SD Inference with Sampling
• 📙Awesome SD Inference with Caching
• 📙Awesome SD Inference with Multi-GPUs
• 📙Other Awesome SD Inference Paper with codes

©️Citations

@misc{Awesome-SD-Inference@2024,
  title={Awesome-SD-Inference: A small curated list of Awesome SD/DiT/ViT/Diffusion Inference with Distributed/Caching/Sampling.},
  url={https://github.com/DefTruth/Awesome-SD-Inference},
  note={Open-source software available at https://github.com/DefTruth/Awesome-SD-Inference},
  author={DefTruth},
  year={2024}
}

📙Awesome SD Inference with Sampling

Date	Title	Paper	Code	Recom
2020.06	🔥[DDPM] Denoising Diffusion Probabilistic Models(@UC Berkeley)	[pdf]	[diffusion]	⭐️⭐️
2020.10	🔥[DDIM] DENOISING DIFFUSION IMPLICIT MODELS(@cs.stanford.edu)	[pdf]	⚠️	⭐️⭐️
2022.02	🔥[PNDM] PSEUDO NUMERICAL METHODS FOR DIFFUSION MODELS ON MANIFOLDS(@)	[pdf]	[PNDM]	⭐️⭐️
2022.02	🔥[DPM-Solver] DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps(@Cheng Lu)	[pdf]	[dpm-solver]	⭐️⭐️
2022.11	🔥[DPM-Solver++] DPM-SOLVER++: FAST SOLVER FOR GUIDED SAMPLING OF DIFFUSION PROBABILISTIC MODELS(@Cheng Lu)	[pdf]	[dpm-solver]	⭐️⭐️
2023.10	🔥[DPM-Solver-v3] DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model Statistics(@Kaiwen Zheng)	[pdf]	[DPM-Solver-v3]	⭐️⭐️
2023.11	🔥[Parallel Sampling] Parallel Sampling of Diffusion Models(@Stanford University)	[pdf]	[paradigms]	⭐️⭐️
2023.11	🔥[SAMPLER SCHEDULER] SAMPLER SCHEDULER FOR DIFFUSION MODELS(@sysu)	[pdf]	⚠️	⭐️⭐️
2024.02	🔥[Parallel Sampling] Accelerating Parallel Sampling of Diffusion Models(@Zhiwei Tang)	[pdf]	[ParaTAA-Diffusion]	⭐️⭐️
2024.01	🔥[YONOS] You Only Need One Step: Fast Super-Resolution with Stable Diffusion via Scale Distillation(@Samsung AI)	[pdf]	⚠️	⭐️⭐️
2024.01	🔥[S^2-DM] S^2-DMs: Skip-Step Diffusion Models(@Yixuan Wang)	[pdf]	⚠️	⭐️⭐️
2024.08	🔥[StepSaver] StepSaver: Predicting Minimum Denoising Steps for Diffusion Model Image Generation(@intel)	[pdf]	⚠️	⭐️⭐️
2024.09	🔥[DC-Solver] DC-Solver: Improving Predictor-Corrector Diffusion Sampler via Dynamic Compensation(@Tsinghua University)	[pdf]	[DC-Solver]	⭐️⭐️

📙Awesome SD Inference with Caching

• UNet Based (DeepCache)

• DiT Based (Fast-Forward Caching)

Date	Title	Paper	Code	Recom
2023.05	🔥🔥[Cache-Enabled Sparse Diffusion] Accelerating Text-to-Image Editing via Cache-Enabled Sparse Diffusion Inference(@pku.edu.cn etc)	[pdf]	⚠️	⭐️⭐️
2023.12	🔥🔥[DeepCache] DeepCache: Accelerating Diffusion Models for Free(@nus.edu)	[pdf]	[DeepCache]	⭐️⭐️
2023.12	🔥🔥[Block Caching] Cache Me if You Can: Accelerating Diffusion Models through Block Caching(@Meta GenAI etc)	[pdf]	⚠️	⭐️⭐️
2023.12	🔥🔥[Approximate Caching] Approximate Caching for Efficiently Serving Diffusion Models(@Adobe)	[pdf]	⚠️	⭐️⭐️
2024.06	🔥🔥[Layer Caching] Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching(@nus.edu)	[pdf]	[learning-to-cache]	⭐️⭐️
2024.07	🔥[ElasticCache-LVLM] Efficient Inference of Vision Instruction-Following Models with Elastic Cache(@Tsinghua University etc)	[pdf]	[ElasticCache]	⭐️
2024.07	🔥🔥[Fast-Forward Caching(DiT)] FORA: Fast-Forward Caching in Diffusion Transformer Acceleration(@microsoft.com etc)	[pdf]	[FORA]	⭐️⭐️
2024.07	🔥🔥[Faster I2V Generation] Faster Image2Video Generation: A Closer Look at CLIP Image Embedding’s Impact on Spatio-Temporal Cross-Attentions(@Ashkan Taghipour etc)	[pdf]	⚠️	⭐️⭐️
2024.04	🔥🔥[T-GATE V1] Cross-Attention Makes Inference Cumbersome in Text-to-Image Diffusion Models(@Wentian Zhang etc)	[pdf]	[T-GATE]	⭐️⭐️
2024.04	🔥🔥[T-GATE V2] Faster Diffusion via Temporal Attention Decomposition(@Haozhe Liu etc)	[pdf]	[T-GATE]	⭐️⭐️

📙Awesome SD Distributed Inference with Multi-GPUs

• UNet Based: Displaced Patch parallelism (DistriFusion)

• DiT Based: Displaced Patch parallelism (PipeFusion)

Date	Title	Paper	Code	Recom
2024.02	🔥🔥[DistriFusion] DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models(@MIT etc)	[pdf]	[distrifuser]	⭐️⭐️
2024.05	🔥🔥[PipeFusion] PipeFusion: Displaced Patch Pipeline Parallelism for Inference of Diffusion Transformer Models(@Tencent etc)	[pdf]	[xDiT]	⭐️⭐️
2024.06	🔥🔥[AsyncDiff] AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising(@nus.edu)	[pdf]	[AsyncDiff]	⭐️⭐️
2024.05	🔥🔥[TensorRT-LLM SDXL] SDXL Distributed Inference with TensorRT-LLM and synchronous comm(@Zars19)	[pdf]	[SDXL-TensorRT-LLM]	⭐️⭐️
2024.06	🔥🔥[Clip Parallelism] Video-Infinity: Distributed Long Video Generation(@nus.edu)	[pdf]	[Video-Infinity]	⭐️⭐️
2024.05	🔥🔥[FIFO-Diffusion] FIFO-Diffusion: Generating Infinite Videos from Text without Training(@Seoul National University)	[pdf]	[FIFO-Diffusion]	⭐️⭐️

📙Other Awesome SD Inference Paper with codes

Date	Title	Paper	Code	Recom
2024.06	🔥[DiTFastAttn] DiTFastAttn: Attention Compression for Diffusion Transformer Models(@Zhihang Yuan etc)	[pdf]	[DiTFastAttn]	⭐️⭐️
2024.08	🔥[Transfusion] Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model(@meta)	[pdf]	[transfusion-pytorch]	⭐️⭐️
2024.08	🔥[VQ4DiT] VQ4DiT: Efficient Post-Training Vector Quantization for Diffusion Transformers(@ZJU)	[pdf]	⚠️	⭐️⭐️
2024.08	🔥[LBQ] Low-Bitwidth Floating Point Quantization for Efficient High-Quality Diffusion Models(@toronto.edu)	[pdf]	⚠️	⭐️⭐️
2024.08	🔥[EE-Diffusion] A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models(@KAIST AI)	[pdf]	[ee-diffusion]	⭐️⭐️
2024.08	🔥[TFM-PTQ] Temporal Feature Matters: A Framework for Diffusion Model Quantization(@SenseTime)	[pdf]	⚠️	⭐️⭐️
2024.08	🔥[Diffusion-RWKV] Diffusion-RWKV: Scaling RWKV-Like Architectures for Diffusion Models(@Zhengcong Fei)	[pdf]	[Diffusion-RWKV]	⭐️⭐️
2024.09	🔥[LinFusion] LINFUSION: 1 GPU, 1 MINUTE, 16K IMAGE(@NUS)	[pdf]	[LinFusion]	⭐️⭐️

©️License

GNU General Public License v3.0

🎉Contribute

Welcome to star & submit a PR to this repo!