Diffusion
- Forward, Backward (Learning), Sampling (Random)
- Continous Diffusion
- VAE, Denoising Autoencoder
- Markov Chains
- U-Net
- DALL-E (OpenAI), Stable Diffusion,
- Imagen, Muse, VEO (Google)
- LLaDa, Mercury Coder (Inception)
Non-equilibrium Thermodynamics
- Langevin dynamics
- Thermodynamic Equilibrium - Boltzmann Distribution
- Wiener Process - Multidimensional Brownian Motion
- Energy Based Models
Gaussian Noise
- Denoising
- Noise/ Variance Schedule
- Derivation by Reparameterization
Variational Inference
- Denoising Diffusion Probabilistic Model (DDPM)
- Noise Prediction Networks
- Denoising Diffusion Implicit Model (DDIM)
Loss Functions
- Variational Lower Bound (VLB)
- Evidence Lower Bound (ELBO)
- Kullback-Leibler divergence (KL divergence)
- Mean Squared Error (MSE)
Score Based Generative Model
- Annealing
- Noise conditional score network (NCSN)
- Equivalence: DDPM and Score BBased Generative Models
Conditional (Guided) Generation
- Classifier Guidance
- Classifier Free Guidance (CFG)
Latent Varible Generative Model
- Latent Diffusion Model (LDM)
- Lower Dimension (Latent) Space
References:
- https://en.wikipedia.org/wiki/Diffusion_model
- https://www.assemblyai.com/blog/diffusion-models-for-machine-learning-introduction
- https://www.ibm.com/think/topics/diffusion-models
- https://hackernoon.com/what-is-a-diffusion-llm-and-why-does-it-matter
- Large Language Diffusion Models (LLaDA): https://arxiv.org/abs/2502.09992
