Diffusion on itu

MDP & Timestep RL for Discrete Diffusion LLMs

Wed, 30 Sep 2026 15:44:00 +0900

Semi-AR (Block AR); Agentic multi turn; Only terminal/outcome reward, \(\gamma=1\) as the convention for LLM RL.

Setup

Index

Some canvas with idx \(c\) (length \(C\)) includes denoising timestep index (not token index) \([n_c, n_c+T_c)\), \(T_c\) is not constant over \(c\) by early-stopping. Global denoising step \(n\).

From \(n\), a tuple of indices can be specified

\[ n\mapsto (j(n),c(n),t(n)) \]

\(j(n)\): (global) turn index
\(c(n)\): global canvas index
\(t(n)\): timestep index in the canvas

thus

\[ n=\sum_{c< c(n)}T_c+t(n). \]

State

For some timestep \(n\), define its realized state \(s_n\) (and r.v. \(S\))

[Flow] Rectified Flow Explained

Wed, 20 Nov 2024 20:39:29 +0800

Here is the (simple) explanation for the framework Rectified Flow.

A simple implementation

Overview

Rectified Flow (RF) is a generative modeling method, which tries to transport data from source distribution \(\pi_0\) (which corresponds to the pure Gaussian distribution \(\pi_0=N(0,I)\)) and the target distribution \(\pi_1\), which is the distribution of clean images.

The overall objective is to align the velocity estimate (using the UNet, denoted as \(v_\theta\) now) to the actual velocity between the source image \(X_0\) and the target image \(X_1\). First, the timesteps here are all normalized between \(t\in[0,1]\), instead of spreading in \(\{0,\cdots, T\}\).

DDPM Sampling, Applications & Training from Scratch

Sun, 10 Nov 2024 01:01:49 -0800

The original project spec at UC Berkeley CS180 is here.

Mirror of CS180 Project 5 Showcase Webpage.

Part A

0. Setup

2 Stages

We first use 3 prompts to let the model generate output images. Here are images and captions displayed below, with different inference steps:

5 steps (i.e. num_inference_steps=5):
- Size: 64px * 64px (Stage 1)
  
  an oil painting of
  a snowy mountain village
  
  a man wearing a hat
  
  a rocket ship
- Size: 256px * 256px (Stage 2)
  
  an oil painting of a snowy mountain village
  
  a man wearing a hat
  
  a rocket ship
20 steps:
- Stage 1:
  
  an oil painting of a
  snowy mountain village
  
  a man wearing a hat
  
  a rocket ship
- Stage 2:
  
  an oil painting of a
  snowy mountain village
  
  a man wearing a hat
  
  a rocket ship
100 steps:
- Stage 1:
  
  an oil painting of
  a snowy mountain village
  
  a man wearing a hat
  
  a rocket ship
- Stage 2:
  
  an oil painting of a
  snowy mountain village
  
  a man wearing a hat
  
  a rocket ship

Reflection on the generation

We find that for 5 steps, the outputs are not so clear, specifically, the noise added are not removed so completely. We can observe lots of noisy dots in the generated images. The generated feature is also not so clear.