Computer Vision (UCB FA24 CS180) on itu

Facial Keypoint Detection by Regression & Pixelwise Classification

Wed, 04 Dec 2024 20:39:29 +0800

The project spec is archived here.

You can find the code here.

Part 1: Nose Tip Detection

For the first part, we use the IMM Face Database for training an initial toy model for nose tip detection. The dataset has 240 facial images of 40 persons and each person has 6 facial images in different viewpoints. All images are annotated with 58 facial keypoints. We use all 6 images of the first 32 persons (index 1-32) as the training set (total 32 x 6 = 192 images) and the images of the remaining 8 persons (index 33-40) (8 * 6 = 48 images) as the validation set.

[Flow] Rectified Flow Explained

Wed, 20 Nov 2024 20:39:29 +0800

Here is the (simple) explanation for the framework Rectified Flow.

A simple implementation

Overview

Rectified Flow (RF) is a generative modeling method, which tries to transport data from source distribution \(\pi_0\) (which corresponds to the pure Gaussian distribution \(\pi_0=N(0,I)\)) and the target distribution \(\pi_1\), which is the distribution of clean images.

The overall objective is to align the velocity estimate (using the UNet, denoted as \(v_\theta\) now) to the actual velocity between the source image \(X_0\) and the target image \(X_1\). First, the timesteps here are all normalized between \(t\in[0,1]\), instead of spreading in \(\{0,\cdots, T\}\).

DDPM Sampling, Applications & Training from Scratch

Sun, 10 Nov 2024 01:01:49 -0800

The original project spec at UC Berkeley CS180 is here.

Mirror of CS180 Project 5 Showcase Webpage.

Part A

0. Setup

2 Stages

We first use 3 prompts to let the model generate output images. Here are images and captions displayed below, with different inference steps:

5 steps (i.e. num_inference_steps=5):
- Size: 64px * 64px (Stage 1)
  
  an oil painting of
  a snowy mountain village
  
  a man wearing a hat
  
  a rocket ship
- Size: 256px * 256px (Stage 2)
  
  an oil painting of a snowy mountain village
  
  a man wearing a hat
  
  a rocket ship
20 steps:
- Stage 1:
  
  an oil painting of a
  snowy mountain village
  
  a man wearing a hat
  
  a rocket ship
- Stage 2:
  
  an oil painting of a
  snowy mountain village
  
  a man wearing a hat
  
  a rocket ship
100 steps:
- Stage 1:
  
  an oil painting of
  a snowy mountain village
  
  a man wearing a hat
  
  a rocket ship
- Stage 2:
  
  an oil painting of a
  snowy mountain village
  
  a man wearing a hat
  
  a rocket ship

Reflection on the generation

We find that for 5 steps, the outputs are not so clear, specifically, the noise added are not removed so completely. We can observe lots of noisy dots in the generated images. The generated feature is also not so clear.

CS180: Project Webpage Catalog

Fri, 06 Sep 2024 18:33:06 -0700

Project 1: Image Alignment using Multi-scale Pyramids
- Description
- Project Webpage
Project 2: Multi-resolution Blending with Laplacian/Gaussian Filters and Stacks
- Description
- Project Webpage
Project 3: Face Morphing
- Description
- Project Webpage
Project 4: Feature Recognition, Matching and Image Stitching
- Description
- Project Webpage
Project 5: Diffusion Models (DDPM, its principles and applications & Rectified Flow)
Final Projects: Collaborated with Junye Wang
- Project Webpage Aggregated
- 1/3: Facial Keypoint Detection