University of Michigan · Machine-learning course research · Jan–Sept 2021

Changing one feature in a generated face

ICA-SeFa: Interpretable Semantic Directions in GANs

Three generated StyleGAN2 faces show the same latent identity with a closed, partly open, and widely open mouth.
Figure 1 from the course report. The middle face is the starting image; the faces on either side show edits in opposite directions.Chen, Qin, Bi, Chen and Zhao · 2021 course report

This project explored how to change a feature such as hair color without changing several others. I helped implement ICA-SeFa, a method for finding these edits in a trained GAN, and ran its human evaluation.

My role

I designed and ran the human study, recruited annotators, analyzed the results, and helped implement the method and write the report.

Human annotators
8

Participants in the course study.

Directions rated per method
35

Manually selected from 200 discovered directions.

Methods compared
2

ICA-SeFa and the SeFa baseline.

Starting point

Checking the same edit across faces

An editing direction changes the inputs used to generate an image. I designed the human study to check whether each direction changed a recognizable feature consistently across different faces.

How it came together

  1. 01

    Find editing directions

    I helped implement a method that applies FastICA to a pretrained generator's weights. It builds on SeFa and finds editing directions without training another neural network.

  2. 02

    Compare the images

    We generated sequences of edits with StyleGAN2 trained on FFHQ, then compared similar features and checked whether other parts of the face changed too.

  3. 03

    Collect human ratings

    I recruited eight annotators, designed and ran the rating study, and analyzed their scores for the selected directions. The team also used CLIP to compare edits with named attributes.

Explore an editing direction

Compare the mouth edits

Select a position to view one of three original Figure 1 frames from the 2021 course report. This viewer displays the saved images without generating frames between them.

Original
Side-by-side ICA-SeFa and SeFa examples compare hair-color and glasses editing directions across five generated faces per strip.
Selected hair-color and glasses examples from Figure 4. Each strip follows one direction; the center image is the shared starting point.2021 team course report, Figure 4

Results for the selected directions

The selected ICA-SeFa examples showed clearer control over individual features, and its highest-rated directions received higher human scores than SeFa's. The study used a manually selected subset, so these results don't describe the average across all discovered directions.

The course experiment found 200 directions per method, manually selected 35 from each for human rating, and reported the ten highest-scoring directions. The mean opinion scores are averages of human ratings, rather than classification accuracy.

Dongyang Zhao implemented the CLIP analysis, which measured how the editing directions correlated with named attributes.

Reports, code, and credits

Course reportPresentation

Houming Chen proposed the idea, developed the theory and algorithm, and led implementation. I ran the human evaluation and helped with implementation. Yutong Bi and Jiaxi Chen contributed to the literature review and theory development.

The images come from our 2021 course report. The full course presentation is also linked below.

Next projectChecking XLNet's answers against its explanations