Project | Demo | Paper | Video
GAN Dissection is a way to inspect the internal representations of a generative adversarial network (GAN) to understand how internal units align with human-interpretable concepts. It is part of NetDissect.
This repo allows you to dissect a GAN model. It provides the dissection results as a static summary or as an interactive visualization. Try our interactive GANPaint demo to interact with GANs and draw images.
Overview
Visualizing and Understanding Generative Adversarial Networks
David Bau, Jun-Yan Zhu, Hendrik Strobelt, Bolei Zhou, Joshua B. Tenenbaum, William T. Freeman, Antonio Torralba
MIT CSAIL, MIT-IBM Watson AI Lab, CUHK, IBM Research
In arXiv, 2018.
Analysis and Applications
Interpretable Units in GANs
Analyzing different layers
Diagnosing and improving GANs
Removing objects from conference rooms
Removing windows from different natural scenes
Inserting new objects into images
Release history
v 0.9 alpha - Nov 26, 2018
Getting Started
Let's set up the environment and dissect a churchoutdoor GAN. This requires some CUDA-enabled GPU and some disk space.
Setup
To install everything needed from this repo, have conda available,
and run:
script/setup_env.sh # Create a conda environment with dependencies
script/make_dirs.sh # Create the dataset and dissect directories
script/download_data.sh # Download support data and demo GANs
source activate netd # Enter the conda environment
pip install -v -e . # Link the local netdissect package into the env
Details. The code depends on python 3, Pytorch 4.1, and several other
packages. For conda users, script/environment.yml provides the details
of the dependencies. For pip users, setup.py lists everything needed.
Data. The download_data.sh script downloads the segmentation dataset
used to dissect classifiers, the segmentation network used to dissect GANs,
and several example GAN models to dissect. The downloads will go into
the directories dataset/ and models/. If you do not wish to download
the example networks, python -m netdissect --download will download
just the data and models needed for netdissect itself.
Dissecting a GAN
GAN example: to dissect three layers of the LSUN living room progressive GAN trained by Karras:
python -m netdissect \
--gan \
--model "netdissect.proggan.from_pth_file('models/karras/livingroom_ lsun.pth')" \
--outdir "dissect/livingroom" \
--layer layer1 layer4 layer7 \
--size 1000
