GMPI (Generative Multiplane Images) is a multi-plane image generation framework that enables 2D GANs to have 3D perception capabilities.

The resulting output is called Generative Multiplanar Imagery (GMPI) and emphasizes that its rendering is not only of high quality but also guarantees a consistent view, which sets GMPI apart from many previous works. Importantly, the number of alpha maps can be dynamically adjusted and varied between training and inference, alleviating memory issues and enabling fast training of GMPI at 1024 2 resolution in less than half a day.

Project Page | paper

environment settings

This code was tested on Ubuntu 18.04 with CUDA 10.2.

conda env create -f environment.yml

Use pretrained checkpoints

download checkpoint

cd /path/to/this/repo

Please downloadpre-training checkpointand put them in${GMPI_ROOT}/ckpts

The structure should be:

+-- ckpts
|  +-- gmpi_pretrained
|  |  +-- FFHQ256
|  |  +-- FFHQ512
|  |  +-- FFHQ1024
|  |  +-- AFHQCat
|  |  +-- MetFaces

Use the following variables for illustration.

# This can be FFHQ256, FFHQ512, FFHQ1024, AFHQCat, or MetFaces
export OUTPUT_DIR=${GMPI_ROOT}/ckpts/gmpi_pretrained/${DATASET_NAME}

# Set this to your favourate seed
export SEED=589

# - When psi = 1.0 there is no truncation, which is used for quantitative results in the paper.
# - To obtain better qualitative results, use psi < 1.0.

render a single image

The following command renders the image${OUTPUT_DIR}/rendered.png,as well as:

  • mpi_alpha.png: alpha map for all planes,
  • mpi_rgb.png: All planes use the same RGB texture,
  • mpi_rgba.png: RGB-alpha image for all planes.
conda activate gmpi && \
export PYTHONPATH=${GMPI_ROOT}:${GMPI_ROOT}/gmpi/models:$PYTHONPATH && \
python ${GMPI_ROOT}/gmpi/eval/vis/ \
--ckpt_path ${OUTPUT_DIR}/generator.pth \
--output_dir ${OUTPUT_DIR} \
--seeds ${SEED} \
--nplanes 96 \
--truncation_psi ${TRUNCATION_PSI} \
--exp_config ${OUTPUT_DIR}/config.pth \
--render_single_image 1

Notice:nplanes = 96Used in papers to report quantitative and qualitative results, but GMPI is capable of producing high-quality results even with 32 planes.nplanesUse a lower value (such as 32) if you encounter CUDA out-of-memory errors.

#GMPI #Homepage #Documentation #Downloads #Multiplane #Image #Generation #Framework #News Fast Delivery

Leave a Comment

Your email address will not be published. Required fields are marked *