• Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric Regression

Aaron S. Jackson, Adrian Bulat, Vasileios Argyriou and Georgios Tzimiropoulos

Try out the code without running it! Check out our online demo [[http://www.cs.nott.ac.uk/~psxasj/3dme/][here]].

Please visit our [[http://aaronsplace.co.uk/papers/jackson2017recon/][project webpage]] for a link to the paper and an
example video run on 300VW. This code is licenses under the MIT
License, as described in the LICENSE file.

This is an unguided version of the Volumetric Regression Network (VRN)
for 3D face reconstruction from a single image. This method approaches
the problem of reconstruction as a segmentation problem, producing a
3D volume, spatially aligned with the input image. A mesh can then be
obtained by taking the isosurface of this volume.

Several example images are included in the examples folder. Most of
these are AFLW images taken from [[http://www.cbsr.ia.ac.cn/users/xiangyuzhu/projects/3DDFA/main.htm][3DDFA]].

If you are running the code to calculate error for a potential
publication, please use the MATLAB version, as this is what was used
to compute the error for the paper.

** Software Requirements

A working installation of Torch7 is required. This can be easily
installed on most platforms using [[https://github.com/torch/distro][torch/distro]]. You will also require
a reasonable CUDA capable GPU.

This project was developed under Linux. I have no idea if it will work
on Windows and it is unlikely that I will be able to help you with
this. If you are running Mac OS, [[https://github.com/AaronJackson/vrn/issues/1][issue #1]] might be of interest to you.

Quick overview of requirements:

• Torch7 (+ nn, cunn, cudnn, image). See "Installation Example" below.
• NVIDIA GPU, with a working CUDA (7.5 or 8.0) and CuDNN (5.1).
• Either,
  • MATLAB
  • bash, ImageMagick, GNU awk, Python 2.7 (+ visvis, imageio, numpy)

Please be wary of the version numbers for CUDA, CuDNN and Python.

Bulat's [[https://github.com/1adrianb/2D-and-3D-face-alignment/][face alignment]] code is included as a submodule. Please check
his README for dependencies.

** Getting Started

#+BEGIN_SRC bash
git clone --recursive https://github.com/AaronJackson/vrn.git
cd vrn
./download.sh
#+END_SRC

*** Running with MATLAB

MATLAB offers better functionality for taking the iso surface of the
volume. It also has some code to calculate per-vertex colouring on the
mesh. If you have MATLAB I recommend this route.

To run, type "run" from MATLAB.

*** Running with Python

No longer is MATLAB an absolute requirement! I've included a slightly
crazy (but don't worry, I had fun writing it) shell script which
performs the face normalisation, and runs the ~vis.py~ script to
render the regressed volume.

Unfortunately this does not yet apply any colouring or texture to the
mesh (you're welcome to contribute) and it has some issues if you
don't have a fully working OpenGL setup. Some GPUs won't like the
background image not being a power of two, so it might make the
results look odd. I'll work on this sometime.

To run it on the included example images without MATLAB, make the
~run.sh~ executable with ~chmod u+x run.sh~ and type ~./run.sh~ from
your terminal.

*** Using your own images

You are, of course, welcome to try out this method on your own set of
images. ~dlib~, the face detector included with Bulat's face alignment
code struggles to find side poses. You are welcome to modify the code
to provide bounding boxes.

*** Available Options

The MATLAB "run.m" script contains a few options which you can
change. Here is a very quick description of them:

• ~input_folder~, as the name suggests, the folder to glob for JPEG
  images.
• ~output_folder~, the directory to store the regressed volumes.
• ~model_file~, the name of the Torch model to load.
• ~gpunum~, specify which GPU to use, starting at 0.
• ~texture~, rudimentary texture mapping by taking the 2D projections
  nearest neighbour (MATLAB only).

• Installation Example

I've had a few requests to describe a little better how to configure
Torch so that everything works correctly. I've tested this on Fedora 24
and CentOS 7. I'm assuming it will also work on Ubuntu if you have the
correct development packages installed.

If you prefer docker, simply run docker build -t 'vrn' .. For an interactive shell, docker run -it vrn bash. Commands may require sudo.

#+BEGIN_SRC bash

Install some dependencies for later. I might have missed some

sudo yum install glog-devel boost-devel
pip install dlib matplotlib numpy visvis imageio

Install the Torch distribution.

mkdir -p $HOME/usr/{local,src}
cd $HOME/usr/local
git clone https://github.com/torch/distro.git
mv distro torch
cd torch
sudo ./install-deps
./install.sh
source $HOME/usr/local/torch/install/bin/torch-activate

Install THPP and fb.python for the face alignment code

cd $HOME/usr/src
git clone https://github.com/1adrianb/thpp.git
cd thpp/thpp
THPP_NOFB=1 ./build.sh

Install fb.python.

cd $HOME/usr/src
git clone https://github.com/facebook/fblualib.git
cd fblualib/fblualib/python
luarocks make rockspec/*

cd $HOME
git clone --recursive https://github.com/AaronJackson/vrn.git
cd vrn
./download.sh
./run.sh
#+END_SRC

• Paper Citation

#+BEGIN_SRC
@article{jackson2017vrn,
title={Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression},
author={Jackson, Aaron S and Bulat, Adrian and Argyriou, Vasileios and Tzimiropoulos, Georgios},
journal={International Conference on Computer Vision},
year={2017}
}
#+END_SRC