Neural Haircut: Prior-Guided Strand-Based Hair Reconstruction

We propose an approach that can accurately reconstruct hair geometry at a strand level from a monocular video or multi-view images captured in uncontrolled lighting conditions. Our method has two stages, with the first stage performing joint reconstruction of coarse hair and bust shapes and hair orientation using implicit volumetric representations. The second stage then estimates a strand-level hair reconstruction by reconciling in a single optimization process the coarse volumetric constraints with hair strand and hairstyle priors learned from the synthetic data. To further increase the reconstruction fidelity, we incorporate image-based losses into the fitting process using a new differentiable renderer. The combined system, named Neural Haircut, achieves high realism and personalization of the reconstructed hairstyles.

We reconstruct the strand-based hair geometry in two stages. First, we obtain a coarse volumetric hair reconstruction in the form of implicit fields. Our second strand-based reconstruction stage is described below.

We reconstruct hair strands using geometry texture. At each iteration, we sample a set of random embeddings from the texture and obtain corresponding hair strands using a pre-trained on synthetic data strand parametric model. These strands are supervised using geometric and rendering-based constraints.

The geometric loss $\mathcal{L}_\text{geom}$ makes sure that hair strands do not deviate from the volumetric reconstruction and have proper orientations.

The silhouette-based and neural rendering losses $\mathcal{L}_\text{render}$ utilize our proposed soft hair rasterization technique based on hair quads with learnable strand-based appearance texture to predict the silhouette and the RGB image.

Lastly, $\mathcal{L}_\text{prior}$ acts as a regularization penalty that improves the physical plausibility of obtained strands using a pre-trained on synthetic hairstyles diffusion model.