SoftRas 是目前支流三角网格可微渲染器之一。

可微渲染通过计算渲染过程的导数，使得从单张图片学习三维构造逐步成为事实。可微渲染目前被宽泛地利用于三维重建，特地是人体重建、人脸重建和三维属性预计等利用中。

装置

conda 装置 PyTorch 环境：

conda create -n torch python=3.8 -yconda activate torchconda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c nvidia -yconda activate torchpython - <<-EOFimport platformimport torchprint(f"Python : {platform.python_version()}")print(f"PyTorch: {torch.__version__}")print(f"  CUDA : {torch.version.cuda}")EOFPython : 3.8.10PyTorch: 1.9.0  CUDA : 11.1

获取代码并装置：

git clone https://github.com/ShichenLiu/SoftRas.gitcd SoftRaspython setup.py install

可设 setup.py 镜像源：

cat <<-EOF > ~/.pydistutils.cfg[easy_install]index_url = http://mirrors.aliyun.com/pypi/simpleEOF

利用

装置模型查看工具：

snap install ogre-meshviewer# 或snap install meshlab

渲染物体

渲染测试：

CUDA_VISIBLE_DEVICES=0 python examples/demo_render.py

渲染后果：

比照前后模型:

ogre-meshviewer data/obj/spot/spot_triangulated.objogre-meshviewer data/results/output_render/saved_spot.obj

Mesh 重建

下载数据集：

bash examples/recon/download_dataset.sh

训练模型：

$ CUDA_VISIBLE_DEVICES=0 python examples/recon/train.py -eid reconLoading dataset: 100%|██████████████████████████| 13/13 [00:35<00:00,  2.74s/it]Iter: [0/250000]    Time 1.189    Loss 0.655    lr 0.000100    sv 0.000100Iter: [100/250000]    Time 0.464    Loss 0.405    lr 0.000100    sv 0.000100...Iter: [250000/250000]    Time 0.450    Loss 0.128    lr 0.000030    sv 0.000030

测试模型：

$ CUDA_VISIBLE_DEVICES=0 python examples/recon/test.py -eid recon \    -d 'data/results/models/recon/checkpoint_0250000.pth.tar'Loading dataset: 100%|██████████████████████████| 13/13 [00:03<00:00,  3.25it/s]Iter: [0/97]    Time 0.419      IoU 0.697=================================Mean IoU: 65.586 for class AirplaneIter: [0/43]    Time 0.095      IoU 0.587=================================Mean IoU: 49.798 for class BenchIter: [0/37]    Time 0.089      IoU 0.621=================================Mean IoU: 68.975 for class CabinetIter: [0/179]   Time 0.088      IoU 0.741Iter: [100/179] Time 0.083      IoU 0.772=================================Mean IoU: 74.224 for class CarIter: [0/162]   Time 0.086      IoU 0.565Iter: [100/162] Time 0.085      IoU 0.522=================================Mean IoU: 52.933 for class ChairIter: [0/26]    Time 0.094      IoU 0.681=================================Mean IoU: 60.553 for class DisplayIter: [0/55]    Time 0.087      IoU 0.526=================================Mean IoU: 45.751 for class LampIter: [0/38]    Time 0.086      IoU 0.580=================================Mean IoU: 65.626 for class LoudspeakerIter: [0/56]    Time 0.090      IoU 0.783=================================Mean IoU: 68.683 for class RifleIter: [0/76]    Time 0.092      IoU 0.647=================================Mean IoU: 68.111 for class SofaIter: [0/204]   Time 0.090      IoU 0.405Iter: [100/204] Time 0.087      IoU 0.435Iter: [200/204] Time 0.086      IoU 0.567=================================Mean IoU: 46.206 for class TableIter: [0/25]    Time 0.097      IoU 0.901=================================Mean IoU: 82.261 for class TelephoneIter: [0/46]    Time 0.087      IoU 0.503=================================Mean IoU: 61.019 for class Watercraft=================================Mean IoU: 62.287 for all classes

Mesh 重建：

# 获取 `softras_recon.py` 进 `examples/recon/`#   https://github.com/ikuokuo/start-3d-recon/blob/master/samples/softras_recon.py# 正文 `iou` 间接返回 0，位于 `examples/recon/models.py` `evaluate_iou()`# 2D 图像重构 3D MeshCUDA_VISIBLE_DEVICES=0 python examples/recon/softras_recon.py \    -s '.' \    -d 'data/results/models/recon/checkpoint_0250000.pth.tar' \    -img 'data/car_64x64.png'ogre-meshviewer data/car_64x64.obj

重建图像：

重建后果：

或重建 ShapeNet 数据集内图像：

# mesh recon images of ShapeNet datasetCUDA_VISIBLE_DEVICES=0 python examples/recon/softras_recon.py \    -s '.' \    -d 'data/results/models/recon/checkpoint_0250000.pth.tar' \    -imgs 'data/datasets/02958343_test_images.npz'

或应用 SoftRas 训练好的模型：

  • SoftRas trained with silhouettes supervision (62+ IoU): google drive
  • SoftRas trained with shading supervision (64+ IoU, test with --shading-model arg): google drive
  • SoftRas reconstructed meshes with color (random sampled): google drive

更多

  • Jrender - Jittor

    • JRender 解读
    • 3D Recon
GoCoding 集体实际的教训分享，可关注公众号！