Adaptive Large-scale Novel View Image Synthesis for Autonomous Driving Datasets 面向自动驾驶数据集的自适应大规模新视角图像合成
Abstract 摘要
Novel view image synthesis for outdoor scenes is affected by inaccurate depth measurements, moving objects, and wide-angle rendering. This paper proposes an adaptive novel-view image synthesis pipeline for large-scale traffic scenes. The pipeline includes three components: 1) 3D surfel-model reconstruction methods with depth refinement and moving-object removal; 2) a self-adaptive rendering scheme for different novel views through surfel-geometry adjustment; and 3) a hyper-parameter tuning scheme based on image-quality evaluation for surfel-model construction and adaptation. Removed backgrounds and other occluded regions within the 3D scene geometry are further inpainted using a Generative Adversarial Network (GAN). The KITTI dataset and CARLA simulator are used for evaluation. Experimental results compare the proposed pipeline with existing approaches in computational efficiency and synthesized-image quality.
面向室外场景的新视角图像合成受到深度测量不准确、动态物体干扰以及广角渲染失真等问题的影响。本文提出一种自适应新视角图像合成流程，用于大规模交通场景。该流程包括三个部分：1）结合深度优化与动态目标去除的三维 surfel 模型重建方法；2）面向不同目标视角的自适应渲染机制，通过调整 surfel 几何属性来适配新视角；3）基于图像质量评估的超参数调优流程，用于 surfel 模型构建与自适应。对于三维几何模型中因遮挡和动态目标移除而缺失的背景区域，进一步使用生成对抗网络（GAN）进行修复。本文在 KITTI 数据集和 CARLA 仿真平台上进行评估，并从计算效率和合成图像质量两个方面与现有方法进行比较。