WeatherKITTI Dataset

Jiyuan Wang, Chunyu Lin, Lang Nie
Beijing jiaotong University
D4RD WeatherDepth WeatherDepth D4RD

WeatherKITTI is currently the most realistic all-weather simulated enhancement of the KITTI dataset. The WeatherKITTI dataset simulates the three weather conditions that most affect visual perception in real-world scenarios: rain, snow, and fog. Each type of weather has two intensity levels: severe and extremely severe. Together with clear weather, these two levels create a weather-enhanced dataset featuring three levels and seven weather scenarios.

Questions and Answer & Some Comparison

Q: Why not use diffusion models instead of the older GAN models?

A: The reason is that current diffusion models struggle with maintaining texture and object consistency. I initially tried using diffusion models for rendering WeatherKITTI, but I abandoned the approach because diffusion altered the scenes themselves. Although some papers have attempted this method and even presented it as a contribution, I respectfully disagree. The image below illustrates the difference between diffusion-based rendering and the GAN+PBR-based approach:

Diffusion-based Rendering

GAN-based Rendering

Download the Dataset

Depth Estimation: This dataset is currently rendered specifically for depth estimation tasks. For other tasks, you can try to reuse the data within the Eigen training, validation, and testing splits.

Test Split. If you want to:

  • Perform zero-shot testing to validate your model's robustness
  • Compare with existing WeatherKITTI and D4RD models
  • Evaluate the robustness of other models

You can opt to download only the test set, which is the standard Eigen split test set, consisting of 697 images with a total size of approximately 3GB:

 Test Dataset

Full Dataset. If you want to:

  • Reproduce WeatherKITTI, D4RD or train your model from scratch
  • Perform fine-tuning
  • Conduct other research

You can download the full dataset, which includes the Eigen training, validation, and test splits, with a total size of approximately 200GB:

 Full Dataset

BibTeX

@article{wang2023weatherdepth,
  title={Weatherdepth: Curriculum contrastive learning for self-supervised depth estimation under adverse weather conditions},
  author={Wang, Jiyuan and Lin, Chunyu and Nie, Lang and Huang, Shujun and Zhao, Yao and Pan, Xing and Ai, Rui},
  journal={arXiv preprint arXiv:2310.05556},
  year={2023}
}
@article{wang2024digging,
  title={Digging into contrastive learning for robust depth estimation with diffusion models},
  author={Wang, Jiyuan and Lin, Chunyu and Nie, Lang and Liao, Kang and Shao, Shuwei and Zhao, Yao},
  journal={arXiv preprint arXiv:2404.09831},
  year={2024}
}