Project proposal (READ MORE)
Project Organisation: MIT Critical Data Colombia.
Here below find all the dataset's versions and descriptions. The dataset employed in this study spans 81 municipalities in Colombia from 2016 to 2018, resulting in a total of 12,636 satellite images. The metadata associated with these images encompasses static sociodemographic variables, indices of poverty, and access to education. Additionally, dynamic epidemiological and climatic metadata were collected, providing a comprehensive spatiotemporal context for the analyses.
Baseline method from satellite extractor: Raw data from Sentinel 2LC1 with recursive artifact removal, clouds removal based on LeastCC and Nearest Interpolation for spatial resolution.
SAT1_dataset_5_best_cities: Top 5 municipalities based on Baseline method from satellite extractor
SAT2_dataset_10_best_cities: Top 10 municipalities based on Baseline method from satellite extractor
RGB-Version: [Link]
SAT3_FULL_COLOMBIA: Top 81 municipalities based on Baseline method from satellite extractor
DATASET_81_CITIES_v1.0: [link]
DATASET_81_CITIES_v2.0: [link]
Creating Cloud-Cloudless Paired Dataset: This dataset, derived from imagery in five Colombian municipalities, consists of 1640 images (820 pairs), where each of the 164 images per municipality is paired with a previously identified optimal cloudless image. The Cloud2CloudlesDataset class organizes these pairs into a new folder (DATASET), with images renamed to indicate ground truth and cloud presence. The class, initialized with source and destination paths, includes tests for image count verification and folder existence confirmation.
Dataset on Rio de Janeiro, 2016-2018: [link] The datasets DATASET_rio_de_janeiro.zip and DATASET_rio_de_janeiro_forward_backwardv2.zip cover central Rio de Janeiro from 2016 to 2023, each containing 416 images per epidemiological week. DATASET_rio_de_janeiro.zip uses single-forward artifact removal, possibly leading to black images, while DATASET_rio_de_janeiro_forward_backwardv2.zip applies forward-backward artifact removal, replacing black images. Visit: https://github.com/sebasmos/satellite.extractor/tree/main/satellite_extractor/PART_1_satellite-augmentation for details.
Landsat Colombia 2008-2016: [Link]
MODIS 2 2008-2016: [link]
SAT4_dataset_10_best_cities_augmented_v1: Augmented data with aligned metadata. Data was extracted using recursive artifact removal, cloud removal based on LeastCC, and Nearest Interpolation for spatial resolution. Implemented here and augmentations applied to RGB channels while leaving other satellite channels unchanged:
SAT5_dataset_10_best_cities_augmented_v2: These images are improved to remove near black images method using a recursive forward-backward artefact removal algorithm with inter-band data augmentation on satellite imagery. Augmented data with aligned metadata. Improved version using Albumentation wrapper modules with extra augmented data. Data extracted using recursive artifact removal, cloud removal based on LeastCC, and Nearest Interpolation for spatial resolution. Implemented Notebook and augmentations applied to RGB channels while leaving other satellite channels unchanged.
The data can be read as (example):
from datasets import load_dataset
dataset = load_dataset("MITCriticalData/Unlabeled_top_10_cities_forward_backward_alg")
Alternatively, for datasets<1.11.0 the lecture of .tiff and .json files is not compatible. In such case we recommend to download the data as:
wget path_to_data/images.zip && unzip images.zip -d images/
wget path_to_data/annotations.zip && unzip annotations.zip -d annotations/
The dataset is released under the terms of MIT. By using this, you are also bound to the respective Terms of Use and License of the original source.
Sebastián Andrés Cajas Ordóñez
This work is supported in part by Oracle for Research through Oracle Cloud Credits and related resources provided by Oracle for Research, as well as the European Space Agency’s Network of Resources Initiative under the sponsoring request ID 1c081a, and MISTI-Colombia, Cali seed fund. LAC is funded by the National Institute of Health through R01 EB017205, DS-I Africa U54 TW012043-01 and Bridge2AI OT2OD032701, and the National Science Foundation through ITEST 2148451.
MIT Critical data Colombia Team: Sebastian A. Cajas, David Restrepo, Kuan-Ting Kuo, Dana Moukheiber, Atika Rahman Paddo, Saptarshi Purkayastha, Leo Anthony Celi, Po-Chih Kuo, Juan Sebastián Osorio-Valencia, Kuan-Ting Ku, Braiam Escobar, Diego M. López, Cheng Che Tsai, Wilson Arbey Diaz, Luis Jesús Martínez, Alessa Álvarez, Siyi Tang, Amara Tariq, Imon Banerjee, Aakanksha Rana, Maria Patricia Arbelaez-Montoy, Cheng Che Tsai, Laura Sofía Daza Rosero, Jhon Fredy Romero Núñez, Wilson Arbey Diaz, Luis Jesús Martínez, Saketh Sundar, Alessa Álvarez, Siyi Tang, Amara Tariq, Imon Banerjee, Aakanksha Rana, Ivan Darío Velez, Maria Patricia Arbelaez-Montoya.
Please cite our work if you find the resources in this repository useful:
@article{cajasmulti,
title={A Multi-Modal Satellite Imagery Dataset for Public Health Analysis in Colombia},
author={Cajas, Sebastian A and Restrepo, David and Moukheiber, Dana and Kuo, Kuan Ting and Wu, Chenwei and Chicangana, David Santiago Garcia and Paddo, Atika Rahman and Moukheiber, Mira and Moukheiber, Lama and Moukheiber, Sulaiman and others}
}
@article{kuo2024denguenet,
title={DengueNet: Dengue Prediction using Spatiotemporal Satellite Imagery for Resource-Limited Countries},
author={Kuo, Kuan-Ting and Moukheiber, Dana and Ordonez, Sebastian Cajas and Restrepo, David and Paddo, Atika Rahman and Chen, Tsung-Yu and Moukheiber, Lama and Moukheiber, Mira and Moukheiber, Sulaiman and Purkayastha, Saptarshi and others},
journal={arXiv preprint arXiv:2401.11114},
year={2024}
}
@article{moukheiber2024multimodal,
title={A multimodal framework for extraction and fusion of satellite images and public health data},
author={Moukheiber, Dana and Restrepo, David and Cajas, Sebasti{\'a}n Andr{\'e}s and Montoya, Mar{\'\i}a Patricia Arbel{\'a}ez and Celi, Leo Anthony and Kuo, Kuan-Ting and L{\'o}pez, Diego M and Moukheiber, Lama and Moukheiber, Mira and Moukheiber, Sulaiman and others},
journal={Scientific Data},
volume={11},
number={1},
pages={634},
year={2024},
publisher={Nature Publishing Group UK London}
}