Update README.md

README.md

@@ -1,3 +1,156 @@
----
-license: cc-by-4.0
----
+---
+license: cc-by-4.0
+---
+
+# Charge: A Comprehensive Benchmark and Dataset for Dynamic Novel View Synthesis 
+
+This repository contains scripts for downloading [Charge](https://charge-benchmark.github.io/) dataset.
+
+## Updates
+
+- [ ] Full data release
+- [x] [12/12/2024] Release of the first scene in the dataset.
+
+## Download
+
+The only dependency of download scripts is the Huggingface Hub client which can be installed as:
+```
+pip install huggingface-hub
+```
+
+You can manually download desired scene, task, and modalities via:
+```
+python downloader.py --output-dir $1 --scenes [list-of-scenes] --modalities [rgb depth mask segmentation normal flow_fw flow_bw] --tasks [Dense Sparse Mono]
+```
+E.g. to download RGB images and dynamic masks for scene 050_0130 in the task of Sparse reconstruction (in the local folder), use:
+ ```
+python downloader.py --output-dir $1 --scenes 050_0130 --modalities rgb mask --tasks Sparse
+```
+
+For convenience, we provide bash scripts to download all data, or RGB data per task:
+- `download_all.sh`
+- `download_rgb_dense.sh`
+- `download_rgb_sparse.sh`
+- `download_rgb_mono.sh`
+
+Use them as following:
+```
+bash download_all.sh path-to-download-location
+```
+
+## Data structure
+
+The data is structured as following:
+```
+Charge_v1_0
+|-- 050_0130
+    |-- Dense
+        |-- Dense_00_00
+            |-- frame_0416.png
+            |-- frame_0416_depth.npy
+            |-- frame_0416_segmentation.png
+            |-- frame_0416_normal.png
+            |-- frame_0416_dyn_mask.png
+            |-- frame_0416_flow_fw.npy
+            |-- frame_0416_flow_bw.npy
+            |   ...
+        |   ...
+        |-- transforms_train.json
+        |-- transforms_test.json
+    |-- Sparse
+        |  ...
+    |-- Mono
+        |  ...
+    |-- segmentation.json
+| ..
+```
+
+The modalities included are as following:
+- `frame_XXXX.png` - RGB image (2048x858)
+- `frame_XXXX_depth.npy` - metric depth
+- `frame_XXXX_segmentation.png` - segmentation map encoded as uint16 image (objects dictionary included in `segmentation.json`)
+- `frame_XXXX_normal.png` - normal map encoded as uint16 image
+- `frame_XXXX_dyn_mask.png` - mask of dynamic content
+- `frame_XXXX_flow_fw.npy` - optical flow (forward)
+- `frame_XXXX_flow_bw.npy` - optical flow (backward)
+
+We include the camera data in the `.json` files found in task directories. In Sparse setup we include 3 scenarios (corresponding to 3, 6, 9 input views). In Mono scenario we include 4 different camera trajectories. For each, for convenience we include 3 testing scenarios: `_lite` - only rig cameras (stereo cameras with different baseline, and orbital camera), `_med` - rig cameras + 4 central cameras from Dense setup (static cameras), `_full` - rig cameras + all Dense cameras. We also include splits for Stereo training and evaluation. 
+
+Each transforms `.json` contains a dictionary:
+```
+{
+    "CameraName": [ #List of camera parameters per frame
+        {   
+            "fov" - field of view
+            "f" - focal length
+            "K" - intrinsics
+            "transformation_matrix" - extrinsics
+            "image_path" - corresponding image
+        },
+        {
+            # Camera data for second frame
+        },
+        ...
+    ], 
+    "CameraName2": ...,
+    ...
+}
+```
+
+## Download size
+
+The dataset is split into several repositories due to size. Below table summarises the size of data in GB. *-optical flow in one direction only (data include both forward and backward flow)
+
+<table>
+<tr>
+<th colspan="2" style="text-align: center; width:12%"> Scene </th> 
+<th style="text-align: right; width:11%"> RGB </th>
+<th style="text-align: right; width:11%"> Depth </th>
+<th style="text-align: right; width:11%"> Segm </th>
+<th style="text-align: right; width:11%"> Normal </th>
+<th style="text-align: right; width:11%"> Mask </th>
+<th style="text-align: right; width:11%"> Flow* </th>
+<th style="text-align: right; width:11%"> Total </th>
+</tr>
+<tr>
+<td rowspan="4" style="writing-mode: vertical-rl; transform: rotate(180deg); text-align: center;"> 050_0130 </td> <td> Dense </td>
+<td style="text-align: right"> 4.1 </td>
+<td style="text-align: right"> 25.2 </td>
+<td style="text-align: right"> 0.6 </td>
+<td style="text-align: right"> 14.5 </td>
+<td style="text-align: right"> 0.1 </td>
+<td style="text-align: right"> 50.3 </td>
+<td style="text-align: right"> 144.9 </td>
+</tr>
+<tr>
+<td> Sparse </td>
+<td style="text-align: right"> 2.0 </td>
+<td style="text-align: right"> 11.7 </td>
+<td style="text-align: right"> 0.3 </td>
+<td style="text-align: right"> 7.0 </td>
+<td style="text-align: right"> 0.1 </td>
+<td style="text-align: right"> 23.3 </td>
+<td style="text-align: right"> 67.5 </td>
+</tr>
+<tr>
+<td> Mono </td>
+<td style="text-align: right"> 2.0 </td>
+<td style="text-align: right"> 12.3 </td>
+<td style="text-align: right"> 0.3 </td>
+<td style="text-align: right"> 7.0 </td>
+<td style="text-align: right"> 0.1 </td>
+<td style="text-align: right"> 24.5 </td>
+<td style="text-align: right"> 70.6 </td>
+</tr>
+<tr>
+<td> Total </td>
+<td style="text-align: right"> 8.1 </td>
+<td style="text-align: right"> 49.1 </td>
+<td style="text-align: right"> 1.2 </td>
+<td style="text-align: right"> 28.4 </td>
+<td style="text-align: right"> 0.2 </td>
+<td style="text-align: right"> 98.1 </td>
+<td style="text-align: right"> 283.0 </td>
+</tr>
+
+</table>