Update README.m with feedback

README.md

@@ -3,7 +3,8 @@ license: apache-2.0
 library_name: superpoint-pruned
 tags:
 - SuperPoint
-- video
+- vision
+- image-matching
 - pruned
 - pytorch
 - onnx
@@ -38,11 +39,11 @@ pipeline_tag: keypoint-detection
   Optimized from SuperPoint
 </h3>
 
-
 </div>
 
 **SuperPoint Pruned is an optimization of the SuperPoint (SP) keypoint detection model. Based on [rpautrat/SuperPoint](https://github.com/rpautrat/SuperPoint) by Rémi Pautrat and Paul-Edouard Sarlin. Optimized for the TensorRT runtime on Jetson Orin Nano devices, but also applicable to other runtime backends. The ideas used can be applied to other similar network architectures.**
 
+This work was done in collaboration with CEA ([Aidge](https://eclipse.dev/aidge/)) for the [DeepGreen](https://deepgreen.ai/) project. You can also find a pruning tutorial for SuperPoint using Aidge operators in the [Aidge codebase](https://gitlab.eclipse.org/eclipse/aidge/aidge/-/tree/main/examples/tutorials/SuperPoint_pruning_tutorial).
 
 ## Example
 
@@ -121,115 +122,126 @@ Although there are relatively few exact overlaps, many predictions lie very clos
 
 </details>
 
-
 ## Quickstart
 
-For installing the package with dependencies for running the basic distillation, evaluation, and export scripts, run:
+Install the required dependencies and the package in editable mode:
+
+```bash
+pip install -r requirements.txt
+pip install -e .
+```
+
+Download the default dataset, **UZH FPV Indoor forward-facing trace #3**, to `src/superpoint_pruning/distillation/data`:
+
+```bash
+superpoint-pruning setup
 ```
+
+Compare the keypoint predictions from the pruned model against the original SuperPoint model on an image:
+
+```bash
+superpoint-pruning plot \
+    --image-name image_1_1200.png \
+    --output ./test.png \
+    --num-keypoints 512 \
+    --pruning-config ./src/superpoint_pruning/weights/16_16_24_32_64.ckpt \
+    --overlay
+```
+
+The resulting comparison is written to `./test.png`. To run the distillation and evaluation scripts, see the [Annex: Run your own distillation and compare inference time](#annex-run-your-own-distillation-and-compare-inference-time) section.
+
+## Limitations
+
+- Other popular SuperPoint implementations exist, especially versions paired with matchers such as LightGlue. This repository does not use them because of their restrictive licenses. The work is meant to show that SuperPoint-like models can be pruned and distilled. You can apply the same methods to those implementations after the necessary structural changes and a distillation run with your configs.
+- The right way to judge a pruned SuperPoint is to test it in the intended downstream task and see if it still retains performance. We don't explore it here since we are not currently looking at a specific task.
+- Distillation used 250 frames from [UZH-FPV Indoor forward facing trace #3](http://rpg.ifi.uzh.ch/datasets/uzh-fpv-newer-versions/v3/indoor_forward_3_snapdragon_with_gt.zip) (767 eval images). Direct training images were excluded from evaluation, but results may still be biased because the frames are sequential video frames. Outdoor forward-facing trace #1 was not used for recovery training.
+- Skipping NMS refinement usually still produces sensible keypoints, but that depends on the task.
+
+## License
+
+This repository is licensed under the **Apache License 2.0**.
+
+### Third-party Attributions
+
+- **SuperPoint implementation and weights** — Rémi Pautrat and Paul-Edouard Sarlin, [rpautrat/SuperPoint](https://github.com/rpautrat/SuperPoint).
+- **SuperPoint method** — Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich, *SuperPoint: Self-Supervised Interest Point Detection and Description*, CVPR Workshops, 2018.
+- **UZH-FPV dataset** — Jeffrey Delmerico, Titus Cieslewski, Henri Rebecq, Matthias Faessler, and Davide Scaramuzza, *Are We Ready for Autonomous Drone Racing? The UZH-FPV Drone Racing Dataset*, ICRA, 2019. [Dataset](https://fpv.ifi.uzh.ch/)
+
+## What's next?
+
+- Experiment with SuperPoint Pruned for computer vision, drone, and edge-device workloads
+- Compress your own models with [Pruna](https://github.com/PrunaAI/pruna) and give us a ⭐️
+
+## Annex: Run your own distillation and compare inference time
+
+For installing the package with dependencies for running the basic distillation, evaluation, and export scripts, run:
+
+```bash
 pip install -r requirements.txt
 pip install -e .
 ```
+
 To install the base dataset images (UZH FPV Indoor forward facing trace #3) used for training to the default path `src/superpoint_pruning/distillation/data`, run:
-```
+
+```bash
 superpoint-pruning setup
 ```
+
 Alternatively, specify the desired trace download link with the `--download-url` flag, e.g., the one for the UZH FPV Outdoor forward facing trace #1, and the local dataset directory name with `--output-dir-name`.
 
-**Run distillation training**
+### Run distillation training
+
 The base PyTorch Lightning training config that we used to produce the `pruned` model can be found under `src/superpoint_pruning/distillation/base_config.yaml`.
 Since this includes the hyperparameters and the data setup used for our training, then this should allow to reproduce the trained checkpoint that is provided under `src/superpoint_pruning/weights/16_16_24_32_64.ckpt`. The training script itself is very basic but can serve as a starting point for further experiments.
 To replicate the training, assuming that the main Indoor data trace is installed in the previous, run:
-```
+
+```bash
 superpoint-pruning setup --generate-gt
 superpoint-pruning train
 ```
+
 The first command computes and presaves the target keypoint and descriptor feature maps and the other initiate the Lightning training script.
 
-**Run evaluation**
+### Run evaluation
+
 The first row of quality evaluation results can be reproduced using the following commands corresponding to the different target models:
-```
+
+```bash
 superpoint-pruning evaluate
 superpoint-pruning evaluate --hierarchical
 superpoint-pruning evaluate --backbone_0_1 32 --hierarchical
 superpoint-pruning evaluate --pruning-config ./src/superpoint_pruning/weights/16_16_24_32_64.ckpt --hierarchical
 superpoint-pruning evaluate --pruning-config ./src/superpoint_pruning/weights/16_16_24_32_64.ckpt --hierarchical --skip-refinement
 ```
+
 Note that in this script and other similar ones, we can define the pruning either with format `--backbone_x_y new_input_channel_size` or by pointing it to a ckpt file named in the format `a_b_...ckpt`, which is translated to `--backbone_0_1 a`, `--backbone_1_0 b` and so on.
 For the evaluation using the outdoor trace (or some other trace), the `--image-dir` parameter has to be specified including the other specific options to use the same image range (`--no-skip --start-idx 1000 --end-idx 1500`).
 
-**Plot keypoint comparisons**
+### Plot keypoint comparisons
 
-For a quick keypoint prediction comparsions plot between a target model and the original on a specific image, one can for example use:
-```
+For a quick keypoint prediction comparison plot between a target model and the original on a specific image, one can for example use:
+
+```bash
 superpoint-pruning plot --image-name image_1_1200.png --output ./test.png --num-keypoints 512 --pruning-config ./src/superpoint_pruning/weights/16_16_24_32_64.ckpt --overlay
 ```
+
 This once again defaults to the UZH FPV Indoor forward facing trace #3 and accepts file names from the dataset.
 
-**Export model to ONNX**
+### Export model to ONNX
+
 To export the desired PyTorch model to ONNX:
-```
+
+```bash
 superpoint-pruning export --backbone_0_1 32
 ```
+
 The model pruning configuration can once again be controlled with same flags as shown in the evaluation example.
 As an example, we have published the ONNX export of the `original` model to https://huggingface.co/EclipseAidge/SuperPoint.
 
-**Run TRT inference benchmark**
+### Run TRT inference benchmark
+
 In case the device supports TensorRT (with the necessary packages installed) and there is a serialized TRT model (here a dummy example of `superpoint.engine`), then the local python benchmark with a TRT wrapper can be run as:
-```
+
+```bash
 superpoint-pruning benchmark --model-path superpoint.engine
 ```
-
-## Limitations
-
-- Other popular SuperPoint implementations exist, especially versions paired with matchers such as LightGlue. This repository does not use them because of their restrictive licenses. The work is meant to show that SuperPoint-like models can be pruned and distilled. You can apply the same methods to those implementations after the necessary structural changes and a distillation run with your configs.
-- The right way to judge a pruned SuperPoint is to test it in the intended downstream task and see if it still retains performance. We don't explore it here since we are not currently looking at a specific task.
-- Distillation used 250 frames from [UZH-FPV Indoor forward facing trace #3](http://rpg.ifi.uzh.ch/datasets/uzh-fpv-newer-versions/v3/indoor_forward_3_snapdragon_with_gt.zip) (767 eval images). Direct training images were excluded from evaluation, but results may still be biased because the frames are sequential video frames. Outdoor forward-facing trace #1 was not used for recovery training.
-- Skipping NMS refinement usually still produces sensible keypoints, but that depends on the task.
-
-## License
-
-This repository is licensed under the **Apache License 2.0**.
-
-### Third-party Attributions
-
-- **SuperPoint implementation and weights** — Rémi Pautrat and Paul-Edouard Sarlin, [rpautrat/SuperPoint](https://github.com/rpautrat/SuperPoint).
-- **SuperPoint method** — Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich, *SuperPoint: Self-Supervised Interest Point Detection and Description*, CVPR Workshops, 2018.
-- **UZH-FPV dataset** — Jeffrey Delmerico, Titus Cieslewski, Henri Rebecq, Matthias Faessler, and Davide Scaramuzza, *Are We Ready for Autonomous Drone Racing? The UZH-FPV Drone Racing Dataset*, ICRA, 2019. [Dataset](https://fpv.ifi.uzh.ch/)
-
-## What's next?
-
-- **Experiment with Superpoint Pruned for more efficiency**
-- Compress your own models with [Pruna](https://github.com/PrunaAI/pruna) and give us a ⭐️ for more efficiency!
-- Want to use our optimized video model right away? Check
-  [P-Video-2](https://www.pruna.ai/p-video-2) and
-  [P-Video-2 documentation](https://docs.pruna.ai/en/stable/docs_pruna_endpoints/performance_models/p-video-2.html).
-
-<style>
-.model-button {
-  display: inline-flex;
-  flex-direction: row;
-  justify-content: center;
-  align-items: center;
-  gap: 8px;
-
-  padding: 8px 20px;
-  border: none;
-  border-radius: 8px;
-
-  background: #9334e9;
-  color: #ffffff;
-
-  font-size: 14px;
-  font-weight: 400;
-  line-height: 1;
-  text-decoration: none;
-  white-space: nowrap;
-  cursor: pointer;
-
-  box-sizing: border-box;
-  overflow: visible;
-  opacity: 1;
-}
-</style>
-
-<a href="https://dashboard.pruna.ai/login?utm_source=huggingface&utm_medium=model_card&utm_campaign=hf_traffic" class="model-button">Try our models</a>