| --- |
| license: mit |
| --- |
| |
| # DOLG in torch and tensorflow (TF2) |
|
|
| Re-implementation (Non Official) of the paper DOLG: Single-Stage Image Retrieval with Deep Orthogonal Fusion of Local and Global Features accepted at ICCV 2021. |
| [paper](https://arxiv.org/pdf/2108.02927.pdf) |
|
|
| The pytorch checkpoint has been converted into tensorflow format (.h5) from this repository : https://github.com/feymanpriv/DOLG (Official) |
|
|
|
|
|
|
| ## Installation |
|
|
| > pip install opencv-python==4.5.5.64 |
|
|
| > pip install huggingface-hub |
|
|
| to install dolg : |
|
|
| > pip install dolg |
| OR |
| > pip install -e . |
|
|
| ## Inference |
|
|
| To do some inference on single sample, you can use python script in examples/ folder or use as follows: |
|
|
| ``` |
| import dolg |
| import numpy as np |
| from dolg.utils.extraction import process_data |
| |
| depth = 50 |
| |
| # for pytorch |
| |
| import torch |
| from dolg.dolg_model_pt import DOLG |
| from dolg.resnet_pt import ResNet |
| |
| backbone = ResNet(depth=depth, num_groups=1, width_per_group=64, bn_eps=1e-5, |
| bn_mom=0.1, trans_fun="bottleneck_transform") |
| model = DOLG(backbone, s4_dim=2048, s3_dim=1024, s2_dim=512, head_reduction_dim=512, |
| with_ma=False, num_classes=None, pretrained=f"r{depth}") |
| img = process_data("image.jpg", "", mode="pt").unsqueeze(0) |
| |
| with torch.no_grad(): |
| output = model(img) |
| print(output) |
| |
| # for tensorflow |
| |
| import tensorflow as tf |
| from dolg.dolg_model_tf2 import DOLG |
| from dolg.resnet_tf2 import ResNet |
| |
| |
| backbone = ResNet(depth=depth, num_groups=1, width_per_group=64, bn_eps=1e-5, |
| bn_mom=0.1, trans_fun="bottleneck_transform", name="globalmodel") |
| model = DOLG(backbone, s4_dim=2048, s3_dim=1024, s2_dim=512, head_reduction_dim=512, |
| with_ma=False, num_classes=None, pretrained=f"r{depth}") |
| img = process_data("image.jpg", "", mode="tf") |
| img = np.expand_dims(img, axis=0) |
| output = model.predict(img) |
| print(output) |
| ``` |
|
|
| ## Data |
|
|
| The model has been trained on google landmark v2. You can find the dataset on the official repository : https://github.com/cvdfoundation/google-landmark . |
|
|
|
|
| # Citation : |
|
|
| ```bibtex |
| |
| @misc{yang2021dolg, |
| title={DOLG: Single-Stage Image Retrieval with Deep Orthogonal Fusion of Local and Global Features}, |
| author={Min Yang and Dongliang He and Miao Fan and Baorong Shi and Xuetong Xue and Fu Li and Errui Ding and Jizhou Huang}, |
| year={2021}, |
| eprint={2108.02927}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.CV} |
| } |
| |
| |
| @misc{https://doi.org/10.48550/arxiv.2004.01804, |
| doi = {10.48550/ARXIV.2004.01804}, |
| |
| url = {https://arxiv.org/abs/2004.01804}, |
| |
| author = {Weyand, Tobias and Araujo, Andre and Cao, Bingyi and Sim, Jack}, |
| |
| keywords = {Computer Vision and Pattern Recognition (cs.CV), FOS: Computer and information sciences, FOS: Computer and information sciences}, |
| |
| title = {Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval}, |
| ``` |
