| --- |
| library_name: transformers |
| license: apache-2.0 |
| tags: |
| - PartAI |
| - time-series-forecasting |
| - Classification |
| - Foundation-Models |
| base_model: |
| - google/timesfm-2.0-500m-pytorch |
| --- |
| |
| # Model Detail |
|
|
| FlaMinGo is a foundation model for time-series classification, fine-tuned on extensive **Persian financial datasets**. Developed by [Part AI](https://partdp.ai/), it extends Google's [TimesFM](https://github.com/google-research/timesfm) model by **adding classification capability**, enabling it to handle both forecasting and classification tasks. |
|
|
| # How to use |
|
|
| To perform inference on your dataset, follow these steps: |
|
|
| 1. Clone our repository |
| 2. Install the required dependencies. Ensure you are using **Python 3.11**: |
|
|
| ``` |
| cd FlaMinGo-timesfm |
| pip install --upgrade pip |
| pip install timesfm[torch] |
| pip install -e . |
| |
| ``` |
|
|
| 3. Download the TimeSFM model weights: |
|
|
| - [TimeSFM 200M Model Weights](https://huggingface.co/google/timesfm-1.0-200m-pytorch/tree/main) |
| - [TimeSFM 500M Model Weights](https://huggingface.co/google/timesfm-2.0-500m-pytorch/tree/main) |
|
|
| ## Configuration |
|
|
| Before running the model, set up the necessary configurations in the `.env` file located in the repository. The following environment variables must be configured: |
|
|
| - `MODEL_PATH`: Path to the TimeSFM model |
| - `SAVE_MODEL_PATH`: Path to save the fine-tuned model for the classification task |
| - `TRAIN_FILE_PATH`: Path to the training dataset |
| - `TEST_FILE_PATH`: Path to the test dataset |
| - `USE_GPU`: Use GPU or CPU |
| - `NUM_LAYERS`: Number of layers in the TimeSFM model (chosen based on the TimeSFM model size: 200M or 500M) |
| - `TARGET_COL_NAME`: Target column index in the dataset |
|
|
|
|
| Example `.env` file: |
|
|
| ``` |
| MODEL_PATH=/app/weights/torch_model.ckpt |
| SAVE_MODEL_PATH=timesfm_classification_model.pth |
| TRAIN_FILE_PATH=/app/dataset/ECG200_TRAIN |
| TEST_FILE_PATH=/app/dataset/ECG200_TEST |
| USE_GPU=cpu |
| NUM_LAYERS=20 |
| TARGET_COL_NAME=0 |
| ``` |
|
|
| ## Fine-Tuning the Model |
|
|
| To fine-tune the TimesFM model on your dataset, navigate to the `experiments/torch_classification_timesfm` directory in FlaMinGo repo and run the following command: |
|
|
| ``` |
| python run_timesfm_classification.py |
| ``` |
| This script will load the dataset, fine-tune the model, and save the fine-tuned model to the path specified in `SAVE_MODEL_PATH`. |
|
|
| Alternatively, you can use the `notebooks/finetune_timesfm_torch_classification.ipynb` Jupyter notebook to fine-tune the model and perform inference interactively. |
|
|
| ## Notes |
|
|
| - Ensure that the dataset paths (`TRAIN_FILE_PATH` and `TEST_FILE_PATH`) are correctly specified in the .env file. |
| - If using a GPU, set `USE_GPU=gpu` in the `.env` file and ensure that CUDA is properly installed. |
| - The number of layers (`NUM_LAYERS`) should match the TimesFM size (200M or 500M). |
| - Ensure that the training and testing datasets are in `.csv` format. |
|
|
|
|
| # Evaluation |
|
|
| FlaMinGo, available in two versions (FlaMinGo-200M and FlaMinGo-500M), was evaluated against three popular time-series classification models (MOMENT, TimesNet, and GPT4TS) on multiple datasets from the UCR collection. |
|
|
|
|
| | **Dataset** | **FlaMinGo (Timesfm-200M)** | **FlaMinGo (Timesfm-500M)** | **MOMENT** | **TimesNet** | **GPT4TS** | |
| |----------------------------------------|:--------------------------------:|:--------------------------------:|:----------:|:------------:|-----------:| |
| | Ham | 69.5 | 63.8 | 58.1 | 68.6 | **78.1** | |
| | InsectWingbeatSound | **62.1** | 54.4 | 60.7 | 52.9 | 59.8 | |
| | ECG5000 | 94 | 91.6 | **94.2** | 58.4 | 58.4 | |
| | ECG200 | **90** | 87 | 76 | 83 | 79 | |
| | BirdChicken | **85** | 75 | 85 | 45 | 55 | |
| | Adiac | 62.9 | 22.3 | **68.8** | 56.5 | 59.8 | |
| | ProximalPhalanxOutlineAgeGroup | 82.4 | 80.5 | 86.3 | **86.8** | 83.9 | |
| | Trace | 92 | 76 | **100** | 76 | 71 | |
| | wafer | 99.5 | 98.5 | **99.7** | 98.9 | 99.4 | |
| | CBF | **99.4** | 90.1 | 96 | 76.1 | 83 | |
| | ElectricDevices | **70** | 64 | 64.6 | 55.2 | 50.6 | |
| | MoteStrain | **80** | 74.6 | 77.4 | 70 | 68.1 | |
| | TwoLeadECG | **90.1** | 80.1 | 84.7 | 63.3 | 65.8 | |
| | ShapeletSim | 89.4 | 63.3 | **96.1** | 50 | 48.9 | |
| | Cricket_Z | **76.7** | 61.5 | 73.1 | 45.9 | 39.7 | |
| | Strawberry | 92.2 | 86.6 | **95.1** | 94.6 | 93.5 | |
| | FordA | 93.1 | 81.3 | **93.6** | 91.3 | 91.4 | |
| | Symbols | 87.6 | 81.8 | **93.6** | 86.4 | 69.4 | |
| | MiddlePhalanxTW | **63.7** | 63.4 | 53.2 | 50.6 | 57.1 | |
| | yoga | 77.5 | 73.6 | **83.4** | 67.2 | 69.1 | |
| | Meat | **93.3** | 93.3 | 91.7 | 43.3 | 66.7 | |
| | MedicalImages | 71.7 | 58.2 | **76.2** | 57.1 | 49.6 | |
| | DistalPhalanxTW | **77.5** | 77 | 61.2 | 60.4 | 61.9 | |
| | ProximalPhalanxOutlineCorrect | 81.1 | 79.7 | 85.6 | **86.9** | 80.1 | |
| | Herring | **59.4** | 59.4 | 59.4 | 53.1 | 57.8 | |
| | ToeSegmentation2 | 86.2 | 73.1 | **91.5** | 73.1 | 73.1 | |
| | OSULeaf | **79** | 57 | 78.5 | 39.7 | 23.1 | |
| | **Average Accuracy** | **81.6** | 73.2 | 80.1 | 66.1 | 68.2 | |
| |
| Our results show that FlaMinGo achieves competitive performance across diverse time-series classification tasks. It outperforms existing models in **13** classification tasks and achieves the **highest overall average accuracy** among all models. In most other cases, it delivers results comparable to the best-performing models. Since FlaMinGo has been fine-tuned on Persian financial data to improve forecasting capability, we also compared its forecasting performance against TimeSFM on crypto-related datasets we gathered. This evaluation provides insights into how fine-tuning affects the model's ability to predict financial trends. |
| |
| | **Dataset** | **MSE (pretrained)** | **MAE (pretrained)** | **MSE (fine-tuned)** | **MAE (fine-tuned)** | |
| |----------------------------------------|:--------------------------------:|:--------------------------------:|:----------:|:------------:| |
| | Crypto-1 | 1.93E-06 | 1.76E-04 | **1.24E-06** | **0.000144** | |
| | Crypto-2 | 4.31E-06 | 2.43E-04 | **3.66E-06** | **0.000218** | |
| | Crypto-3 | 7.36E-06 | 3.31E-04 | **6.54E-06** | **0.000307** | |
| | Crypto-4 | 6.71 | 0.3 | **5.92** | **0.28** | |
| | Crypto-5 | 7.26E-07 | 1.08E-04 | **6.68E-07** | **0.0001** | |
| | Crypto-6 | 54028.91 | 29.82 | **40619.24** | **25.42** | |
| | Crypto-7 | 0.19 | 0.0499 | **0.18** | **0.0458** | |
| | Crypto-8 | 0.00181 | 0.00543 | **0.00168** | **0.00515** | |
| | **Average Improvement** | | | **MSE: 14.88%** | **MAE: 9.74%** | |
| |
| |
| |
| |
| These results demonstrate the effectiveness of fine-tuning on Persian financial data, improving the model’s adaptability to structured time-series patterns. **It is also worth noting** that the fine-tuned model was evaluated on cryptocurrency datasets, which pose greater complexity, achieving a **14.88% improvement in MSE** and a **9.74% improvement in MAE**, further showcasing its enhanced predictive performance. |
| ## Contact us |
| |
| If you have any questions regarding this model, you can reach us via the [community](https://huggingface.co/PartAI/FlaMinGo-timesfm/discussions) on Hugging Face. |
