Text Generation
Transformers
Safetensors
English
lfm3_ts
time-series
time-series-question-answering
anomaly-detection
multimodal
lfm3
liquid
chatts
custom_code
conversational
Instructions to use nardi/LFM3-TS with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use nardi/LFM3-TS with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="nardi/LFM3-TS", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("nardi/LFM3-TS", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use nardi/LFM3-TS with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "nardi/LFM3-TS" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nardi/LFM3-TS", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/nardi/LFM3-TS
- SGLang
How to use nardi/LFM3-TS with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "nardi/LFM3-TS" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nardi/LFM3-TS", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "nardi/LFM3-TS" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "nardi/LFM3-TS", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use nardi/LFM3-TS with Docker Model Runner:
docker model run hf.co/nardi/LFM3-TS
Commit ·
8f94540
0
Parent(s):
LFM3-TS
Browse files- .gitattributes +36 -0
- LICENSE +71 -0
- LICENSE-Apache-2.0.txt +203 -0
- NOTICE +56 -0
- README.md +115 -0
- chat_template.jinja +284 -0
- config.json +138 -0
- configuration_lfm3_ts.py +165 -0
- feature_extraction_lfm3_ts.py +116 -0
- generation_config.json +16 -0
- model.safetensors +3 -0
- modeling_lfm3_ts.py +629 -0
- processing_lfm3_ts.py +285 -0
- processor_config.json +17 -0
- tokenizer.json +3 -0
- tokenizer_config.json +69 -0
.gitattributes
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
*.7z filter=lfs diff=lfs merge=lfs -text
|
| 2 |
+
*.arrow filter=lfs diff=lfs merge=lfs -text
|
| 3 |
+
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 4 |
+
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
| 5 |
+
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
| 6 |
+
*.ftz filter=lfs diff=lfs merge=lfs -text
|
| 7 |
+
*.gz filter=lfs diff=lfs merge=lfs -text
|
| 8 |
+
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 9 |
+
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 10 |
+
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
| 11 |
+
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 12 |
+
*.model filter=lfs diff=lfs merge=lfs -text
|
| 13 |
+
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
| 14 |
+
*.npy filter=lfs diff=lfs merge=lfs -text
|
| 15 |
+
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 16 |
+
*.onnx filter=lfs diff=lfs merge=lfs -text
|
| 17 |
+
*.ot filter=lfs diff=lfs merge=lfs -text
|
| 18 |
+
*.parquet filter=lfs diff=lfs merge=lfs -text
|
| 19 |
+
*.pb filter=lfs diff=lfs merge=lfs -text
|
| 20 |
+
*.pickle filter=lfs diff=lfs merge=lfs -text
|
| 21 |
+
*.pkl filter=lfs diff=lfs merge=lfs -text
|
| 22 |
+
*.pt filter=lfs diff=lfs merge=lfs -text
|
| 23 |
+
*.pth filter=lfs diff=lfs merge=lfs -text
|
| 24 |
+
*.rar filter=lfs diff=lfs merge=lfs -text
|
| 25 |
+
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 26 |
+
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
| 27 |
+
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
+
*.tar filter=lfs diff=lfs merge=lfs -text
|
| 29 |
+
*.tflite filter=lfs diff=lfs merge=lfs -text
|
| 30 |
+
*.tgz filter=lfs diff=lfs merge=lfs -text
|
| 31 |
+
*.wasm filter=lfs diff=lfs merge=lfs -text
|
| 32 |
+
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 33 |
+
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
+
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
+
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
LICENSE
ADDED
|
@@ -0,0 +1,71 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
LFM Open License v1.0
|
| 2 |
+
|
| 3 |
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
| 4 |
+
|
| 5 |
+
1. Definitions.
|
| 6 |
+
|
| 7 |
+
"License" shall mean the terms and conditions for use, reproduction, and distribution as defined by this document.
|
| 8 |
+
|
| 9 |
+
"Licensor" shall mean Liquid AI, Inc.
|
| 10 |
+
|
| 11 |
+
"Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity.
|
| 12 |
+
|
| 13 |
+
"You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License.
|
| 14 |
+
|
| 15 |
+
"Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files.
|
| 16 |
+
|
| 17 |
+
"Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types.
|
| 18 |
+
|
| 19 |
+
"Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work.
|
| 20 |
+
|
| 21 |
+
"Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof.
|
| 22 |
+
|
| 23 |
+
"Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution."
|
| 24 |
+
|
| 25 |
+
"Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work.
|
| 26 |
+
|
| 27 |
+
"Commercial Use" shall mean any use of the Work for direct or indirect commercial advantage or monetary compensation.
|
| 28 |
+
|
| 29 |
+
"Qualified Non-Profit Organization" shall mean a Legal Entity that is organized and operated exclusively for religious, charitable, scientific, testing for public safety, literary, or educational purposes, and which is exempt from federal income tax under Section 501(c)(3) of the United States Internal Revenue Code of 1986, as amended, or any equivalent non-profit or charitable organization in a foreign jurisdiction.
|
| 30 |
+
|
| 31 |
+
"Non-Commercial or Research Purposes" shall mean purposes that do not involve any use of the Work or a Derivative Work for Commercial Use.
|
| 32 |
+
|
| 33 |
+
"Threshold" shall mean annual revenue of 10 million United States dollars ($10,000,000) or more.
|
| 34 |
+
|
| 35 |
+
2. Grant of Copyright License. Subject to the terms and conditions of this License, including the Commercial Use limitation set forth in Section 5, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form.
|
| 36 |
+
|
| 37 |
+
3. Grant of Patent License. Subject to the terms and conditions of this License, including the Commercial Use limitation set forth in Section 5, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed.
|
| 38 |
+
|
| 39 |
+
4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions:
|
| 40 |
+
|
| 41 |
+
(a) You must give any other recipients of the Work or Derivative Works a copy of this License; and
|
| 42 |
+
|
| 43 |
+
(b) You must cause any modified files to carry prominent notices stating that You changed the files; and
|
| 44 |
+
|
| 45 |
+
(c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and
|
| 46 |
+
|
| 47 |
+
(d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License.
|
| 48 |
+
|
| 49 |
+
You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License.
|
| 50 |
+
|
| 51 |
+
5. Commercial Use Limitation.
|
| 52 |
+
|
| 53 |
+
(a) The rights granted under this License for Commercial Use are conditioned upon You or Your Legal Entity not exceeding the Threshold.
|
| 54 |
+
|
| 55 |
+
(b) Any Commercial Use of the Work or a Derivative Work by a Legal Entity that exceeds the Threshold is not licensed under this Agreement.
|
| 56 |
+
|
| 57 |
+
(c) The Threshold shall not apply to a Qualified Non-Profit Organization's use of the Work or a Derivative Work for Non-Commercial or Research Purposes.
|
| 58 |
+
|
| 59 |
+
6. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions.
|
| 60 |
+
|
| 61 |
+
7. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except for the reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file.
|
| 62 |
+
|
| 63 |
+
8. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.
|
| 64 |
+
|
| 65 |
+
9. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages.
|
| 66 |
+
|
| 67 |
+
10. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability.
|
| 68 |
+
|
| 69 |
+
11. Termination. This License will terminate automatically and immediately if You fail to comply with any of its terms and conditions. Upon termination, You must cease all use of the Work and any Derivative Works and delete all copies in Your possession.
|
| 70 |
+
|
| 71 |
+
END OF TERMS AND CONDITIONS
|
LICENSE-Apache-2.0.txt
ADDED
|
@@ -0,0 +1,203 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Copyright 2018- The Hugging Face team. All rights reserved.
|
| 2 |
+
|
| 3 |
+
Apache License
|
| 4 |
+
Version 2.0, January 2004
|
| 5 |
+
http://www.apache.org/licenses/
|
| 6 |
+
|
| 7 |
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
| 8 |
+
|
| 9 |
+
1. Definitions.
|
| 10 |
+
|
| 11 |
+
"License" shall mean the terms and conditions for use, reproduction,
|
| 12 |
+
and distribution as defined by Sections 1 through 9 of this document.
|
| 13 |
+
|
| 14 |
+
"Licensor" shall mean the copyright owner or entity authorized by
|
| 15 |
+
the copyright owner that is granting the License.
|
| 16 |
+
|
| 17 |
+
"Legal Entity" shall mean the union of the acting entity and all
|
| 18 |
+
other entities that control, are controlled by, or are under common
|
| 19 |
+
control with that entity. For the purposes of this definition,
|
| 20 |
+
"control" means (i) the power, direct or indirect, to cause the
|
| 21 |
+
direction or management of such entity, whether by contract or
|
| 22 |
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
| 23 |
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
| 24 |
+
|
| 25 |
+
"You" (or "Your") shall mean an individual or Legal Entity
|
| 26 |
+
exercising permissions granted by this License.
|
| 27 |
+
|
| 28 |
+
"Source" form shall mean the preferred form for making modifications,
|
| 29 |
+
including but not limited to software source code, documentation
|
| 30 |
+
source, and configuration files.
|
| 31 |
+
|
| 32 |
+
"Object" form shall mean any form resulting from mechanical
|
| 33 |
+
transformation or translation of a Source form, including but
|
| 34 |
+
not limited to compiled object code, generated documentation,
|
| 35 |
+
and conversions to other media types.
|
| 36 |
+
|
| 37 |
+
"Work" shall mean the work of authorship, whether in Source or
|
| 38 |
+
Object form, made available under the License, as indicated by a
|
| 39 |
+
copyright notice that is included in or attached to the work
|
| 40 |
+
(an example is provided in the Appendix below).
|
| 41 |
+
|
| 42 |
+
"Derivative Works" shall mean any work, whether in Source or Object
|
| 43 |
+
form, that is based on (or derived from) the Work and for which the
|
| 44 |
+
editorial revisions, annotations, elaborations, or other modifications
|
| 45 |
+
represent, as a whole, an original work of authorship. For the purposes
|
| 46 |
+
of this License, Derivative Works shall not include works that remain
|
| 47 |
+
separable from, or merely link (or bind by name) to the interfaces of,
|
| 48 |
+
the Work and Derivative Works thereof.
|
| 49 |
+
|
| 50 |
+
"Contribution" shall mean any work of authorship, including
|
| 51 |
+
the original version of the Work and any modifications or additions
|
| 52 |
+
to that Work or Derivative Works thereof, that is intentionally
|
| 53 |
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
| 54 |
+
or by an individual or Legal Entity authorized to submit on behalf of
|
| 55 |
+
the copyright owner. For the purposes of this definition, "submitted"
|
| 56 |
+
means any form of electronic, verbal, or written communication sent
|
| 57 |
+
to the Licensor or its representatives, including but not limited to
|
| 58 |
+
communication on electronic mailing lists, source code control systems,
|
| 59 |
+
and issue tracking systems that are managed by, or on behalf of, the
|
| 60 |
+
Licensor for the purpose of discussing and improving the Work, but
|
| 61 |
+
excluding communication that is conspicuously marked or otherwise
|
| 62 |
+
designated in writing by the copyright owner as "Not a Contribution."
|
| 63 |
+
|
| 64 |
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
| 65 |
+
on behalf of whom a Contribution has been received by Licensor and
|
| 66 |
+
subsequently incorporated within the Work.
|
| 67 |
+
|
| 68 |
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
| 69 |
+
this License, each Contributor hereby grants to You a perpetual,
|
| 70 |
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
| 71 |
+
copyright license to reproduce, prepare Derivative Works of,
|
| 72 |
+
publicly display, publicly perform, sublicense, and distribute the
|
| 73 |
+
Work and such Derivative Works in Source or Object form.
|
| 74 |
+
|
| 75 |
+
3. Grant of Patent License. Subject to the terms and conditions of
|
| 76 |
+
this License, each Contributor hereby grants to You a perpetual,
|
| 77 |
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
| 78 |
+
(except as stated in this section) patent license to make, have made,
|
| 79 |
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
| 80 |
+
where such license applies only to those patent claims licensable
|
| 81 |
+
by such Contributor that are necessarily infringed by their
|
| 82 |
+
Contribution(s) alone or by combination of their Contribution(s)
|
| 83 |
+
with the Work to which such Contribution(s) was submitted. If You
|
| 84 |
+
institute patent litigation against any entity (including a
|
| 85 |
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
| 86 |
+
or a Contribution incorporated within the Work constitutes direct
|
| 87 |
+
or contributory patent infringement, then any patent licenses
|
| 88 |
+
granted to You under this License for that Work shall terminate
|
| 89 |
+
as of the date such litigation is filed.
|
| 90 |
+
|
| 91 |
+
4. Redistribution. You may reproduce and distribute copies of the
|
| 92 |
+
Work or Derivative Works thereof in any medium, with or without
|
| 93 |
+
modifications, and in Source or Object form, provided that You
|
| 94 |
+
meet the following conditions:
|
| 95 |
+
|
| 96 |
+
(a) You must give any other recipients of the Work or
|
| 97 |
+
Derivative Works a copy of this License; and
|
| 98 |
+
|
| 99 |
+
(b) You must cause any modified files to carry prominent notices
|
| 100 |
+
stating that You changed the files; and
|
| 101 |
+
|
| 102 |
+
(c) You must retain, in the Source form of any Derivative Works
|
| 103 |
+
that You distribute, all copyright, patent, trademark, and
|
| 104 |
+
attribution notices from the Source form of the Work,
|
| 105 |
+
excluding those notices that do not pertain to any part of
|
| 106 |
+
the Derivative Works; and
|
| 107 |
+
|
| 108 |
+
(d) If the Work includes a "NOTICE" text file as part of its
|
| 109 |
+
distribution, then any Derivative Works that You distribute must
|
| 110 |
+
include a readable copy of the attribution notices contained
|
| 111 |
+
within such NOTICE file, excluding those notices that do not
|
| 112 |
+
pertain to any part of the Derivative Works, in at least one
|
| 113 |
+
of the following places: within a NOTICE text file distributed
|
| 114 |
+
as part of the Derivative Works; within the Source form or
|
| 115 |
+
documentation, if provided along with the Derivative Works; or,
|
| 116 |
+
within a display generated by the Derivative Works, if and
|
| 117 |
+
wherever such third-party notices normally appear. The contents
|
| 118 |
+
of the NOTICE file are for informational purposes only and
|
| 119 |
+
do not modify the License. You may add Your own attribution
|
| 120 |
+
notices within Derivative Works that You distribute, alongside
|
| 121 |
+
or as an addendum to the NOTICE text from the Work, provided
|
| 122 |
+
that such additional attribution notices cannot be construed
|
| 123 |
+
as modifying the License.
|
| 124 |
+
|
| 125 |
+
You may add Your own copyright statement to Your modifications and
|
| 126 |
+
may provide additional or different license terms and conditions
|
| 127 |
+
for use, reproduction, or distribution of Your modifications, or
|
| 128 |
+
for any such Derivative Works as a whole, provided Your use,
|
| 129 |
+
reproduction, and distribution of the Work otherwise complies with
|
| 130 |
+
the conditions stated in this License.
|
| 131 |
+
|
| 132 |
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
| 133 |
+
any Contribution intentionally submitted for inclusion in the Work
|
| 134 |
+
by You to the Licensor shall be under the terms and conditions of
|
| 135 |
+
this License, without any additional terms or conditions.
|
| 136 |
+
Notwithstanding the above, nothing herein shall supersede or modify
|
| 137 |
+
the terms of any separate license agreement you may have executed
|
| 138 |
+
with Licensor regarding such Contributions.
|
| 139 |
+
|
| 140 |
+
6. Trademarks. This License does not grant permission to use the trade
|
| 141 |
+
names, trademarks, service marks, or product names of the Licensor,
|
| 142 |
+
except as required for reasonable and customary use in describing the
|
| 143 |
+
origin of the Work and reproducing the content of the NOTICE file.
|
| 144 |
+
|
| 145 |
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
| 146 |
+
agreed to in writing, Licensor provides the Work (and each
|
| 147 |
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
| 148 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
| 149 |
+
implied, including, without limitation, any warranties or conditions
|
| 150 |
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
| 151 |
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
| 152 |
+
appropriateness of using or redistributing the Work and assume any
|
| 153 |
+
risks associated with Your exercise of permissions under this License.
|
| 154 |
+
|
| 155 |
+
8. Limitation of Liability. In no event and under no legal theory,
|
| 156 |
+
whether in tort (including negligence), contract, or otherwise,
|
| 157 |
+
unless required by applicable law (such as deliberate and grossly
|
| 158 |
+
negligent acts) or agreed to in writing, shall any Contributor be
|
| 159 |
+
liable to You for damages, including any direct, indirect, special,
|
| 160 |
+
incidental, or consequential damages of any character arising as a
|
| 161 |
+
result of this License or out of the use or inability to use the
|
| 162 |
+
Work (including but not limited to damages for loss of goodwill,
|
| 163 |
+
work stoppage, computer failure or malfunction, or any and all
|
| 164 |
+
other commercial damages or losses), even if such Contributor
|
| 165 |
+
has been advised of the possibility of such damages.
|
| 166 |
+
|
| 167 |
+
9. Accepting Warranty or Additional Liability. While redistributing
|
| 168 |
+
the Work or Derivative Works thereof, You may choose to offer,
|
| 169 |
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
| 170 |
+
or other liability obligations and/or rights consistent with this
|
| 171 |
+
License. However, in accepting such obligations, You may act only
|
| 172 |
+
on Your own behalf and on Your sole responsibility, not on behalf
|
| 173 |
+
of any other Contributor, and only if You agree to indemnify,
|
| 174 |
+
defend, and hold each Contributor harmless for any liability
|
| 175 |
+
incurred by, or claims asserted against, such Contributor by reason
|
| 176 |
+
of your accepting any such warranty or additional liability.
|
| 177 |
+
|
| 178 |
+
END OF TERMS AND CONDITIONS
|
| 179 |
+
|
| 180 |
+
APPENDIX: How to apply the Apache License to your work.
|
| 181 |
+
|
| 182 |
+
To apply the Apache License to your work, attach the following
|
| 183 |
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
| 184 |
+
replaced with your own identifying information. (Don't include
|
| 185 |
+
the brackets!) The text should be enclosed in the appropriate
|
| 186 |
+
comment syntax for the file format. We also recommend that a
|
| 187 |
+
file or class name and description of purpose be included on the
|
| 188 |
+
same "printed page" as the copyright notice for easier
|
| 189 |
+
identification within third-party archives.
|
| 190 |
+
|
| 191 |
+
Copyright [yyyy] [name of copyright owner]
|
| 192 |
+
|
| 193 |
+
Licensed under the Apache License, Version 2.0 (the "License");
|
| 194 |
+
you may not use this file except in compliance with the License.
|
| 195 |
+
You may obtain a copy of the License at
|
| 196 |
+
|
| 197 |
+
http://www.apache.org/licenses/LICENSE-2.0
|
| 198 |
+
|
| 199 |
+
Unless required by applicable law or agreed to in writing, software
|
| 200 |
+
distributed under the License is distributed on an "AS IS" BASIS,
|
| 201 |
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
| 202 |
+
See the License for the specific language governing permissions and
|
| 203 |
+
limitations under the License.
|
NOTICE
ADDED
|
@@ -0,0 +1,56 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
LFM3-TS
|
| 2 |
+
Copyright 2026 Nardi Xhepi
|
| 3 |
+
|
| 4 |
+
LFM3-TS is distributed under the LFM Open License v1.0. A copy of the licence is in the file LICENSE.
|
| 5 |
+
|
| 6 |
+
This is an independent project. It is not affiliated with, sponsored by or endorsed by Liquid AI, Inc. The names
|
| 7 |
+
"Liquid AI", "LFM" and "LFM3" belong to Liquid AI, Inc. and are used here only to describe the origin of the work.
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
1. LFM3-1.7B (Liquid AI, Inc.)
|
| 11 |
+
|
| 12 |
+
The LFM3-TS model weights are a Derivative Work of LFM3-1.7B by Liquid AI, Inc.
|
| 13 |
+
(https://huggingface.co/LiquidAI/LFM3-1.7B), which is licensed under the LFM Open License v1.0.
|
| 14 |
+
|
| 15 |
+
NOTICE OF CHANGES. The weights of LFM3-1.7B were modified by Nardi Xhepi:
|
| 16 |
+
|
| 17 |
+
- the weights of the language model were fine-tuned on time-series question answering, except the token
|
| 18 |
+
embedding matrix (shared with the output layer), which is unchanged;
|
| 19 |
+
- a time-series encoder (`model.ts_tower`) and a projector (`model.multi_modal_projector`) were added; these
|
| 20 |
+
weights are new and are not part of LFM3-1.7B;
|
| 21 |
+
- the tokenizer gained the name `<|ts|>` for a previously unused token id (125033), and the chat template gained
|
| 22 |
+
a `timeseries` content part.
|
| 23 |
+
|
| 24 |
+
The distribution of LFM3-1.7B this work started from contains no NOTICE file.
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
2. Hugging Face Transformers
|
| 28 |
+
|
| 29 |
+
The files
|
| 30 |
+
|
| 31 |
+
src/lfm3_ts/configuration_lfm3_ts.py
|
| 32 |
+
src/lfm3_ts/modeling_lfm3_ts.py
|
| 33 |
+
src/lfm3_ts/processing_lfm3_ts.py
|
| 34 |
+
|
| 35 |
+
(shipped with the model checkpoint as configuration_lfm3_ts.py, modeling_lfm3_ts.py and processing_lfm3_ts.py) are
|
| 36 |
+
adapted from `transformers.models.lfm3_vl` of Hugging Face Transformers
|
| 37 |
+
(https://github.com/huggingface/transformers), Copyright 2026 The HuggingFace Team, licensed under the Apache
|
| 38 |
+
License, Version 2.0. They were changed by Nardi Xhepi: the vision tower and the image inputs are replaced by a
|
| 39 |
+
time-series encoder and time-series inputs. A copy of the Apache License, Version 2.0 is in
|
| 40 |
+
LICENSES/Apache-2.0.txt.
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
3. Method and data
|
| 44 |
+
|
| 45 |
+
The method follows ChatTS (Xie et al., "ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced
|
| 46 |
+
Understanding and Reasoning", arXiv:2412.03104). No ChatTS code or weights are included.
|
| 47 |
+
|
| 48 |
+
Training used the ChatTS synthetic training data (https://huggingface.co/datasets/ChatTSRepo/ChatTS-Training-Dataset),
|
| 49 |
+
windows of real series from TSB-AD-U (https://github.com/TheDatumOrg/TSB-AD) and prompts from databricks-dolly-15k
|
| 50 |
+
(https://huggingface.co/datasets/databricks/databricks-dolly-15k). None of these datasets is redistributed here.
|
| 51 |
+
|
| 52 |
+
The real series were gathered by TSB-AD from their original sources. Those used in training, with their licences as
|
| 53 |
+
stated by each source: the MIT-BIH Supraventricular Arrhythmia, Long-Term ECG and Arrhythmia databases of PhysioNet
|
| 54 |
+
(Open Data Commons Attribution License v1.0); OPPORTUNITY and Daphnet of the UCI Machine Learning Repository and
|
| 55 |
+
CATS (CC BY 4.0); the Server Machine Dataset of OmniAnomaly (MIT); Exathlon (CC BY-NC-SA 4.0); WSD, SED and Power
|
| 56 |
+
(no licence stated).
|
README.md
ADDED
|
@@ -0,0 +1,115 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: lfm1.0
|
| 4 |
+
license_link: LICENSE
|
| 5 |
+
base_model: LiquidAI/LFM3-1.7B
|
| 6 |
+
base_model_relation: finetune
|
| 7 |
+
library_name: transformers
|
| 8 |
+
pipeline_tag: text-generation
|
| 9 |
+
language:
|
| 10 |
+
- en
|
| 11 |
+
tags:
|
| 12 |
+
- time-series
|
| 13 |
+
- anomaly-detection
|
| 14 |
+
- time-series-question-answering
|
| 15 |
+
- multimodal
|
| 16 |
+
- lfm3
|
| 17 |
+
- liquid
|
| 18 |
+
- chatts
|
| 19 |
+
- custom_code
|
| 20 |
+
datasets:
|
| 21 |
+
- ChatTSRepo/ChatTS-Training-Dataset
|
| 22 |
+
- databricks/databricks-dolly-15k
|
| 23 |
+
---
|
| 24 |
+
|
| 25 |
+
# LFM3-TS
|
| 26 |
+
|
| 27 |
+
A small language model that reads time series. You give it a series and a question, and it answers in text: is
|
| 28 |
+
there an anomaly, where, and what does it look like.
|
| 29 |
+
|
| 30 |
+
LFM3-TS is [LFM3-1.7B](https://huggingface.co/LiquidAI/LFM3-1.7B) from Liquid AI with a time-series encoder in front,
|
| 31 |
+
built on the ideas and the synthetic data of [ChatTS](https://arxiv.org/abs/2412.03104). It has 1.8 billion
|
| 32 |
+
parameters.
|
| 33 |
+
|
| 34 |
+
- **Code and training recipe:** https://github.com/nardi-xhepi/lfm3-ts
|
| 35 |
+
- **Blog post:** https://nardi-xhepi.github.io/blog/posts/lfm3-ts.html
|
| 36 |
+
- **Author:** Nardi Xhepi
|
| 37 |
+
|
| 38 |
+
This is an independent project, not affiliated with Liquid AI.
|
| 39 |
+
|
| 40 |
+
## How to use
|
| 41 |
+
|
| 42 |
+
```bash
|
| 43 |
+
pip install -U "transformers>=5.23"
|
| 44 |
+
```
|
| 45 |
+
|
| 46 |
+
You also need PyTorch and NumPy. LFM3 was added to `transformers` in version 5.23; on Colab, which already has
|
| 47 |
+
PyTorch, run only the line above and restart the runtime.
|
| 48 |
+
|
| 49 |
+
```python
|
| 50 |
+
import numpy as np
|
| 51 |
+
from transformers import AutoModelForCausalLM, AutoProcessor
|
| 52 |
+
|
| 53 |
+
model = AutoModelForCausalLM.from_pretrained("nardi/LFM3-TS", trust_remote_code=True)
|
| 54 |
+
processor = AutoProcessor.from_pretrained("nardi/LFM3-TS", trust_remote_code=True)
|
| 55 |
+
|
| 56 |
+
series = np.sin(np.arange(512) / 8)
|
| 57 |
+
series[300] += 4 # a spike
|
| 58 |
+
|
| 59 |
+
print(model.chat(processor, series, "Is there an anomaly in this time series? If so, where is it and what does it look like?"))
|
| 60 |
+
```
|
| 61 |
+
|
| 62 |
+
`series` is a 1D array or list (NaN marks a missing value). Points are numbered from 0. The model was trained on
|
| 63 |
+
series of up to 2,048 points, in English.
|
| 64 |
+
|
| 65 |
+
## Model
|
| 66 |
+
|
| 67 |
+
| | |
|
| 68 |
+
|---|---|
|
| 69 |
+
| Language model | LFM3-1.7B, every weight fine-tuned except the token embeddings |
|
| 70 |
+
| Time-series encoder | patches of 8 points, 4 feed-forward and 2 attention layers, width 1024 |
|
| 71 |
+
| Projector | MLP to the width of the language model (2048) |
|
| 72 |
+
| Tokens per series | one per patch, plus 2: a series of 512 points costs 66 tokens |
|
| 73 |
+
| Parameters | 1.8 billion |
|
| 74 |
+
|
| 75 |
+
The series is standardized for the encoder, and its offset and scale are written in the prompt as text, so the
|
| 76 |
+
answers can give real values.
|
| 77 |
+
|
| 78 |
+
## Results
|
| 79 |
+
|
| 80 |
+
**The ChatTS benchmark** ([sets A and B](https://zenodo.org/records/14349206)), scored with the scoring code of the
|
| 81 |
+
ChatTS repository:
|
| 82 |
+
|
| 83 |
+
| | Parameters | Set A, categorical | Set A, numerical | Set B, categorical | Set B, numerical |
|
| 84 |
+
|---|---|---|---|---|---|
|
| 85 |
+
| **LFM3-TS** | 1.8B | 82.0% | 77.5% | 79.5% | 76.8% |
|
| 86 |
+
| ChatTS-8B | 8B | 83.8% | 82.3% | 84.6% | 82.9% |
|
| 87 |
+
|
| 88 |
+
**Anomalies in real series**: 234 windows of 512 points from seven sources of
|
| 89 |
+
[TSB-AD-U](https://github.com/TheDatumOrg/TSB-AD) that neither model was trained on (105 with a labelled anomaly, 129
|
| 90 |
+
without). An answer is correct when it says "yes" and names a point within 16 points of the labelled stretch.
|
| 91 |
+
|
| 92 |
+
| | Parameters | Precision | Recall | F1 | False alarms |
|
| 93 |
+
|---|---|---|---|---|---|
|
| 94 |
+
| **LFM3-TS** | 1.8B | 66% | 64% | 0.65 | 28 of 129 |
|
| 95 |
+
| ChatTS-8B | 8B | 40% | 53% | 0.46 | 61 of 129 |
|
| 96 |
+
|
| 97 |
+
## Training
|
| 98 |
+
|
| 99 |
+
Three stages, each starting from the weights of the one before: generated questions that can only be answered by
|
| 100 |
+
reading the series; the same questions on series of up to 2,048 points; then a mixture of ChatTS synthetic data
|
| 101 |
+
(48%), generated questions (19%), plain chat (15%), real series with labelled anomalies (10%) and real series with
|
| 102 |
+
an event put in (8%). The commands are in the code repository.
|
| 103 |
+
|
| 104 |
+
## Licence
|
| 105 |
+
|
| 106 |
+
LFM3-TS is released under the [LFM Open License v1.0](LICENSE), the licence of LFM3-1.7B, which the weights derive
|
| 107 |
+
from. See `NOTICE` for what was changed and for the sources and licences of the training data.
|
| 108 |
+
|
| 109 |
+
## Credits
|
| 110 |
+
|
| 111 |
+
[LFM3-1.7B](https://huggingface.co/LiquidAI/LFM3-1.7B) (Liquid AI) ·
|
| 112 |
+
[ChatTS](https://github.com/NetManAIOps/ChatTS) (Xie et al.) ·
|
| 113 |
+
[TSB-AD](https://github.com/TheDatumOrg/TSB-AD) (Liu and Paparrizos) ·
|
| 114 |
+
[databricks-dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k) ·
|
| 115 |
+
[Hugging Face Transformers](https://github.com/huggingface/transformers)
|
chat_template.jinja
ADDED
|
@@ -0,0 +1,284 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{{- bos_token -}}
|
| 2 |
+
{%- set preserve_thinking = preserve_thinking | default(false) -%}
|
| 3 |
+
{%- set enable_thinking = enable_thinking | default(true) -%}
|
| 4 |
+
|
| 5 |
+
{%- macro render_value(v) -%}
|
| 6 |
+
{%- if v is string -%}
|
| 7 |
+
{{- v -}}
|
| 8 |
+
{%- elif v is boolean -%}
|
| 9 |
+
{{- "true" if v else "false" -}}
|
| 10 |
+
{%- elif v is none -%}
|
| 11 |
+
{{- "null" -}}
|
| 12 |
+
{%- elif v is mapping or (v is sequence and v is not string) -%}
|
| 13 |
+
{{- v | tojson -}}
|
| 14 |
+
{%- else -%}
|
| 15 |
+
{{- v -}}
|
| 16 |
+
{%- endif -%}
|
| 17 |
+
{%- endmacro -%}
|
| 18 |
+
|
| 19 |
+
{%- macro parse_content(content) -%}
|
| 20 |
+
{%- if content is string -%}
|
| 21 |
+
{{- content -}}
|
| 22 |
+
{%- elif content is mapping and content.get("type") == "image" -%}
|
| 23 |
+
{{- "<|vision_start|><|image|><|vision_end|>" -}}
|
| 24 |
+
{%- elif content is mapping and content.get("type") == "video" -%}
|
| 25 |
+
{{- "<|vision_start|><|video|><|vision_end|>" -}}
|
| 26 |
+
{%- elif content is mapping and content.get("type") == "timeseries" -%}
|
| 27 |
+
{{- "<|ts|>" -}}
|
| 28 |
+
{%- elif content is mapping and content.get("type") == "text" -%}
|
| 29 |
+
{{- (content.get("text") or "") | string -}}
|
| 30 |
+
{%- elif content is mapping -%}
|
| 31 |
+
{{- content | tojson -}}
|
| 32 |
+
{%- elif content is iterable -%}
|
| 33 |
+
{%- set _ns = namespace(result="") -%}
|
| 34 |
+
{%- for item in content -%}
|
| 35 |
+
{%- if item is string -%}
|
| 36 |
+
{%- set _ns.result = _ns.result + item -%}
|
| 37 |
+
{%- elif item is mapping and item.get("type") == "image" -%}
|
| 38 |
+
{%- set _ns.result = _ns.result + "<|vision_start|><|image|><|vision_end|>" -%}
|
| 39 |
+
{%- elif item is mapping and item.get("type") == "video" -%}
|
| 40 |
+
{%- set _ns.result = _ns.result + "<|vision_start|><|video|><|vision_end|>" -%}
|
| 41 |
+
{%- elif item is mapping and item.get("type") == "timeseries" -%}
|
| 42 |
+
{%- set _ns.result = _ns.result + "<|ts|>" -%}
|
| 43 |
+
{%- elif item is mapping and item.get("type") == "text" -%}
|
| 44 |
+
{%- set _ns.result = _ns.result + ((item.get("text") or "") | string) -%}
|
| 45 |
+
{%- else -%}
|
| 46 |
+
{%- set _ns.result = _ns.result + (item | tojson) -%}
|
| 47 |
+
{%- endif -%}
|
| 48 |
+
{%- endfor -%}
|
| 49 |
+
{{- _ns.result -}}
|
| 50 |
+
{%- elif content is not none -%}
|
| 51 |
+
{{- content | tojson -}}
|
| 52 |
+
{%- endif -%}
|
| 53 |
+
{%- endmacro -%}
|
| 54 |
+
|
| 55 |
+
{%- macro render_tool_calls(tool_calls) -%}
|
| 56 |
+
{%- for tool_call in tool_calls -%}
|
| 57 |
+
{%- if not loop.first -%}{{- "\n\n" -}}{%- endif -%}
|
| 58 |
+
{%- set func = tool_call["function"] if "function" in tool_call else tool_call -%}
|
| 59 |
+
{%- set func_args = func.get("arguments") -%}
|
| 60 |
+
{%- if func_args is string and (func_args | trim) not in ["", "{}", "null"] -%}
|
| 61 |
+
{{- raise_exception("Tool call arguments must be a mapping, got a JSON-encoded string: parse arguments with json.loads() before applying the chat template") -}}
|
| 62 |
+
{%- endif -%}
|
| 63 |
+
{{- "<tool_call>\n<function=" + func["name"] + ">\n" -}}
|
| 64 |
+
{%- if func_args is mapping -%}
|
| 65 |
+
{%- for arg_name, arg_value in func_args.items() -%}
|
| 66 |
+
{{- "<parameter=" + arg_name + ">\n" + render_value(arg_value) + "\n</parameter>\n" -}}
|
| 67 |
+
{%- endfor -%}
|
| 68 |
+
{%- endif -%}
|
| 69 |
+
{{- "</function>\n</tool_call>" -}}
|
| 70 |
+
{%- endfor -%}
|
| 71 |
+
{%- endmacro -%}
|
| 72 |
+
|
| 73 |
+
{%- macro tool_to_json(tool) -%}
|
| 74 |
+
{%- set fn = tool.function if tool.function is defined else tool -%}
|
| 75 |
+
{%- set ns_t = namespace(first=true) -%}
|
| 76 |
+
{{- "{" -}}
|
| 77 |
+
{%- for k, v in fn.items() -%}
|
| 78 |
+
{%- if k != "defer_loading" -%}
|
| 79 |
+
{%- if not ns_t.first -%}{{- ", " -}}{%- endif -%}
|
| 80 |
+
{%- set ns_t.first = false -%}
|
| 81 |
+
{{- k | tojson -}}{{- ": " -}}{{- v | tojson -}}
|
| 82 |
+
{%- endif -%}
|
| 83 |
+
{%- endfor -%}
|
| 84 |
+
{{- "}" -}}
|
| 85 |
+
{%- endmacro -%}
|
| 86 |
+
|
| 87 |
+
{%- macro is_tool_reference(item) -%}
|
| 88 |
+
{{- "1" if item is mapping and item.get("type") == "tool_reference" else "" -}}
|
| 89 |
+
{%- endmacro -%}
|
| 90 |
+
|
| 91 |
+
{%- macro referenced_tool_to_json(name) -%}
|
| 92 |
+
{%- set _ns = namespace(found=none) -%}
|
| 93 |
+
{%- for tool in tools or [] -%}
|
| 94 |
+
{%- set fn = tool.function if tool.function is defined else tool -%}
|
| 95 |
+
{%- if fn.name == name -%}
|
| 96 |
+
{%- set _ns.found = fn -%}
|
| 97 |
+
{%- endif -%}
|
| 98 |
+
{%- endfor -%}
|
| 99 |
+
{%- if _ns.found is none -%}
|
| 100 |
+
{{- raise_exception("Tool reference '" + name | string + "' not found in available tools. Every tool referenced by a tool_reference must be in `tools` on every request.") -}}
|
| 101 |
+
{%- endif -%}
|
| 102 |
+
{{- tool_to_json(_ns.found) -}}
|
| 103 |
+
{%- endmacro -%}
|
| 104 |
+
|
| 105 |
+
{%- macro render_tool_response(content) -%}
|
| 106 |
+
{%- if content is string and "<tool_response>" in content -%}
|
| 107 |
+
{{- content -}}
|
| 108 |
+
{%- else -%}
|
| 109 |
+
{{- "<tool_response>\n" -}}
|
| 110 |
+
{%- if content is string -%}
|
| 111 |
+
{{- content -}}
|
| 112 |
+
{%- elif content is mapping -%}
|
| 113 |
+
{{- parse_content(content) -}}
|
| 114 |
+
{%- elif content is iterable -%}
|
| 115 |
+
{%- for item in content -%}
|
| 116 |
+
{%- if not loop.first -%}{{- "\n" -}}{%- endif -%}
|
| 117 |
+
{%- if is_tool_reference(item) -%}
|
| 118 |
+
{%- if loop.first or not is_tool_reference(loop.previtem) -%}
|
| 119 |
+
{{- "<additional_tools>\n" -}}
|
| 120 |
+
{%- endif -%}
|
| 121 |
+
{{- referenced_tool_to_json(item.name) -}}
|
| 122 |
+
{%- if loop.last or not is_tool_reference(loop.nextitem) -%}
|
| 123 |
+
{{- "\n</additional_tools>" -}}
|
| 124 |
+
{%- endif -%}
|
| 125 |
+
{%- elif item is mapping and item.result is defined and item.result is mapping and item.result.tools is defined -%}
|
| 126 |
+
{{- "<additional_tools>\n" -}}
|
| 127 |
+
{%- set _tl = item.result.tools -%}
|
| 128 |
+
{%- if _tl is string -%}
|
| 129 |
+
{{- _tl -}}
|
| 130 |
+
{%- elif _tl is mapping -%}
|
| 131 |
+
{{- tool_to_json(_tl) -}}
|
| 132 |
+
{%- else -%}
|
| 133 |
+
{%- for t in _tl -%}
|
| 134 |
+
{%- if not loop.first -%}{{- "\n" -}}{%- endif -%}
|
| 135 |
+
{{- tool_to_json(t) -}}
|
| 136 |
+
{%- endfor -%}
|
| 137 |
+
{%- endif -%}
|
| 138 |
+
{{- "\n</additional_tools>" -}}
|
| 139 |
+
{%- else -%}
|
| 140 |
+
{{- parse_content(item) -}}
|
| 141 |
+
{%- endif -%}
|
| 142 |
+
{%- endfor -%}
|
| 143 |
+
{%- elif content is not none -%}
|
| 144 |
+
{{- content | tojson -}}
|
| 145 |
+
{%- endif -%}
|
| 146 |
+
{{- "\n</tool_response>" -}}
|
| 147 |
+
{%- endif -%}
|
| 148 |
+
{%- endmacro -%}
|
| 149 |
+
|
| 150 |
+
{%- set ns = namespace(system_prompt="", last_user_index=-1) -%}
|
| 151 |
+
{%- if messages and messages[0]["role"] == "system" -%}
|
| 152 |
+
{%- if messages[0].get("content") -%}
|
| 153 |
+
{%- set ns.system_prompt = parse_content(messages[0]["content"]) -%}
|
| 154 |
+
{%- endif -%}
|
| 155 |
+
{%- set messages = messages[1:] -%}
|
| 156 |
+
{%- endif -%}
|
| 157 |
+
{%- if tools -%}
|
| 158 |
+
{%- set tools_ns = namespace(lines=[]) -%}
|
| 159 |
+
{%- for tool in tools -%}
|
| 160 |
+
{%- set fn = tool.function if tool.function is defined else tool -%}
|
| 161 |
+
{%- set _dyn = (fn.defer_loading is defined and fn.defer_loading) or (tool.defer_loading is defined and tool.defer_loading) -%}
|
| 162 |
+
{%- if not _dyn -%}
|
| 163 |
+
{%- set tools_ns.lines = tools_ns.lines + [tool_to_json(fn)] -%}
|
| 164 |
+
{%- endif -%}
|
| 165 |
+
{%- endfor -%}
|
| 166 |
+
{%- if tools_ns.lines -%}
|
| 167 |
+
{%- set ns.system_prompt = ns.system_prompt + ("\n\n" if ns.system_prompt else "") + "<tools>\n" + (tools_ns.lines | join("\n")) + "\n</tools>" -%}
|
| 168 |
+
{%- endif -%}
|
| 169 |
+
{%- endif -%}
|
| 170 |
+
{%- if ns.system_prompt -%}
|
| 171 |
+
{{- "<|im_start|>system\n" + ns.system_prompt + "<|im_end|>\n" -}}
|
| 172 |
+
{%- endif -%}
|
| 173 |
+
{%- for message in messages -%}
|
| 174 |
+
{%- if message["role"] == "user" -%}
|
| 175 |
+
{%- set ns.last_user_index = loop.index0 -%}
|
| 176 |
+
{%- endif -%}
|
| 177 |
+
{%- endfor -%}
|
| 178 |
+
{%- set carry = namespace(pending=[]) -%}
|
| 179 |
+
{%- for message in messages -%}
|
| 180 |
+
{%- set _next = messages[loop.index0 + 1] if not loop.last else none -%}
|
| 181 |
+
{%- set _merge = message.role == "tool" and not message.get("additional_tools")
|
| 182 |
+
and _next is not none and _next.role == "tool" and not _next.get("additional_tools")
|
| 183 |
+
and message.get("tool_call_id") and _next.get("tool_call_id") == message.get("tool_call_id") -%}
|
| 184 |
+
{%- set _items = [] -%}
|
| 185 |
+
{%- if message.role == "tool" and not message.get("additional_tools") -%}
|
| 186 |
+
{%- set _c = message.get("content") -%}
|
| 187 |
+
{%- if _c is string -%}
|
| 188 |
+
{%- set _items = [{"type": "text", "text": _c}] if _c else [] -%}
|
| 189 |
+
{%- elif _c is mapping -%}
|
| 190 |
+
{%- set _items = [_c] -%}
|
| 191 |
+
{%- elif _c is iterable -%}
|
| 192 |
+
{%- set _items = _c | list -%}
|
| 193 |
+
{%- elif _c is not none -%}
|
| 194 |
+
{%- set _items = [_c] -%}
|
| 195 |
+
{%- endif -%}
|
| 196 |
+
{%- endif -%}
|
| 197 |
+
{%- if _merge -%}
|
| 198 |
+
{%- set carry.pending = carry.pending + _items -%}
|
| 199 |
+
{%- else -%}
|
| 200 |
+
{{- "<|im_start|>" + message.role + "\n" -}}
|
| 201 |
+
{%- if message.role == "assistant" -%}
|
| 202 |
+
{%- generation -%}
|
| 203 |
+
{%- set thinking = message.thinking if message.thinking is defined else message.reasoning if message.reasoning is defined else message.reasoning_content -%}
|
| 204 |
+
{%- set keep_thinking = preserve_thinking or loop.index0 > ns.last_user_index -%}
|
| 205 |
+
{%- if thinking is defined and keep_thinking -%}
|
| 206 |
+
{{- "<think>" + (thinking or "") + "</think>" -}}
|
| 207 |
+
{%- elif keep_thinking and "</think>" not in (message.get("content") if message.get("content") is string else "") -%}
|
| 208 |
+
{{- "<think></think>" -}}
|
| 209 |
+
{%- endif -%}
|
| 210 |
+
{%- set _cfm_tag = "CONTINUE_FINAL_MESSAGE_TAG " -%}
|
| 211 |
+
{%- set _has_cfm = false -%}
|
| 212 |
+
{%- set content = "" -%}
|
| 213 |
+
{%- if message.get("content") -%}
|
| 214 |
+
{%- set content = parse_content(message.content) -%}
|
| 215 |
+
{%- endif -%}
|
| 216 |
+
{%- if not keep_thinking -%}
|
| 217 |
+
{%- if "</think>" in content -%}
|
| 218 |
+
{%- set content = content.split("</think>")[-1] | trim -%}
|
| 219 |
+
{%- endif -%}
|
| 220 |
+
{%- set content = "<think></think>" + content -%}
|
| 221 |
+
{%- endif -%}
|
| 222 |
+
{%- if content.endswith(_cfm_tag) -%}
|
| 223 |
+
{%- set _has_cfm = true -%}
|
| 224 |
+
{%- set _trunc_len = (content | length) - (_cfm_tag | length) -%}
|
| 225 |
+
{%- set content = content[:_trunc_len] -%}
|
| 226 |
+
{%- endif -%}
|
| 227 |
+
{{- content -}}
|
| 228 |
+
{%- if message.tool_calls -%}
|
| 229 |
+
{{- render_tool_calls(message.tool_calls) -}}
|
| 230 |
+
{%- endif -%}
|
| 231 |
+
{%- if _has_cfm -%}
|
| 232 |
+
{{- _cfm_tag -}}
|
| 233 |
+
{%- endif -%}
|
| 234 |
+
{{- "<|im_end|>\n" -}}
|
| 235 |
+
{%- endgeneration -%}
|
| 236 |
+
{%- elif message.role == "tool" -%}
|
| 237 |
+
{%- if message.get("additional_tools") -%}
|
| 238 |
+
{%- set _at = message["additional_tools"] -%}
|
| 239 |
+
{%- if _at is mapping or (_at is sequence and _at is not string) -%}
|
| 240 |
+
{%- set _src = _at -%}
|
| 241 |
+
{%- else -%}
|
| 242 |
+
{%- set _src = message.get("content") -%}
|
| 243 |
+
{%- endif -%}
|
| 244 |
+
{%- if _src is string and "<tool_response>" in _src -%}
|
| 245 |
+
{{- _src -}}
|
| 246 |
+
{%- elif _src is string and "<additional_tools>" in _src -%}
|
| 247 |
+
{{- "<tool_response>\n" + _src + "\n</tool_response>" -}}
|
| 248 |
+
{%- elif _src -%}
|
| 249 |
+
{{- "<tool_response>\n<additional_tools>\n" -}}
|
| 250 |
+
{%- if _src is string -%}
|
| 251 |
+
{{- _src -}}
|
| 252 |
+
{%- elif _src is mapping -%}
|
| 253 |
+
{{- tool_to_json(_src) -}}
|
| 254 |
+
{%- else -%}
|
| 255 |
+
{%- for t in _src -%}
|
| 256 |
+
{%- if not loop.first -%}{{- "\n" -}}{%- endif -%}
|
| 257 |
+
{{- tool_to_json(t) -}}
|
| 258 |
+
{%- endfor -%}
|
| 259 |
+
{%- endif -%}
|
| 260 |
+
{{- "\n</additional_tools>\n</tool_response>" -}}
|
| 261 |
+
{%- endif -%}
|
| 262 |
+
{%- elif carry.pending -%}
|
| 263 |
+
{{- render_tool_response(carry.pending + _items) -}}
|
| 264 |
+
{%- set carry.pending = [] -%}
|
| 265 |
+
{%- else -%}
|
| 266 |
+
{{- render_tool_response("" if message.get("content") is none else message.get("content")) -}}
|
| 267 |
+
{%- endif -%}
|
| 268 |
+
{{- "<|im_end|>\n" -}}
|
| 269 |
+
{%- else %}
|
| 270 |
+
{%- if message.get("content") -%}
|
| 271 |
+
{{- parse_content(message["content"]) -}}
|
| 272 |
+
{%- endif -%}
|
| 273 |
+
{{- "<|im_end|>\n" -}}
|
| 274 |
+
{%- endif %}
|
| 275 |
+
{%- endif -%}
|
| 276 |
+
{%- endfor -%}
|
| 277 |
+
{%- if add_generation_prompt -%}
|
| 278 |
+
{{- "<|im_start|>assistant\n" -}}
|
| 279 |
+
{%- if enable_thinking -%}
|
| 280 |
+
{{- "<think>" -}}
|
| 281 |
+
{%- else -%}
|
| 282 |
+
{{- "<think></think>" -}}
|
| 283 |
+
{%- endif -%}
|
| 284 |
+
{%- endif -%}
|
config.json
ADDED
|
@@ -0,0 +1,138 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"Lfm3TsForConditionalGeneration"
|
| 4 |
+
],
|
| 5 |
+
"auto_map": {
|
| 6 |
+
"AutoConfig": "configuration_lfm3_ts.Lfm3TsConfig",
|
| 7 |
+
"AutoModelForCausalLM": "modeling_lfm3_ts.Lfm3TsForConditionalGeneration"
|
| 8 |
+
},
|
| 9 |
+
"dtype": "bfloat16",
|
| 10 |
+
"model_type": "lfm3_ts",
|
| 11 |
+
"projector_bias": true,
|
| 12 |
+
"projector_hidden_act": "gelu",
|
| 13 |
+
"projector_hidden_size": 4096,
|
| 14 |
+
"projector_layer_norm_eps": 1e-06,
|
| 15 |
+
"projector_output_scale": 0.020457027480006218,
|
| 16 |
+
"projector_position_dim": 0,
|
| 17 |
+
"text_config": {
|
| 18 |
+
"_name_or_path": "LiquidAI/LFM3-1.7B",
|
| 19 |
+
"architectures": [
|
| 20 |
+
"Lfm3ForCausalLM"
|
| 21 |
+
],
|
| 22 |
+
"attention_bias": false,
|
| 23 |
+
"attention_dropout": 0.0,
|
| 24 |
+
"auto_map": {
|
| 25 |
+
"AutoConfig": "configuration_lfm3.Lfm3Config",
|
| 26 |
+
"AutoModel": "modeling_lfm3.Lfm3Model",
|
| 27 |
+
"AutoModelForCausalLM": "modeling_lfm3.Lfm3ForCausalLM"
|
| 28 |
+
},
|
| 29 |
+
"bos_token_id": 124894,
|
| 30 |
+
"chunk_size_feed_forward": 0,
|
| 31 |
+
"conv_L_cache": 7,
|
| 32 |
+
"conv_bias": false,
|
| 33 |
+
"dtype": "bfloat16",
|
| 34 |
+
"eos_token_id": 124900,
|
| 35 |
+
"head_dim": 128,
|
| 36 |
+
"hidden_size": 2048,
|
| 37 |
+
"id2label": {
|
| 38 |
+
"0": "LABEL_0",
|
| 39 |
+
"1": "LABEL_1"
|
| 40 |
+
},
|
| 41 |
+
"initializer_range": 0.02,
|
| 42 |
+
"intermediate_size": 7680,
|
| 43 |
+
"is_encoder_decoder": false,
|
| 44 |
+
"label2id": {
|
| 45 |
+
"LABEL_0": 0,
|
| 46 |
+
"LABEL_1": 1
|
| 47 |
+
},
|
| 48 |
+
"layer_types": [
|
| 49 |
+
"conv",
|
| 50 |
+
"conv",
|
| 51 |
+
"full_attention",
|
| 52 |
+
"conv",
|
| 53 |
+
"sliding_attention",
|
| 54 |
+
"conv",
|
| 55 |
+
"full_attention",
|
| 56 |
+
"conv",
|
| 57 |
+
"sliding_attention",
|
| 58 |
+
"conv",
|
| 59 |
+
"full_attention",
|
| 60 |
+
"conv",
|
| 61 |
+
"sliding_attention",
|
| 62 |
+
"conv",
|
| 63 |
+
"full_attention",
|
| 64 |
+
"conv",
|
| 65 |
+
"sliding_attention",
|
| 66 |
+
"conv",
|
| 67 |
+
"full_attention",
|
| 68 |
+
"conv",
|
| 69 |
+
"sliding_attention",
|
| 70 |
+
"conv",
|
| 71 |
+
"full_attention",
|
| 72 |
+
"conv"
|
| 73 |
+
],
|
| 74 |
+
"max_position_embeddings": 262144,
|
| 75 |
+
"model_type": "lfm3",
|
| 76 |
+
"norm_eps": 1e-05,
|
| 77 |
+
"num_attention_heads": 16,
|
| 78 |
+
"num_hidden_layers": 24,
|
| 79 |
+
"num_key_value_heads": 4,
|
| 80 |
+
"output_attentions": false,
|
| 81 |
+
"output_hidden_states": false,
|
| 82 |
+
"pad_token_id": 124893,
|
| 83 |
+
"problem_type": null,
|
| 84 |
+
"return_dict": true,
|
| 85 |
+
"rope_parameters": {
|
| 86 |
+
"full_attention": {
|
| 87 |
+
"partial_rotary_factor": 0.5,
|
| 88 |
+
"rope_theta": 1000000.0,
|
| 89 |
+
"rope_type": "default"
|
| 90 |
+
},
|
| 91 |
+
"sliding_attention": {
|
| 92 |
+
"rope_theta": 10000,
|
| 93 |
+
"rope_type": "default"
|
| 94 |
+
}
|
| 95 |
+
},
|
| 96 |
+
"sliding_window": 256,
|
| 97 |
+
"tie_word_embeddings": true,
|
| 98 |
+
"use_attention_gate": true,
|
| 99 |
+
"use_cache": true,
|
| 100 |
+
"use_sandwich_norm": true,
|
| 101 |
+
"vocab_size": 128000
|
| 102 |
+
},
|
| 103 |
+
"tie_word_embeddings": true,
|
| 104 |
+
"transformers_version": "5.23.0",
|
| 105 |
+
"ts_config": {
|
| 106 |
+
"_name_or_path": "",
|
| 107 |
+
"architectures": null,
|
| 108 |
+
"chunk_size_feed_forward": 0,
|
| 109 |
+
"dtype": "bfloat16",
|
| 110 |
+
"hidden_act": "gelu",
|
| 111 |
+
"hidden_size": 1024,
|
| 112 |
+
"id2label": {
|
| 113 |
+
"0": "LABEL_0",
|
| 114 |
+
"1": "LABEL_1"
|
| 115 |
+
},
|
| 116 |
+
"initializer_range": 0.02,
|
| 117 |
+
"intermediate_size": 2560,
|
| 118 |
+
"is_encoder_decoder": false,
|
| 119 |
+
"label2id": {
|
| 120 |
+
"LABEL_0": 0,
|
| 121 |
+
"LABEL_1": 1
|
| 122 |
+
},
|
| 123 |
+
"model_type": "lfm3_ts_encoder",
|
| 124 |
+
"norm_eps": 1e-05,
|
| 125 |
+
"num_attention_heads": 8,
|
| 126 |
+
"num_attention_layers": 2,
|
| 127 |
+
"num_hidden_layers": 4,
|
| 128 |
+
"output_attentions": false,
|
| 129 |
+
"output_hidden_states": false,
|
| 130 |
+
"patch_size": 8,
|
| 131 |
+
"position_embedding_dim": 16,
|
| 132 |
+
"problem_type": null,
|
| 133 |
+
"return_dict": true,
|
| 134 |
+
"use_differences": false,
|
| 135 |
+
"value_compression": "none"
|
| 136 |
+
},
|
| 137 |
+
"ts_token_id": 125033
|
| 138 |
+
}
|
configuration_lfm3_ts.py
ADDED
|
@@ -0,0 +1,165 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Copyright 2026 The HuggingFace Team. All rights reserved.
|
| 2 |
+
# Copyright 2026 Nardi Xhepi.
|
| 3 |
+
#
|
| 4 |
+
# NOTICE OF CHANGES: this file is part of LFM3-TS. It is adapted from `transformers.models.lfm3_vl`
|
| 5 |
+
# (Hugging Face Transformers, Apache License 2.0) and was changed by Nardi Xhepi: the vision tower and image
|
| 6 |
+
# inputs are replaced by a time-series encoder and time-series inputs.
|
| 7 |
+
#
|
| 8 |
+
# The portions taken from Transformers remain available under the Apache License, Version 2.0; a copy is in
|
| 9 |
+
# LICENSES/Apache-2.0.txt. LFM3-TS as a whole is distributed under the LFM Open License v1.0: see LICENSE and
|
| 10 |
+
# NOTICE at the root of the repository.
|
| 11 |
+
#
|
| 12 |
+
# Unless required by applicable law or agreed to in writing, this software is distributed on an "AS IS" BASIS,
|
| 13 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
| 14 |
+
"""LFM3-TS configuration."""
|
| 15 |
+
|
| 16 |
+
from huggingface_hub.dataclasses import strict
|
| 17 |
+
from transformers.configuration_utils import PreTrainedConfig
|
| 18 |
+
from transformers.models.auto import CONFIG_MAPPING, AutoConfig
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
@strict
|
| 22 |
+
class Lfm3TsEncoderConfig(PreTrainedConfig):
|
| 23 |
+
r"""
|
| 24 |
+
Configuration of the LFM3-TS time-series encoder.
|
| 25 |
+
|
| 26 |
+
Args:
|
| 27 |
+
patch_size (`int`, *optional*, defaults to 8):
|
| 28 |
+
The number of consecutive time steps packed into one patch, i.e. one language token.
|
| 29 |
+
position_embedding_dim (`int`, *optional*, defaults to 16):
|
| 30 |
+
The size of the sinusoidal encoding of each time step's index, given to the encoder next to its value so
|
| 31 |
+
that a patch knows where in the series it sits. Must be even; 0 disables it.
|
| 32 |
+
use_differences (`bool`, *optional*, defaults to `False`):
|
| 33 |
+
Whether each time step also carries its change from the previous step, which makes spikes, shifts and
|
| 34 |
+
slopes explicit.
|
| 35 |
+
value_compression (`str`, *optional*, defaults to `"none"`):
|
| 36 |
+
`"asinh"` passes the standardized values (and differences) through `asinh` before the patch embedding:
|
| 37 |
+
close to the identity for ordinary values, logarithmic for outliers. A spike of 15 standard deviations
|
| 38 |
+
otherwise dominates its patch and drowns the position of the patch along with it.
|
| 39 |
+
hidden_size (`int`, *optional*, defaults to 1024):
|
| 40 |
+
The width of the encoder.
|
| 41 |
+
intermediate_size (`int`, *optional*, defaults to 2560):
|
| 42 |
+
The width of the feed-forward layer inside each encoder layer.
|
| 43 |
+
num_hidden_layers (`int`, *optional*, defaults to 4):
|
| 44 |
+
The number of residual feed-forward layers applied to each patch.
|
| 45 |
+
num_attention_layers (`int`, *optional*, defaults to 0):
|
| 46 |
+
The number of bidirectional self-attention layers, with rotary position embeddings, applied across the
|
| 47 |
+
patches of a series after the feed-forward layers. 0 keeps the patches independent, as in ChatTS.
|
| 48 |
+
num_attention_heads (`int`, *optional*, defaults to 8):
|
| 49 |
+
The number of heads of the attention layers.
|
| 50 |
+
hidden_act (`str`, *optional*, defaults to `"gelu"`):
|
| 51 |
+
The activation of the encoder layers.
|
| 52 |
+
norm_eps (`float`, *optional*, defaults to 1e-05):
|
| 53 |
+
The epsilon used by the normalization layers.
|
| 54 |
+
initializer_range (`float`, *optional*, defaults to 0.02):
|
| 55 |
+
The standard deviation of the weight initializer.
|
| 56 |
+
"""
|
| 57 |
+
|
| 58 |
+
model_type = "lfm3_ts_encoder"
|
| 59 |
+
base_config_key = "ts_config"
|
| 60 |
+
|
| 61 |
+
patch_size: int = 8
|
| 62 |
+
position_embedding_dim: int = 16
|
| 63 |
+
use_differences: bool = False
|
| 64 |
+
value_compression: str = "none"
|
| 65 |
+
hidden_size: int = 1024
|
| 66 |
+
intermediate_size: int = 2560
|
| 67 |
+
num_hidden_layers: int = 4
|
| 68 |
+
num_attention_layers: int = 0
|
| 69 |
+
num_attention_heads: int = 8
|
| 70 |
+
hidden_act: str = "gelu"
|
| 71 |
+
norm_eps: float = 1e-5
|
| 72 |
+
initializer_range: float = 0.02
|
| 73 |
+
|
| 74 |
+
def validate_architecture(self):
|
| 75 |
+
"""Part of ``@strict``-powered validation."""
|
| 76 |
+
if self.patch_size < 1:
|
| 77 |
+
raise ValueError(f"`patch_size` must be positive, got {self.patch_size}.")
|
| 78 |
+
if self.position_embedding_dim < 0 or self.position_embedding_dim % 2:
|
| 79 |
+
raise ValueError(f"`position_embedding_dim` must be even and >= 0, got {self.position_embedding_dim}.")
|
| 80 |
+
if self.value_compression not in ("none", "asinh"):
|
| 81 |
+
raise ValueError(f'`value_compression` must be "none" or "asinh", got {self.value_compression!r}.')
|
| 82 |
+
if self.num_attention_layers and (self.hidden_size // self.num_attention_heads) % 2:
|
| 83 |
+
raise ValueError("`hidden_size // num_attention_heads` must be even for rotary position embeddings.")
|
| 84 |
+
if self.num_attention_layers and self.hidden_size % self.num_attention_heads:
|
| 85 |
+
raise ValueError("`hidden_size` must be a multiple of `num_attention_heads`.")
|
| 86 |
+
|
| 87 |
+
|
| 88 |
+
@strict
|
| 89 |
+
class Lfm3TsConfig(PreTrainedConfig):
|
| 90 |
+
r"""
|
| 91 |
+
Configuration of an LFM3-TS model: an LFM3 language model reading time series through a patch encoder.
|
| 92 |
+
|
| 93 |
+
Args:
|
| 94 |
+
text_config (`dict` or `PreTrainedConfig`, *optional*):
|
| 95 |
+
The configuration of the LFM3 language model. Defaults to `Lfm3Config()`.
|
| 96 |
+
ts_config (`dict` or `Lfm3TsEncoderConfig`, *optional*):
|
| 97 |
+
The configuration of the time-series encoder. Defaults to `Lfm3TsEncoderConfig()`.
|
| 98 |
+
ts_token_id (`int`, *optional*, defaults to 125033):
|
| 99 |
+
The id of the placeholder token whose embeddings are replaced by time-series features. 125033 is the
|
| 100 |
+
first unused row of the LFM3 embedding matrix, so adding the token does not resize the embeddings.
|
| 101 |
+
projector_hidden_act (`str`, *optional*, defaults to `"gelu"`):
|
| 102 |
+
The activation of the multimodal projector.
|
| 103 |
+
projector_hidden_size (`int`, *optional*, defaults to 4096):
|
| 104 |
+
The hidden width of the multimodal projector.
|
| 105 |
+
projector_bias (`bool`, *optional*, defaults to `True`):
|
| 106 |
+
Whether the projector's linear layers have a bias.
|
| 107 |
+
projector_layer_norm_eps (`float`, *optional*, defaults to 1e-06):
|
| 108 |
+
The epsilon of the layer normalizations of the projector.
|
| 109 |
+
projector_position_dim (`int`, *optional*, defaults to 0):
|
| 110 |
+
The size of a sinusoidal encoding of each patch's first index, projected and added to the patch's token
|
| 111 |
+
after the content path and its normalization. Whatever the patch holds, its token then states where it
|
| 112 |
+
sits. Must be even; 0 disables it.
|
| 113 |
+
projector_output_scale (`float`, *optional*, defaults to 0.02):
|
| 114 |
+
The per-dimension scale the projected features start at. It should equal that of the language model's
|
| 115 |
+
token embeddings (0.0207 for LFM3-1.7B), which enter the residual stream unnormalized: features a few
|
| 116 |
+
times larger drown the layers' updates at their positions.
|
| 117 |
+
"""
|
| 118 |
+
|
| 119 |
+
model_type = "lfm3_ts"
|
| 120 |
+
sub_configs = {"text_config": AutoConfig, "ts_config": Lfm3TsEncoderConfig}
|
| 121 |
+
|
| 122 |
+
text_config: dict | PreTrainedConfig | None = None
|
| 123 |
+
ts_config: dict | PreTrainedConfig | None = None
|
| 124 |
+
ts_token_id: int = 125_033
|
| 125 |
+
projector_hidden_act: str = "gelu"
|
| 126 |
+
projector_hidden_size: int = 4096
|
| 127 |
+
projector_bias: bool = True
|
| 128 |
+
projector_layer_norm_eps: float = 1e-6
|
| 129 |
+
projector_position_dim: int = 0
|
| 130 |
+
projector_output_scale: float = 0.02
|
| 131 |
+
tie_word_embeddings: bool = True
|
| 132 |
+
|
| 133 |
+
def __post_init__(self, **kwargs):
|
| 134 |
+
if "lfm3" not in CONFIG_MAPPING:
|
| 135 |
+
import transformers
|
| 136 |
+
|
| 137 |
+
raise ImportError(
|
| 138 |
+
f"LFM3-TS needs transformers 5.23 or later, the first release that includes LFM3, and found "
|
| 139 |
+
f"{transformers.__version__}. Run `pip install -U \"transformers>=5.23\"` and restart the runtime."
|
| 140 |
+
)
|
| 141 |
+
if isinstance(self.ts_config, dict):
|
| 142 |
+
self.ts_config = self.sub_configs["ts_config"](**self.ts_config)
|
| 143 |
+
elif self.ts_config is None:
|
| 144 |
+
self.ts_config = self.sub_configs["ts_config"]()
|
| 145 |
+
|
| 146 |
+
if isinstance(self.text_config, dict):
|
| 147 |
+
self.text_config = {"model_type": "lfm3", **self.text_config}
|
| 148 |
+
self.text_config = CONFIG_MAPPING[self.text_config["model_type"]](**self.text_config)
|
| 149 |
+
elif self.text_config is None:
|
| 150 |
+
self.text_config = CONFIG_MAPPING["lfm3"]()
|
| 151 |
+
self.validate_ts_fields()
|
| 152 |
+
|
| 153 |
+
super().__post_init__(**kwargs)
|
| 154 |
+
|
| 155 |
+
def validate_ts_fields(self):
|
| 156 |
+
"""Part of ``@strict``-powered validation, so `validate()` and `save_pretrained` refuse later edits too."""
|
| 157 |
+
# an `inputs_embeds` forward embeds the placeholder id to find it
|
| 158 |
+
vocab_size = self.text_config.vocab_size
|
| 159 |
+
if self.projector_position_dim < 0 or self.projector_position_dim % 2:
|
| 160 |
+
raise ValueError(f"`projector_position_dim` must be even and >= 0, got {self.projector_position_dim}.")
|
| 161 |
+
if not 0 <= self.ts_token_id < vocab_size:
|
| 162 |
+
raise ValueError(f"ts_token_id={self.ts_token_id} must lie in [0, text_config.vocab_size={vocab_size})")
|
| 163 |
+
|
| 164 |
+
|
| 165 |
+
__all__ = ["Lfm3TsConfig", "Lfm3TsEncoderConfig"]
|
feature_extraction_lfm3_ts.py
ADDED
|
@@ -0,0 +1,116 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Copyright 2026 Nardi Xhepi.
|
| 2 |
+
#
|
| 3 |
+
# This file is part of LFM3-TS, distributed under the LFM Open License v1.0: see LICENSE and NOTICE at the root
|
| 4 |
+
# of the repository. Unless required by applicable law or agreed to in writing, this software is distributed on
|
| 5 |
+
# an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
| 6 |
+
"""Feature extractor for LFM3-TS: normalizes each series and cuts it into patches."""
|
| 7 |
+
|
| 8 |
+
from dataclasses import dataclass
|
| 9 |
+
|
| 10 |
+
import numpy as np
|
| 11 |
+
from transformers.feature_extraction_utils import BatchFeature, FeatureExtractionMixin
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
@dataclass
|
| 15 |
+
class Lfm3TsSeriesStats:
|
| 16 |
+
"""What normalization removed from a series; the processor writes it into the prompt so values stay recoverable."""
|
| 17 |
+
|
| 18 |
+
offset: float
|
| 19 |
+
scale: float
|
| 20 |
+
length: int
|
| 21 |
+
num_patches: int
|
| 22 |
+
minimum: float
|
| 23 |
+
maximum: float
|
| 24 |
+
first: float
|
| 25 |
+
last: float
|
| 26 |
+
|
| 27 |
+
|
| 28 |
+
class Lfm3TsFeatureExtractor(FeatureExtractionMixin):
|
| 29 |
+
r"""
|
| 30 |
+
Turns univariate series into the `input_values` and `input_values_mask` of an LFM3-TS model.
|
| 31 |
+
|
| 32 |
+
Each series is standardized with its own mean and standard deviation over its observed values, then cut into
|
| 33 |
+
patches of `patch_size` time steps. NaN marks a missing value: it is zeroed and masked out, as is the padding
|
| 34 |
+
that fills the last patch and aligns series of different lengths.
|
| 35 |
+
|
| 36 |
+
Args:
|
| 37 |
+
patch_size (`int`, *optional*, defaults to 8):
|
| 38 |
+
The number of time steps per patch. Must equal the model's `ts_config.patch_size`.
|
| 39 |
+
eps (`float`, *optional*, defaults to 1e-06):
|
| 40 |
+
Series whose standard deviation is below `eps` times their magnitude are treated as constant and left
|
| 41 |
+
unscaled.
|
| 42 |
+
"""
|
| 43 |
+
|
| 44 |
+
model_input_names = ["input_values", "input_values_mask"]
|
| 45 |
+
|
| 46 |
+
def __init__(self, patch_size: int = 8, eps: float = 1e-6, **kwargs):
|
| 47 |
+
super().__init__(**kwargs)
|
| 48 |
+
self.patch_size = patch_size
|
| 49 |
+
self.eps = eps
|
| 50 |
+
|
| 51 |
+
def get_num_patches(self, length: int) -> int:
|
| 52 |
+
"""Number of patches covering `length` time steps."""
|
| 53 |
+
return -(-length // self.patch_size)
|
| 54 |
+
|
| 55 |
+
def preprocess(self, timeseries: list) -> tuple[np.ndarray, np.ndarray, list[Lfm3TsSeriesStats]]:
|
| 56 |
+
"""Normalizes and patches a list of 1D series.
|
| 57 |
+
|
| 58 |
+
Returns:
|
| 59 |
+
The values and the observation mask, both of shape `(num_series, max_patches, patch_size)`, and the
|
| 60 |
+
statistics of each series.
|
| 61 |
+
"""
|
| 62 |
+
normalized, stats = [], []
|
| 63 |
+
for index, series in enumerate(timeseries):
|
| 64 |
+
series = np.asarray(series, dtype=np.float64)
|
| 65 |
+
if series.ndim != 1:
|
| 66 |
+
raise ValueError(f"Series {index} must be one-dimensional, got shape {series.shape}.")
|
| 67 |
+
observed = np.isfinite(series)
|
| 68 |
+
if not observed.any():
|
| 69 |
+
raise ValueError(f"Series {index} has no observed value.")
|
| 70 |
+
# trailing missing values carry no information: the series ends at its last observed step
|
| 71 |
+
length = int(np.flatnonzero(observed)[-1]) + 1
|
| 72 |
+
series, observed = series[:length], observed[:length]
|
| 73 |
+
|
| 74 |
+
offset = float(series[observed].mean())
|
| 75 |
+
scale = float(series[observed].std())
|
| 76 |
+
if scale <= self.eps * max(1.0, abs(offset)):
|
| 77 |
+
scale = 1.0
|
| 78 |
+
values = np.where(observed, (series - offset) / scale, 0.0)
|
| 79 |
+
normalized.append((values, observed))
|
| 80 |
+
seen = series[observed]
|
| 81 |
+
stats.append(
|
| 82 |
+
Lfm3TsSeriesStats(
|
| 83 |
+
offset=offset,
|
| 84 |
+
scale=scale,
|
| 85 |
+
length=length,
|
| 86 |
+
num_patches=self.get_num_patches(length),
|
| 87 |
+
minimum=float(seen.min()),
|
| 88 |
+
maximum=float(seen.max()),
|
| 89 |
+
first=float(seen[0]),
|
| 90 |
+
last=float(seen[-1]),
|
| 91 |
+
)
|
| 92 |
+
)
|
| 93 |
+
|
| 94 |
+
max_patches = max(stat.num_patches for stat in stats)
|
| 95 |
+
input_values = np.zeros((len(stats), max_patches * self.patch_size), dtype=np.float32)
|
| 96 |
+
input_values_mask = np.zeros_like(input_values)
|
| 97 |
+
for index, (values, observed) in enumerate(normalized):
|
| 98 |
+
input_values[index, : len(values)] = values
|
| 99 |
+
input_values_mask[index, : len(values)] = observed
|
| 100 |
+
shape = (len(stats), max_patches, self.patch_size)
|
| 101 |
+
return input_values.reshape(shape), input_values_mask.reshape(shape), stats
|
| 102 |
+
|
| 103 |
+
def __call__(self, timeseries: list, return_tensors: str | None = None, **kwargs) -> BatchFeature:
|
| 104 |
+
"""
|
| 105 |
+
Args:
|
| 106 |
+
timeseries (`list`):
|
| 107 |
+
One-dimensional series (arrays or lists of numbers), possibly of different lengths.
|
| 108 |
+
return_tensors (`str`, *optional*):
|
| 109 |
+
The tensor type to return, e.g. `"pt"`.
|
| 110 |
+
"""
|
| 111 |
+
input_values, input_values_mask, _ = self.preprocess(timeseries)
|
| 112 |
+
data = {"input_values": input_values, "input_values_mask": input_values_mask}
|
| 113 |
+
return BatchFeature(data=data, tensor_type=return_tensors)
|
| 114 |
+
|
| 115 |
+
|
| 116 |
+
__all__ = ["Lfm3TsFeatureExtractor", "Lfm3TsSeriesStats"]
|
generation_config.json
ADDED
|
@@ -0,0 +1,16 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"_from_model_config": true,
|
| 3 |
+
"bos_token_id": 124894,
|
| 4 |
+
"do_sample": true,
|
| 5 |
+
"eos_token_id": [
|
| 6 |
+
124900
|
| 7 |
+
],
|
| 8 |
+
"output_attentions": false,
|
| 9 |
+
"output_hidden_states": false,
|
| 10 |
+
"pad_token_id": 124893,
|
| 11 |
+
"repetition_penalty": 1.1,
|
| 12 |
+
"temperature": 0.1,
|
| 13 |
+
"top_k": 50,
|
| 14 |
+
"transformers_version": "5.23.0",
|
| 15 |
+
"use_cache": true
|
| 16 |
+
}
|
model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e11577553e71d7be7e6c6c96d787642bf6838ae5efb41f30b810c6ab77fe0843
|
| 3 |
+
size 3562909848
|
modeling_lfm3_ts.py
ADDED
|
@@ -0,0 +1,629 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Copyright 2026 The HuggingFace Team. All rights reserved.
|
| 2 |
+
# Copyright 2026 Nardi Xhepi.
|
| 3 |
+
#
|
| 4 |
+
# NOTICE OF CHANGES: this file is part of LFM3-TS. It is adapted from `transformers.models.lfm3_vl`
|
| 5 |
+
# (Hugging Face Transformers, Apache License 2.0) and was changed by Nardi Xhepi: the vision tower and image
|
| 6 |
+
# inputs are replaced by a time-series encoder and time-series inputs.
|
| 7 |
+
#
|
| 8 |
+
# The portions taken from Transformers remain available under the Apache License, Version 2.0; a copy is in
|
| 9 |
+
# LICENSES/Apache-2.0.txt. LFM3-TS as a whole is distributed under the LFM Open License v1.0: see LICENSE and
|
| 10 |
+
# NOTICE at the root of the repository.
|
| 11 |
+
#
|
| 12 |
+
# Unless required by applicable law or agreed to in writing, this software is distributed on an "AS IS" BASIS,
|
| 13 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
| 14 |
+
"""LFM3-TS model: a time-series tower and a projector in front of the LFM3 language model."""
|
| 15 |
+
|
| 16 |
+
import math
|
| 17 |
+
from dataclasses import dataclass
|
| 18 |
+
|
| 19 |
+
import numpy as np
|
| 20 |
+
import torch
|
| 21 |
+
import torch.nn.functional as F
|
| 22 |
+
from torch import nn
|
| 23 |
+
from transformers import initialization as init
|
| 24 |
+
from transformers.activations import ACT2FN
|
| 25 |
+
from transformers.cache_utils import Cache
|
| 26 |
+
from transformers.generation import GenerationMixin
|
| 27 |
+
from transformers.modeling_outputs import BaseModelOutputWithPast, BaseModelOutputWithPooling, CausalLMOutputWithPast
|
| 28 |
+
from transformers.modeling_utils import PreTrainedModel
|
| 29 |
+
from transformers.models.lfm3.modeling_lfm3 import Lfm3Model
|
| 30 |
+
from transformers.utils import can_return_tuple, torch_compilable_check
|
| 31 |
+
|
| 32 |
+
from .configuration_lfm3_ts import Lfm3TsConfig, Lfm3TsEncoderConfig
|
| 33 |
+
|
| 34 |
+
|
| 35 |
+
def get_ts_patch_counts(input_values_mask: torch.Tensor) -> torch.Tensor:
|
| 36 |
+
"""Number of patches each series occupies: up to and including its last patch holding an observed time step.
|
| 37 |
+
|
| 38 |
+
Args:
|
| 39 |
+
input_values_mask (`torch.Tensor` of shape `(num_series, max_patches, patch_size)`):
|
| 40 |
+
1 for observed time steps, 0 for padding and missing values.
|
| 41 |
+
|
| 42 |
+
Returns:
|
| 43 |
+
`torch.LongTensor` of shape `(num_series,)`.
|
| 44 |
+
"""
|
| 45 |
+
observed = input_values_mask.bool().any(-1)
|
| 46 |
+
index = torch.arange(1, observed.shape[1] + 1, device=observed.device)
|
| 47 |
+
return (observed * index).amax(-1)
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
def _as_series_list(series) -> list[np.ndarray]:
|
| 51 |
+
"""One 1D array per series, from a 1D array or list of numbers, a list of them, or a 2D array of rows."""
|
| 52 |
+
if hasattr(series, "detach"):
|
| 53 |
+
series = series.detach().cpu().numpy()
|
| 54 |
+
if hasattr(series, "to_numpy"):
|
| 55 |
+
series = series.to_numpy()
|
| 56 |
+
if isinstance(series, np.ndarray):
|
| 57 |
+
items = [series] if series.ndim == 1 else list(series)
|
| 58 |
+
elif isinstance(series, (list, tuple)) and len(series) and all(
|
| 59 |
+
isinstance(value, (int, float, np.number)) for value in series
|
| 60 |
+
):
|
| 61 |
+
items = [series]
|
| 62 |
+
elif isinstance(series, (list, tuple)):
|
| 63 |
+
items = list(series)
|
| 64 |
+
else:
|
| 65 |
+
raise ValueError(f"`series` must be an array or a list of numbers, or a list of them, got {type(series)}.")
|
| 66 |
+
arrays = []
|
| 67 |
+
for index, item in enumerate(items):
|
| 68 |
+
if hasattr(item, "detach"):
|
| 69 |
+
item = item.detach().cpu().numpy()
|
| 70 |
+
array = np.asarray(item, dtype=np.float64)
|
| 71 |
+
if array.ndim != 1 or not array.size:
|
| 72 |
+
raise ValueError(f"Series {index} must be one-dimensional and not empty, got shape {array.shape}.")
|
| 73 |
+
arrays.append(array)
|
| 74 |
+
if not arrays:
|
| 75 |
+
raise ValueError("`series` is empty.")
|
| 76 |
+
return arrays
|
| 77 |
+
|
| 78 |
+
|
| 79 |
+
def build_chat_messages(series, question: str) -> list[dict]:
|
| 80 |
+
"""The conversation `Lfm3TsForConditionalGeneration.chat` sends: one user message holding the series and the
|
| 81 |
+
question, in the wording the released model was trained on for one series
|
| 82 |
+
(`"This is a time series of length N: "`, the series, `". "`, the question).
|
| 83 |
+
|
| 84 |
+
Args:
|
| 85 |
+
series:
|
| 86 |
+
A 1D array or list of numbers; or several series, as a list of them or as the rows of a 2D array; or
|
| 87 |
+
`None` for a question without a series. NaN marks a missing value.
|
| 88 |
+
question (`str`):
|
| 89 |
+
What to ask.
|
| 90 |
+
"""
|
| 91 |
+
question = question.strip()
|
| 92 |
+
if series is None:
|
| 93 |
+
return [{"role": "user", "content": [{"type": "text", "text": question}]}]
|
| 94 |
+
arrays = _as_series_list(series)
|
| 95 |
+
if len(arrays) == 1:
|
| 96 |
+
content = [
|
| 97 |
+
{"type": "text", "text": f"This is a time series of length {len(arrays[0])}: "},
|
| 98 |
+
{"type": "timeseries", "timeseries": arrays[0]},
|
| 99 |
+
]
|
| 100 |
+
else:
|
| 101 |
+
content = [{"type": "text", "text": f"There are {len(arrays)} time series. "}]
|
| 102 |
+
for index, array in enumerate(arrays):
|
| 103 |
+
if index:
|
| 104 |
+
content.append({"type": "text", "text": "; "})
|
| 105 |
+
content.append({"type": "text", "text": f"Series {index + 1}, of length {len(array)}: "})
|
| 106 |
+
content.append({"type": "timeseries", "timeseries": array})
|
| 107 |
+
content.append({"type": "text", "text": f". {question}"})
|
| 108 |
+
return [{"role": "user", "content": content}]
|
| 109 |
+
|
| 110 |
+
|
| 111 |
+
class Lfm3TsEncoderLayer(nn.Module):
|
| 112 |
+
"""A pre-norm residual feed-forward layer applied to each patch on its own."""
|
| 113 |
+
|
| 114 |
+
def __init__(self, config: Lfm3TsEncoderConfig):
|
| 115 |
+
super().__init__()
|
| 116 |
+
self.norm = nn.LayerNorm(config.hidden_size, eps=config.norm_eps)
|
| 117 |
+
self.fc1 = nn.Linear(config.hidden_size, config.intermediate_size)
|
| 118 |
+
self.act = ACT2FN[config.hidden_act]
|
| 119 |
+
self.fc2 = nn.Linear(config.intermediate_size, config.hidden_size)
|
| 120 |
+
|
| 121 |
+
def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
|
| 122 |
+
return hidden_states + self.fc2(self.act(self.fc1(self.norm(hidden_states))))
|
| 123 |
+
|
| 124 |
+
|
| 125 |
+
def apply_rotary_position_embedding(states: torch.Tensor, cos: torch.Tensor, sin: torch.Tensor) -> torch.Tensor:
|
| 126 |
+
"""Rotates each pair of channels of `(batch, heads, patches, head_dim)` states by its patch's angle."""
|
| 127 |
+
even, odd = states[..., 0::2], states[..., 1::2]
|
| 128 |
+
return torch.stack([even * cos - odd * sin, even * sin + odd * cos], dim=-1).flatten(-2)
|
| 129 |
+
|
| 130 |
+
|
| 131 |
+
class Lfm3TsAttentionLayer(nn.Module):
|
| 132 |
+
"""A pre-norm bidirectional self-attention layer over the patches of one series, with rotary position
|
| 133 |
+
embeddings, so that a patch can read its context: a period or a trend spans several patches."""
|
| 134 |
+
|
| 135 |
+
def __init__(self, config: Lfm3TsEncoderConfig):
|
| 136 |
+
super().__init__()
|
| 137 |
+
self.num_heads = config.num_attention_heads
|
| 138 |
+
self.head_dim = config.hidden_size // config.num_attention_heads
|
| 139 |
+
self.attention_norm = nn.LayerNorm(config.hidden_size, eps=config.norm_eps)
|
| 140 |
+
self.q_proj = nn.Linear(config.hidden_size, config.hidden_size, bias=False)
|
| 141 |
+
self.k_proj = nn.Linear(config.hidden_size, config.hidden_size, bias=False)
|
| 142 |
+
self.v_proj = nn.Linear(config.hidden_size, config.hidden_size, bias=False)
|
| 143 |
+
self.o_proj = nn.Linear(config.hidden_size, config.hidden_size, bias=False)
|
| 144 |
+
self.feed_forward = Lfm3TsEncoderLayer(config)
|
| 145 |
+
|
| 146 |
+
def forward(
|
| 147 |
+
self, hidden_states: torch.Tensor, cos: torch.Tensor, sin: torch.Tensor, attention_mask: torch.Tensor
|
| 148 |
+
) -> torch.Tensor:
|
| 149 |
+
num_series, num_patches, hidden_size = hidden_states.shape
|
| 150 |
+
states = self.attention_norm(hidden_states)
|
| 151 |
+
split = lambda tensor: tensor.view(num_series, num_patches, self.num_heads, self.head_dim).transpose(1, 2)
|
| 152 |
+
queries = apply_rotary_position_embedding(split(self.q_proj(states)), cos, sin)
|
| 153 |
+
keys = apply_rotary_position_embedding(split(self.k_proj(states)), cos, sin)
|
| 154 |
+
attended = F.scaled_dot_product_attention(queries, keys, split(self.v_proj(states)), attn_mask=attention_mask)
|
| 155 |
+
attended = attended.transpose(1, 2).reshape(num_series, num_patches, hidden_size)
|
| 156 |
+
return self.feed_forward(hidden_states + self.o_proj(attended))
|
| 157 |
+
|
| 158 |
+
|
| 159 |
+
class Lfm3TsPreTrainedModel(PreTrainedModel):
|
| 160 |
+
config: Lfm3TsConfig
|
| 161 |
+
base_model_prefix = "model"
|
| 162 |
+
supports_gradient_checkpointing = True
|
| 163 |
+
_skip_keys_device_placement = ["past_key_values"]
|
| 164 |
+
|
| 165 |
+
_supports_flash_attn = True
|
| 166 |
+
_supports_sdpa = True
|
| 167 |
+
_supports_flex_attn = True
|
| 168 |
+
_supports_attention_backend = True
|
| 169 |
+
|
| 170 |
+
def _init_weights(self, module):
|
| 171 |
+
super()._init_weights(module)
|
| 172 |
+
if isinstance(module, Lfm3TsMultiModalProjector):
|
| 173 |
+
scale = module.output_scale
|
| 174 |
+
init.normal_(module.boundary_embeddings, mean=0.0, std=scale)
|
| 175 |
+
init.constant_(module.output_norm.weight, scale)
|
| 176 |
+
init.zeros_(module.output_norm.bias)
|
| 177 |
+
if module.position_embed is not None:
|
| 178 |
+
# sinusoids have a mean square of 1/2: the position starts at the scale of the content
|
| 179 |
+
init.normal_(module.position_embed.weight, mean=0.0, std=scale / math.sqrt(module.position_dim / 2))
|
| 180 |
+
|
| 181 |
+
|
| 182 |
+
class Lfm3TsEncoder(Lfm3TsPreTrainedModel):
|
| 183 |
+
"""The LFM3-TS time-series encoder: a linear patch embedding over the values, observation mask and position of
|
| 184 |
+
each time step of a patch, followed by residual feed-forward layers. With `num_attention_layers=0` patches are
|
| 185 |
+
encoded independently and only the language model relates them; attention layers let them read each other."""
|
| 186 |
+
|
| 187 |
+
config: Lfm3TsEncoderConfig
|
| 188 |
+
main_input_name = "input_values"
|
| 189 |
+
_input_embed_layer = "patch_embed"
|
| 190 |
+
|
| 191 |
+
def __init__(self, config: Lfm3TsEncoderConfig):
|
| 192 |
+
super().__init__(config)
|
| 193 |
+
# each time step carries its value, its observation mask and the encoding of its index in the series
|
| 194 |
+
channels = 2 + int(config.use_differences) + config.position_embedding_dim
|
| 195 |
+
self.patch_embed = nn.Linear(channels * config.patch_size, config.hidden_size)
|
| 196 |
+
self.layers = nn.ModuleList([Lfm3TsEncoderLayer(config) for _ in range(config.num_hidden_layers)])
|
| 197 |
+
self.attention_layers = nn.ModuleList(
|
| 198 |
+
[Lfm3TsAttentionLayer(config) for _ in range(config.num_attention_layers)]
|
| 199 |
+
)
|
| 200 |
+
self.post_init()
|
| 201 |
+
|
| 202 |
+
def forward(
|
| 203 |
+
self, input_values: torch.Tensor, input_values_mask: torch.Tensor | None = None, **kwargs
|
| 204 |
+
) -> BaseModelOutputWithPooling:
|
| 205 |
+
r"""
|
| 206 |
+
input_values (`torch.FloatTensor` of shape `(num_series, max_patches, patch_size)`):
|
| 207 |
+
The normalized values of every series, cut into patches.
|
| 208 |
+
input_values_mask (`torch.Tensor` of shape `(num_series, max_patches, patch_size)`, *optional*):
|
| 209 |
+
1 for observed time steps, 0 for padding and missing values. Defaults to all observed.
|
| 210 |
+
"""
|
| 211 |
+
if input_values_mask is None:
|
| 212 |
+
input_values_mask = torch.ones_like(input_values)
|
| 213 |
+
dtype = self.patch_embed.weight.dtype
|
| 214 |
+
# outliers are compressed so that no value outweighs the rest of its patch
|
| 215 |
+
compress = torch.asinh if self.config.value_compression == "asinh" else (lambda tensor: tensor)
|
| 216 |
+
patches = [compress(input_values.to(dtype)), input_values_mask.to(dtype)]
|
| 217 |
+
if self.config.use_differences:
|
| 218 |
+
# the change from the previous step, across patch borders; zero where either step is unobserved
|
| 219 |
+
values, observed = input_values.to(dtype).flatten(1), input_values_mask.to(dtype).flatten(1)
|
| 220 |
+
differences = torch.zeros_like(values)
|
| 221 |
+
differences[:, 1:] = (values[:, 1:] - values[:, :-1]) * observed[:, 1:] * observed[:, :-1]
|
| 222 |
+
patches.append(compress(differences).view_as(input_values))
|
| 223 |
+
if self.config.position_embedding_dim:
|
| 224 |
+
patches.append(self.get_position_features(input_values_mask).to(dtype))
|
| 225 |
+
hidden_states = self.patch_embed(torch.cat(patches, dim=-1))
|
| 226 |
+
for layer in self.layers:
|
| 227 |
+
hidden_states = layer(hidden_states)
|
| 228 |
+
|
| 229 |
+
if self.attention_layers:
|
| 230 |
+
num_patches = hidden_states.shape[1]
|
| 231 |
+
head_dim = self.config.hidden_size // self.config.num_attention_heads
|
| 232 |
+
device = hidden_states.device
|
| 233 |
+
frequencies = 1.0 / (10000.0 ** (torch.arange(0, head_dim, 2, device=device) / head_dim))
|
| 234 |
+
angles = torch.arange(num_patches, device=device)[:, None] * frequencies[None, :]
|
| 235 |
+
cos, sin = angles.cos().to(dtype), angles.sin().to(dtype)
|
| 236 |
+
# a patch attends to the patches of its own series, not to the padding that aligns the batch
|
| 237 |
+
is_patch = torch.arange(num_patches, device=device)[None, :] < get_ts_patch_counts(input_values_mask)[:, None]
|
| 238 |
+
attention_mask = is_patch[:, None, None, :]
|
| 239 |
+
for layer in self.attention_layers:
|
| 240 |
+
hidden_states = layer(hidden_states, cos, sin, attention_mask)
|
| 241 |
+
return BaseModelOutputWithPooling(last_hidden_state=hidden_states)
|
| 242 |
+
|
| 243 |
+
|
| 244 |
+
def get_position_features(self, input_values_mask: torch.Tensor) -> torch.Tensor:
|
| 245 |
+
"""Sinusoidal encoding of the index of every observed time step, zero elsewhere.
|
| 246 |
+
|
| 247 |
+
Without it a patch holds no trace of where it sits, and the language model could only locate an event by
|
| 248 |
+
counting tokens. Sinusoids need no training and extend to any length.
|
| 249 |
+
|
| 250 |
+
Returns:
|
| 251 |
+
`torch.FloatTensor` of shape `(num_series, max_patches, patch_size * position_embedding_dim)`.
|
| 252 |
+
"""
|
| 253 |
+
num_series, max_patches, patch_size = input_values_mask.shape
|
| 254 |
+
half = self.config.position_embedding_dim // 2
|
| 255 |
+
device = input_values_mask.device
|
| 256 |
+
positions = torch.arange(max_patches * patch_size, device=device, dtype=torch.float32)
|
| 257 |
+
frequencies = torch.exp(-math.log(10000.0) * torch.arange(half, device=device) / half)
|
| 258 |
+
angles = positions[:, None] * frequencies[None, :]
|
| 259 |
+
features = torch.cat([angles.sin(), angles.cos()], dim=-1).view(max_patches, patch_size, 2 * half)
|
| 260 |
+
features = features[None] * input_values_mask[..., None].to(features.dtype)
|
| 261 |
+
return features.flatten(2)
|
| 262 |
+
|
| 263 |
+
|
| 264 |
+
class Lfm3TsMultiModalProjector(nn.Module):
|
| 265 |
+
"""Projects each patch to one language token and frames every series with a learned start and end embedding.
|
| 266 |
+
|
| 267 |
+
The boundaries live here, not in the vocabulary, so that every parameter introduced for time series sits outside
|
| 268 |
+
the language model and the backbone can stay frozen."""
|
| 269 |
+
|
| 270 |
+
def __init__(self, config: Lfm3TsConfig):
|
| 271 |
+
super().__init__()
|
| 272 |
+
self.layer_norm = nn.LayerNorm(config.ts_config.hidden_size, eps=config.projector_layer_norm_eps)
|
| 273 |
+
self.linear_1 = nn.Linear(
|
| 274 |
+
config.ts_config.hidden_size, config.projector_hidden_size, bias=config.projector_bias
|
| 275 |
+
)
|
| 276 |
+
self.act = ACT2FN[config.projector_hidden_act]
|
| 277 |
+
self.linear_2 = nn.Linear(
|
| 278 |
+
config.projector_hidden_size, config.text_config.hidden_size, bias=config.projector_bias
|
| 279 |
+
)
|
| 280 |
+
# brings the features to the scale of the token embeddings they sit among; its weight starts at that scale
|
| 281 |
+
self.output_norm = nn.LayerNorm(config.text_config.hidden_size, eps=config.projector_layer_norm_eps)
|
| 282 |
+
self.output_scale = config.projector_output_scale
|
| 283 |
+
# the position of a patch joins its token after the normalization, where the content cannot outweigh it
|
| 284 |
+
self.patch_size = config.ts_config.patch_size
|
| 285 |
+
self.position_dim = config.projector_position_dim
|
| 286 |
+
self.position_embed = (
|
| 287 |
+
nn.Linear(self.position_dim, config.text_config.hidden_size, bias=False) if self.position_dim else None
|
| 288 |
+
)
|
| 289 |
+
self.boundary_embeddings = nn.Parameter(torch.empty(2, config.text_config.hidden_size))
|
| 290 |
+
|
| 291 |
+
def forward(self, ts_features: torch.Tensor) -> torch.Tensor:
|
| 292 |
+
hidden_states = self.layer_norm(ts_features)
|
| 293 |
+
hidden_states = self.linear_1(hidden_states)
|
| 294 |
+
hidden_states = self.act(hidden_states)
|
| 295 |
+
hidden_states = self.linear_2(hidden_states)
|
| 296 |
+
hidden_states = self.output_norm(hidden_states)
|
| 297 |
+
if self.position_embed is not None:
|
| 298 |
+
hidden_states = hidden_states + self.position_embed(self.get_position_features(hidden_states))
|
| 299 |
+
return hidden_states
|
| 300 |
+
|
| 301 |
+
def get_position_features(self, hidden_states: torch.Tensor) -> torch.Tensor:
|
| 302 |
+
"""Sinusoidal encoding of the index of the first time step of each patch, of shape `(num_patches, position_dim)`."""
|
| 303 |
+
half = self.position_dim // 2
|
| 304 |
+
device = hidden_states.device
|
| 305 |
+
starts = torch.arange(hidden_states.shape[1], device=device, dtype=torch.float32) * self.patch_size
|
| 306 |
+
frequencies = torch.exp(-math.log(10000.0) * torch.arange(half, device=device) / half)
|
| 307 |
+
angles = starts[:, None] * frequencies[None, :]
|
| 308 |
+
return torch.cat([angles.sin(), angles.cos()], dim=-1).to(hidden_states.dtype)
|
| 309 |
+
|
| 310 |
+
|
| 311 |
+
@dataclass
|
| 312 |
+
class Lfm3TsModelOutputWithPast(BaseModelOutputWithPast):
|
| 313 |
+
r"""
|
| 314 |
+
ts_hidden_states (`torch.FloatTensor`, *optional*):
|
| 315 |
+
A `torch.FloatTensor` of size `(num_ts_tokens, hidden_size)`: the projected time-series features that
|
| 316 |
+
replaced the placeholder tokens.
|
| 317 |
+
"""
|
| 318 |
+
|
| 319 |
+
ts_hidden_states: torch.FloatTensor | None = None
|
| 320 |
+
|
| 321 |
+
|
| 322 |
+
@dataclass
|
| 323 |
+
class Lfm3TsCausalLMOutputWithPast(CausalLMOutputWithPast):
|
| 324 |
+
r"""
|
| 325 |
+
ts_hidden_states (`torch.FloatTensor`, *optional*):
|
| 326 |
+
A `torch.FloatTensor` of size `(num_ts_tokens, hidden_size)`: the projected time-series features that
|
| 327 |
+
replaced the placeholder tokens.
|
| 328 |
+
"""
|
| 329 |
+
|
| 330 |
+
ts_hidden_states: torch.FloatTensor | None = None
|
| 331 |
+
|
| 332 |
+
|
| 333 |
+
class Lfm3TsModel(Lfm3TsPreTrainedModel):
|
| 334 |
+
"""The LFM3-TS model: a time-series encoder and an LFM3 language model, without a language modeling head."""
|
| 335 |
+
|
| 336 |
+
def __init__(self, config: Lfm3TsConfig):
|
| 337 |
+
super().__init__(config)
|
| 338 |
+
self.ts_tower = Lfm3TsEncoder(config.ts_config)
|
| 339 |
+
self.multi_modal_projector = Lfm3TsMultiModalProjector(config)
|
| 340 |
+
self.language_model = Lfm3Model(config.text_config)
|
| 341 |
+
self.post_init()
|
| 342 |
+
|
| 343 |
+
def get_input_embeddings(self) -> nn.Module:
|
| 344 |
+
return self.language_model.get_input_embeddings()
|
| 345 |
+
|
| 346 |
+
def set_input_embeddings(self, value: nn.Module):
|
| 347 |
+
self.language_model.set_input_embeddings(value)
|
| 348 |
+
|
| 349 |
+
def get_ts_features(
|
| 350 |
+
self, input_values: torch.Tensor, input_values_mask: torch.Tensor | None = None, **kwargs
|
| 351 |
+
) -> BaseModelOutputWithPooling:
|
| 352 |
+
r"""
|
| 353 |
+
Encodes and projects the series. `pooler_output` holds one tensor per series of shape
|
| 354 |
+
`(num_patches + 2, hidden_size)`: a start embedding, one token per patch, and an end embedding.
|
| 355 |
+
|
| 356 |
+
input_values (`torch.FloatTensor` of shape `(num_series, max_patches, patch_size)`):
|
| 357 |
+
The normalized values of every series, cut into patches.
|
| 358 |
+
input_values_mask (`torch.Tensor` of shape `(num_series, max_patches, patch_size)`, *optional*):
|
| 359 |
+
1 for observed time steps, 0 for padding and missing values. Defaults to all observed.
|
| 360 |
+
"""
|
| 361 |
+
if input_values_mask is None:
|
| 362 |
+
input_values_mask = torch.ones_like(input_values)
|
| 363 |
+
ts_outputs = self.ts_tower(input_values, input_values_mask, **kwargs)
|
| 364 |
+
ts_features = self.multi_modal_projector(ts_outputs.last_hidden_state)
|
| 365 |
+
start, end = self.multi_modal_projector.boundary_embeddings.to(ts_features.dtype).split(1)
|
| 366 |
+
patch_counts = get_ts_patch_counts(input_values_mask).tolist()
|
| 367 |
+
ts_outputs.pooler_output = tuple(
|
| 368 |
+
torch.cat([start, features[:count], end]) for features, count in zip(ts_features, patch_counts)
|
| 369 |
+
)
|
| 370 |
+
return ts_outputs
|
| 371 |
+
|
| 372 |
+
def get_placeholder_mask(
|
| 373 |
+
self,
|
| 374 |
+
input_ids: torch.LongTensor | None,
|
| 375 |
+
inputs_embeds: torch.FloatTensor,
|
| 376 |
+
ts_features: torch.FloatTensor,
|
| 377 |
+
) -> torch.Tensor:
|
| 378 |
+
"""Placeholder mask from `input_ids` or `inputs_embeds`, checked against the feature count."""
|
| 379 |
+
if input_ids is None:
|
| 380 |
+
special_ts_mask = inputs_embeds == self.get_input_embeddings()(
|
| 381 |
+
torch.full((), self.config.ts_token_id, dtype=torch.long, device=inputs_embeds.device)
|
| 382 |
+
)
|
| 383 |
+
special_ts_mask = special_ts_mask.all(-1)
|
| 384 |
+
else:
|
| 385 |
+
special_ts_mask = input_ids == self.config.ts_token_id
|
| 386 |
+
|
| 387 |
+
n_ts_tokens = special_ts_mask.sum()
|
| 388 |
+
torch_compilable_check(
|
| 389 |
+
n_ts_tokens * inputs_embeds.shape[-1] == ts_features.numel(),
|
| 390 |
+
f"Time-series features and time-series tokens do not match, tokens: {n_ts_tokens}, "
|
| 391 |
+
f"features: {ts_features.shape[0]}",
|
| 392 |
+
)
|
| 393 |
+
return special_ts_mask.unsqueeze(-1).to(inputs_embeds.device)
|
| 394 |
+
|
| 395 |
+
@can_return_tuple
|
| 396 |
+
def forward(
|
| 397 |
+
self,
|
| 398 |
+
input_ids: torch.LongTensor | None = None,
|
| 399 |
+
attention_mask: torch.Tensor | None = None,
|
| 400 |
+
position_ids: torch.LongTensor | None = None,
|
| 401 |
+
past_key_values: Cache | None = None,
|
| 402 |
+
inputs_embeds: torch.FloatTensor | None = None,
|
| 403 |
+
input_values: torch.FloatTensor | None = None,
|
| 404 |
+
input_values_mask: torch.Tensor | None = None,
|
| 405 |
+
use_cache: bool | None = None,
|
| 406 |
+
**kwargs,
|
| 407 |
+
) -> tuple | Lfm3TsModelOutputWithPast:
|
| 408 |
+
r"""
|
| 409 |
+
input_values (`torch.FloatTensor` of shape `(num_series, max_patches, patch_size)`, *optional*):
|
| 410 |
+
The normalized values of every series in the batch, in the order their placeholders appear.
|
| 411 |
+
input_values_mask (`torch.Tensor` of shape `(num_series, max_patches, patch_size)`, *optional*):
|
| 412 |
+
1 for observed time steps, 0 for padding and missing values. Defaults to all observed.
|
| 413 |
+
"""
|
| 414 |
+
if (input_ids is None) ^ (inputs_embeds is not None):
|
| 415 |
+
raise ValueError("You must specify exactly one of input_ids or inputs_embeds")
|
| 416 |
+
|
| 417 |
+
if inputs_embeds is None:
|
| 418 |
+
inputs_embeds = self.get_input_embeddings()(input_ids)
|
| 419 |
+
|
| 420 |
+
ts_features = None
|
| 421 |
+
if input_values is not None:
|
| 422 |
+
ts_outputs = self.get_ts_features(input_values, input_values_mask)
|
| 423 |
+
ts_features = torch.cat(ts_outputs.pooler_output, dim=0).to(inputs_embeds.device, inputs_embeds.dtype)
|
| 424 |
+
ts_mask = self.get_placeholder_mask(input_ids, inputs_embeds=inputs_embeds, ts_features=ts_features)
|
| 425 |
+
inputs_embeds = inputs_embeds.masked_scatter(ts_mask, ts_features)
|
| 426 |
+
|
| 427 |
+
outputs = self.language_model(
|
| 428 |
+
attention_mask=attention_mask,
|
| 429 |
+
position_ids=position_ids,
|
| 430 |
+
past_key_values=past_key_values,
|
| 431 |
+
inputs_embeds=inputs_embeds,
|
| 432 |
+
use_cache=use_cache,
|
| 433 |
+
**kwargs,
|
| 434 |
+
)
|
| 435 |
+
|
| 436 |
+
return Lfm3TsModelOutputWithPast(
|
| 437 |
+
last_hidden_state=outputs.last_hidden_state,
|
| 438 |
+
past_key_values=outputs.past_key_values,
|
| 439 |
+
hidden_states=outputs.hidden_states,
|
| 440 |
+
attentions=outputs.attentions,
|
| 441 |
+
ts_hidden_states=ts_features,
|
| 442 |
+
)
|
| 443 |
+
|
| 444 |
+
|
| 445 |
+
class Lfm3TsForConditionalGeneration(Lfm3TsPreTrainedModel, GenerationMixin):
|
| 446 |
+
"""The LFM3-TS model: a time-series encoder and an LFM3 language model with a language modeling head."""
|
| 447 |
+
|
| 448 |
+
_tied_weights_keys = {"lm_head.weight": "model.language_model.embed_tokens.weight"}
|
| 449 |
+
|
| 450 |
+
def __init__(self, config: Lfm3TsConfig):
|
| 451 |
+
super().__init__(config)
|
| 452 |
+
self.model = Lfm3TsModel(config)
|
| 453 |
+
self.lm_head = nn.Linear(config.text_config.hidden_size, config.text_config.vocab_size, bias=False)
|
| 454 |
+
self.post_init()
|
| 455 |
+
|
| 456 |
+
def get_input_embeddings(self) -> nn.Module:
|
| 457 |
+
return self.model.get_input_embeddings()
|
| 458 |
+
|
| 459 |
+
def set_input_embeddings(self, value: nn.Module):
|
| 460 |
+
self.model.set_input_embeddings(value)
|
| 461 |
+
|
| 462 |
+
def get_output_embeddings(self) -> nn.Module:
|
| 463 |
+
return self.lm_head
|
| 464 |
+
|
| 465 |
+
def get_ts_features(
|
| 466 |
+
self, input_values: torch.Tensor, input_values_mask: torch.Tensor | None = None, **kwargs
|
| 467 |
+
) -> BaseModelOutputWithPooling:
|
| 468 |
+
return self.model.get_ts_features(input_values, input_values_mask, **kwargs)
|
| 469 |
+
|
| 470 |
+
@classmethod
|
| 471 |
+
def from_text_pretrained(
|
| 472 |
+
cls, pretrained_model_name_or_path: str = "LiquidAI/LFM3-1.7B", ts_config=None, **kwargs
|
| 473 |
+
) -> "Lfm3TsForConditionalGeneration":
|
| 474 |
+
"""Builds an LFM3-TS model around a pretrained LFM3 text checkpoint.
|
| 475 |
+
|
| 476 |
+
The language model takes the checkpoint's weights; the time-series tower and projector are freshly
|
| 477 |
+
initialized and need training. The projector's output starts at the scale of the checkpoint's embeddings.
|
| 478 |
+
`kwargs` go to the text model's `from_pretrained` (`dtype`, `device_map`, ...), except those naming an
|
| 479 |
+
`Lfm3TsConfig` field (`ts_token_id`, `projector_hidden_size`, ...).
|
| 480 |
+
"""
|
| 481 |
+
from transformers import AutoModelForCausalLM
|
| 482 |
+
|
| 483 |
+
config_fields = {
|
| 484 |
+
"ts_token_id",
|
| 485 |
+
"projector_hidden_act",
|
| 486 |
+
"projector_hidden_size",
|
| 487 |
+
"projector_bias",
|
| 488 |
+
"projector_position_dim",
|
| 489 |
+
"projector_output_scale",
|
| 490 |
+
}
|
| 491 |
+
config_kwargs = {key: kwargs.pop(key) for key in list(kwargs) if key in config_fields}
|
| 492 |
+
text_model = AutoModelForCausalLM.from_pretrained(pretrained_model_name_or_path, **kwargs)
|
| 493 |
+
if "projector_output_scale" not in config_kwargs:
|
| 494 |
+
# start the time-series features at the scale of the embeddings they will sit among
|
| 495 |
+
embeddings = text_model.get_input_embeddings().weight
|
| 496 |
+
config_kwargs["projector_output_scale"] = float(embeddings.detach().float().pow(2).mean().sqrt())
|
| 497 |
+
config = Lfm3TsConfig(text_config=text_model.config, ts_config=ts_config, **config_kwargs)
|
| 498 |
+
|
| 499 |
+
model = cls(config)
|
| 500 |
+
# adopt the loaded module itself: casting a fresh copy with `.to(dtype)` would also cast the buffers that
|
| 501 |
+
# `from_pretrained` keeps in full precision, and the outputs would drift from the text model's
|
| 502 |
+
model.model.language_model = text_model.model
|
| 503 |
+
model.lm_head = text_model.lm_head
|
| 504 |
+
reference = text_model.get_input_embeddings().weight
|
| 505 |
+
model.model.ts_tower.to(device=reference.device, dtype=reference.dtype)
|
| 506 |
+
model.model.multi_modal_projector.to(device=reference.device, dtype=reference.dtype)
|
| 507 |
+
model.generation_config = text_model.generation_config
|
| 508 |
+
return model
|
| 509 |
+
|
| 510 |
+
@torch.no_grad()
|
| 511 |
+
def chat(self, processor, series, question: str, max_new_tokens: int = 160, **generate_kwargs) -> str:
|
| 512 |
+
r"""
|
| 513 |
+
Asks one question about one or several series and returns the answer as text.
|
| 514 |
+
|
| 515 |
+
Args:
|
| 516 |
+
processor (`Lfm3TsProcessor`):
|
| 517 |
+
The processor of the checkpoint.
|
| 518 |
+
series:
|
| 519 |
+
A 1D array or list of numbers; or several series, as a list of them or as the rows of a 2D array;
|
| 520 |
+
or `None` for a question without a series. NaN marks a missing value.
|
| 521 |
+
question (`str`):
|
| 522 |
+
What to ask, e.g. `"Is there an anomaly? If yes, where?"`.
|
| 523 |
+
max_new_tokens (`int`, *optional*, defaults to 160):
|
| 524 |
+
The longest answer, in tokens.
|
| 525 |
+
generate_kwargs:
|
| 526 |
+
Passed to `generate`. Decoding is greedy unless `do_sample` is given.
|
| 527 |
+
|
| 528 |
+
Example:
|
| 529 |
+
|
| 530 |
+
```python
|
| 531 |
+
>>> series = np.sin(np.arange(512) / 8)
|
| 532 |
+
>>> series[300] += 4
|
| 533 |
+
>>> model.chat(processor, series, "Is there an anomaly? If yes, where?")
|
| 534 |
+
```"""
|
| 535 |
+
messages = build_chat_messages(series, question)
|
| 536 |
+
inputs = processor.apply_chat_template(
|
| 537 |
+
messages, add_generation_prompt=True, tokenize=True, enable_thinking=False, return_tensors="pt"
|
| 538 |
+
).to(self.device)
|
| 539 |
+
generate_kwargs.setdefault("do_sample", False)
|
| 540 |
+
output = self.generate(**inputs, max_new_tokens=max_new_tokens, **generate_kwargs)
|
| 541 |
+
return processor.decode(output[0, inputs["input_ids"].shape[1] :], skip_special_tokens=True).strip()
|
| 542 |
+
|
| 543 |
+
@can_return_tuple
|
| 544 |
+
def forward(
|
| 545 |
+
self,
|
| 546 |
+
input_ids: torch.LongTensor | None = None,
|
| 547 |
+
attention_mask: torch.Tensor | None = None,
|
| 548 |
+
position_ids: torch.LongTensor | None = None,
|
| 549 |
+
past_key_values: Cache | None = None,
|
| 550 |
+
inputs_embeds: torch.FloatTensor | None = None,
|
| 551 |
+
input_values: torch.FloatTensor | None = None,
|
| 552 |
+
input_values_mask: torch.Tensor | None = None,
|
| 553 |
+
labels: torch.LongTensor | None = None,
|
| 554 |
+
use_cache: bool | None = None,
|
| 555 |
+
logits_to_keep: int | torch.Tensor = 0,
|
| 556 |
+
**kwargs,
|
| 557 |
+
) -> tuple | Lfm3TsCausalLMOutputWithPast:
|
| 558 |
+
r"""
|
| 559 |
+
input_values (`torch.FloatTensor` of shape `(num_series, max_patches, patch_size)`, *optional*):
|
| 560 |
+
The normalized values of every series in the batch, in the order their placeholders appear.
|
| 561 |
+
input_values_mask (`torch.Tensor` of shape `(num_series, max_patches, patch_size)`, *optional*):
|
| 562 |
+
1 for observed time steps, 0 for padding and missing values. Defaults to all observed.
|
| 563 |
+
labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*):
|
| 564 |
+
Labels for the language modeling loss; positions set to `-100` are ignored.
|
| 565 |
+
|
| 566 |
+
Example:
|
| 567 |
+
|
| 568 |
+
```python
|
| 569 |
+
>>> import numpy as np
|
| 570 |
+
>>> from lfm3_ts import Lfm3TsForConditionalGeneration, Lfm3TsProcessor
|
| 571 |
+
|
| 572 |
+
>>> model = Lfm3TsForConditionalGeneration.from_text_pretrained("LiquidAI/LFM3-1.7B")
|
| 573 |
+
>>> processor = Lfm3TsProcessor.from_text_pretrained("LiquidAI/LFM3-1.7B")
|
| 574 |
+
|
| 575 |
+
>>> conversation = [
|
| 576 |
+
... {
|
| 577 |
+
... "role": "user",
|
| 578 |
+
... "content": [
|
| 579 |
+
... {"type": "timeseries", "timeseries": np.sin(np.arange(256) / 8)},
|
| 580 |
+
... {"type": "text", "text": "Where are the anomalies in this series?"},
|
| 581 |
+
... ],
|
| 582 |
+
... },
|
| 583 |
+
... ]
|
| 584 |
+
>>> inputs = processor.apply_chat_template(
|
| 585 |
+
... conversation, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt"
|
| 586 |
+
... )
|
| 587 |
+
>>> outputs = model.generate(**inputs, max_new_tokens=64)
|
| 588 |
+
>>> processor.batch_decode(outputs, skip_special_tokens=True)[0]
|
| 589 |
+
```"""
|
| 590 |
+
outputs = self.model(
|
| 591 |
+
input_ids=input_ids,
|
| 592 |
+
attention_mask=attention_mask,
|
| 593 |
+
position_ids=position_ids,
|
| 594 |
+
past_key_values=past_key_values,
|
| 595 |
+
inputs_embeds=inputs_embeds,
|
| 596 |
+
input_values=input_values,
|
| 597 |
+
input_values_mask=input_values_mask,
|
| 598 |
+
use_cache=use_cache,
|
| 599 |
+
**kwargs,
|
| 600 |
+
)
|
| 601 |
+
|
| 602 |
+
hidden_states = outputs[0]
|
| 603 |
+
# Only compute necessary logits, and do not upcast them to float if we are not computing the loss
|
| 604 |
+
slice_indices = slice(-logits_to_keep, None) if isinstance(logits_to_keep, int) else logits_to_keep
|
| 605 |
+
logits = self.lm_head(hidden_states[:, slice_indices, :])
|
| 606 |
+
|
| 607 |
+
loss = None
|
| 608 |
+
if labels is not None:
|
| 609 |
+
loss = self.loss_function(
|
| 610 |
+
logits=logits, labels=labels, vocab_size=self.config.text_config.vocab_size, **kwargs
|
| 611 |
+
)
|
| 612 |
+
|
| 613 |
+
return Lfm3TsCausalLMOutputWithPast(
|
| 614 |
+
loss=loss,
|
| 615 |
+
logits=logits,
|
| 616 |
+
past_key_values=outputs.past_key_values,
|
| 617 |
+
hidden_states=outputs.hidden_states,
|
| 618 |
+
attentions=outputs.attentions,
|
| 619 |
+
ts_hidden_states=outputs.ts_hidden_states,
|
| 620 |
+
)
|
| 621 |
+
|
| 622 |
+
|
| 623 |
+
__all__ = [
|
| 624 |
+
"build_chat_messages",
|
| 625 |
+
"Lfm3TsEncoder",
|
| 626 |
+
"Lfm3TsForConditionalGeneration",
|
| 627 |
+
"Lfm3TsModel",
|
| 628 |
+
"Lfm3TsPreTrainedModel",
|
| 629 |
+
]
|
processing_lfm3_ts.py
ADDED
|
@@ -0,0 +1,285 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Copyright 2026 The HuggingFace Team. All rights reserved.
|
| 2 |
+
# Copyright 2026 Nardi Xhepi.
|
| 3 |
+
#
|
| 4 |
+
# NOTICE OF CHANGES: this file is part of LFM3-TS. It is adapted from `transformers.models.lfm3_vl`
|
| 5 |
+
# (Hugging Face Transformers, Apache License 2.0) and was changed by Nardi Xhepi: the vision tower and image
|
| 6 |
+
# inputs are replaced by a time-series encoder and time-series inputs.
|
| 7 |
+
#
|
| 8 |
+
# The portions taken from Transformers remain available under the Apache License, Version 2.0; a copy is in
|
| 9 |
+
# LICENSES/Apache-2.0.txt. LFM3-TS as a whole is distributed under the LFM Open License v1.0: see LICENSE and
|
| 10 |
+
# NOTICE at the root of the repository.
|
| 11 |
+
#
|
| 12 |
+
# Unless required by applicable law or agreed to in writing, this software is distributed on an "AS IS" BASIS,
|
| 13 |
+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
| 14 |
+
"""Processor for LFM3-TS."""
|
| 15 |
+
|
| 16 |
+
import numpy as np
|
| 17 |
+
from transformers.feature_extraction_utils import BatchFeature
|
| 18 |
+
from transformers.processing_utils import ProcessorMixin
|
| 19 |
+
|
| 20 |
+
from .feature_extraction_lfm3_ts import Lfm3TsFeatureExtractor, Lfm3TsSeriesStats
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
TS_TOKEN = "<|ts|>"
|
| 24 |
+
|
| 25 |
+
# The LFM3 chat template renders `image`, `video` and `text` content parts; these add a `timeseries` part. Each pair is
|
| 26 |
+
# (anchor found in the LFM3 template, branch inserted before it).
|
| 27 |
+
_CHAT_TEMPLATE_PATCHES = (
|
| 28 |
+
(
|
| 29 |
+
'{%- elif content is mapping and content.get("type") == "text" -%}',
|
| 30 |
+
'{%- elif content is mapping and content.get("type") == "timeseries" -%}\n'
|
| 31 |
+
' {{- "' + TS_TOKEN + '" -}}\n ',
|
| 32 |
+
),
|
| 33 |
+
(
|
| 34 |
+
'{%- elif item is mapping and item.get("type") == "text" -%}',
|
| 35 |
+
'{%- elif item is mapping and item.get("type") == "timeseries" -%}\n'
|
| 36 |
+
' {%- set _ns.result = _ns.result + "' + TS_TOKEN + '" -%}\n ',
|
| 37 |
+
),
|
| 38 |
+
)
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
def add_timeseries_to_chat_template(chat_template: str) -> str:
|
| 42 |
+
"""Teaches an LFM3 chat template to render `{"type": "timeseries"}` content parts as the placeholder token."""
|
| 43 |
+
for anchor, branch in _CHAT_TEMPLATE_PATCHES:
|
| 44 |
+
if chat_template.count(anchor) != 1:
|
| 45 |
+
raise ValueError(f"Cannot add time series to this chat template: expected exactly one `{anchor}`.")
|
| 46 |
+
chat_template = chat_template.replace(anchor, branch + anchor)
|
| 47 |
+
return chat_template
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
def _is_array(item) -> bool:
|
| 51 |
+
return hasattr(item, "ndim") and hasattr(item, "shape")
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
def _flatten_timeseries(timeseries) -> list[np.ndarray]:
|
| 55 |
+
"""Flat list of items, one per placeholder: 1D series or 2D `(num_channels, length)` multivariate series."""
|
| 56 |
+
if _is_array(timeseries):
|
| 57 |
+
return [np.asarray(timeseries, dtype=np.float64)]
|
| 58 |
+
if not isinstance(timeseries, (list, tuple)):
|
| 59 |
+
raise ValueError(f"`timeseries` must be arrays or lists of numbers, got {type(timeseries)}.")
|
| 60 |
+
if len(timeseries) > 0 and all(isinstance(value, (int, float, np.number)) for value in timeseries):
|
| 61 |
+
return [np.asarray(timeseries, dtype=np.float64)]
|
| 62 |
+
return [item for nested in timeseries for item in _flatten_timeseries(nested)]
|
| 63 |
+
|
| 64 |
+
|
| 65 |
+
class Lfm3TsProcessor(ProcessorMixin):
|
| 66 |
+
r"""
|
| 67 |
+
Wraps an `Lfm3TsFeatureExtractor` and an LFM3 tokenizer.
|
| 68 |
+
|
| 69 |
+
Each `<|ts|>` in the text stands for one item of `timeseries`. The processor replaces it with the item's
|
| 70 |
+
normalization statistics in plain text, followed by as many `<|ts|>` tokens as the model will fill with features:
|
| 71 |
+
|
| 72 |
+
```text
|
| 73 |
+
[offset: 12.5000 | scale: 3.2000 | length: 256 | min: 4.1000 | max: 21.7000 | first: 9.8000 | last: 15.2000]<|ts|>...<|ts|>
|
| 74 |
+
```
|
| 75 |
+
|
| 76 |
+
The statistics keep the original values recoverable (the encoder only sees standardized values), and the
|
| 77 |
+
extremes and end points spare the model the arithmetic for the values most often asked about. A 2D item of
|
| 78 |
+
shape `(num_channels, length)` expands to one such block per channel, each on its own line.
|
| 79 |
+
|
| 80 |
+
Args:
|
| 81 |
+
feature_extractor (`Lfm3TsFeatureExtractor`):
|
| 82 |
+
The feature extractor.
|
| 83 |
+
tokenizer (`PreTrainedTokenizerBase`):
|
| 84 |
+
An LFM3 tokenizer holding the `<|ts|>` token.
|
| 85 |
+
chat_template (`str`, *optional*):
|
| 86 |
+
A chat template that renders `{"type": "timeseries"}` content parts as `<|ts|>`.
|
| 87 |
+
position_marker_every (`int`, *optional*, defaults to 0):
|
| 88 |
+
Writes the index of a time step, as text like `<64>`, before the token of the patch that starts there,
|
| 89 |
+
every this many time steps. Must
|
| 90 |
+
be a multiple of the patch size; 0 writes none.
|
| 91 |
+
value_ruler (`bool`, *optional*, defaults to `False`):
|
| 92 |
+
Also writes, in the statistics, the values that stand at -2, -1, 0, +1 and +2 in the standardized units
|
| 93 |
+
the encoder sees. A checkpoint keeps the setting it was trained with.
|
| 94 |
+
"""
|
| 95 |
+
|
| 96 |
+
def __init__(
|
| 97 |
+
self,
|
| 98 |
+
feature_extractor=None,
|
| 99 |
+
tokenizer=None,
|
| 100 |
+
chat_template=None,
|
| 101 |
+
position_marker_every: int = 0,
|
| 102 |
+
value_ruler: bool = False,
|
| 103 |
+
**kwargs,
|
| 104 |
+
):
|
| 105 |
+
if position_marker_every % feature_extractor.patch_size:
|
| 106 |
+
raise ValueError(
|
| 107 |
+
f"position_marker_every={position_marker_every} must be a multiple of the patch size "
|
| 108 |
+
f"{feature_extractor.patch_size}."
|
| 109 |
+
)
|
| 110 |
+
self.position_marker_every = position_marker_every
|
| 111 |
+
self.value_ruler = value_ruler
|
| 112 |
+
self.ts_token = getattr(tokenizer, "ts_token", TS_TOKEN)
|
| 113 |
+
self.ts_token_id = tokenizer.convert_tokens_to_ids(self.ts_token)
|
| 114 |
+
super().__init__(feature_extractor, tokenizer, chat_template=chat_template)
|
| 115 |
+
|
| 116 |
+
@classmethod
|
| 117 |
+
def from_text_pretrained(
|
| 118 |
+
cls,
|
| 119 |
+
pretrained_model_name_or_path: str = "LiquidAI/LFM3-1.7B",
|
| 120 |
+
patch_size: int = 8,
|
| 121 |
+
position_marker_every: int = 0,
|
| 122 |
+
value_ruler: bool = False,
|
| 123 |
+
**kwargs,
|
| 124 |
+
) -> "Lfm3TsProcessor":
|
| 125 |
+
"""Builds a processor from an LFM3 text checkpoint: its tokenizer gains the `<|ts|>` token and its chat
|
| 126 |
+
template a `timeseries` content part. `kwargs` go to the tokenizer's `from_pretrained`."""
|
| 127 |
+
from transformers import AutoTokenizer
|
| 128 |
+
|
| 129 |
+
tokenizer = AutoTokenizer.from_pretrained(
|
| 130 |
+
pretrained_model_name_or_path, extra_special_tokens={"ts_token": TS_TOKEN}, **kwargs
|
| 131 |
+
)
|
| 132 |
+
chat_template = add_timeseries_to_chat_template(tokenizer.chat_template)
|
| 133 |
+
tokenizer.chat_template = chat_template
|
| 134 |
+
return cls(
|
| 135 |
+
Lfm3TsFeatureExtractor(patch_size=patch_size),
|
| 136 |
+
tokenizer,
|
| 137 |
+
chat_template=chat_template,
|
| 138 |
+
position_marker_every=position_marker_every,
|
| 139 |
+
value_ruler=value_ruler,
|
| 140 |
+
)
|
| 141 |
+
|
| 142 |
+
def format_series(self, stats: Lfm3TsSeriesStats) -> str:
|
| 143 |
+
"""The text standing for one series: its statistics, then a start, per-patch and end placeholder."""
|
| 144 |
+
header = (
|
| 145 |
+
f"[offset: {stats.offset:.4f} | scale: {stats.scale:.4f} | length: {stats.length}"
|
| 146 |
+
f" | min: {stats.minimum:.4f} | max: {stats.maximum:.4f}"
|
| 147 |
+
f" | first: {stats.first:.4f} | last: {stats.last:.4f}"
|
| 148 |
+
)
|
| 149 |
+
if self.value_ruler:
|
| 150 |
+
marks = ", ".join(f"{level:+d}: {stats.offset + level * stats.scale:.4f}" for level in (-2, -1, 0, 1, 2))
|
| 151 |
+
header += f" | levels {marks}"
|
| 152 |
+
header += "]"
|
| 153 |
+
if not self.position_marker_every:
|
| 154 |
+
return header + self.ts_token * (stats.num_patches + 2)
|
| 155 |
+
patch_size = self.feature_extractor.patch_size
|
| 156 |
+
pieces = [header, self.ts_token]
|
| 157 |
+
for patch in range(stats.num_patches):
|
| 158 |
+
index = patch * patch_size
|
| 159 |
+
if patch and index % self.position_marker_every == 0:
|
| 160 |
+
pieces.append(f"<{index}>")
|
| 161 |
+
pieces.append(self.ts_token)
|
| 162 |
+
return "".join(pieces) + self.ts_token
|
| 163 |
+
|
| 164 |
+
def __call__(self, text=None, timeseries=None, return_tensors=None, **kwargs) -> BatchFeature:
|
| 165 |
+
"""
|
| 166 |
+
Args:
|
| 167 |
+
text (`str` or `list[str]`):
|
| 168 |
+
The prompts, with one `<|ts|>` per item of `timeseries`.
|
| 169 |
+
timeseries (*optional*):
|
| 170 |
+
A series, or a list of them (or one list per prompt), in the order of their placeholders. A 1D array
|
| 171 |
+
or list of numbers is one series; a 2D array of shape `(num_channels, length)` is a multivariate
|
| 172 |
+
series. NaN marks a missing value.
|
| 173 |
+
return_tensors (`str`, *optional*):
|
| 174 |
+
The tensor type to return, e.g. `"pt"`.
|
| 175 |
+
kwargs:
|
| 176 |
+
Passed to the tokenizer (`padding`, `truncation`, `add_special_tokens`, ...).
|
| 177 |
+
|
| 178 |
+
Returns:
|
| 179 |
+
`BatchFeature` with `input_ids`, `attention_mask` and, when `timeseries` is given, `input_values` and
|
| 180 |
+
`input_values_mask` of shape `(num_series, max_patches, patch_size)`.
|
| 181 |
+
"""
|
| 182 |
+
if text is None:
|
| 183 |
+
raise ValueError("You must provide `text`.")
|
| 184 |
+
prompts = [text] if isinstance(text, str) else list(text)
|
| 185 |
+
items = [] if timeseries is None else _flatten_timeseries(timeseries)
|
| 186 |
+
|
| 187 |
+
num_placeholders = sum(prompt.count(self.ts_token) for prompt in prompts)
|
| 188 |
+
if num_placeholders != len(items):
|
| 189 |
+
raise ValueError(
|
| 190 |
+
f"Found {num_placeholders} {self.ts_token} tokens in the text but received {len(items)} time series."
|
| 191 |
+
)
|
| 192 |
+
|
| 193 |
+
data = {}
|
| 194 |
+
if items:
|
| 195 |
+
channels, channels_per_item = [], []
|
| 196 |
+
for index, item in enumerate(items):
|
| 197 |
+
if item.ndim not in (1, 2):
|
| 198 |
+
raise ValueError(f"Time series {index} must be 1D or 2D (num_channels, length), got {item.shape}.")
|
| 199 |
+
item_channels = [item] if item.ndim == 1 else list(item)
|
| 200 |
+
channels.extend(item_channels)
|
| 201 |
+
channels_per_item.append(len(item_channels))
|
| 202 |
+
|
| 203 |
+
input_values, input_values_mask, stats = self.feature_extractor.preprocess(channels)
|
| 204 |
+
data.update(input_values=input_values, input_values_mask=input_values_mask)
|
| 205 |
+
|
| 206 |
+
blocks = iter(self.format_series(stat) for stat in stats)
|
| 207 |
+
replacements = iter("\n".join(next(blocks) for _ in range(count)) for count in channels_per_item)
|
| 208 |
+
expanded = []
|
| 209 |
+
for prompt in prompts:
|
| 210 |
+
pieces = prompt.split(self.ts_token)
|
| 211 |
+
expanded.append("".join(piece + next(replacements) for piece in pieces[:-1]) + pieces[-1])
|
| 212 |
+
prompts = expanded
|
| 213 |
+
|
| 214 |
+
text_inputs = self.tokenizer(prompts, **kwargs)
|
| 215 |
+
return BatchFeature(data={**text_inputs, **data}, tensor_type=return_tensors)
|
| 216 |
+
|
| 217 |
+
def apply_chat_template(
|
| 218 |
+
self,
|
| 219 |
+
conversation,
|
| 220 |
+
chat_template: str | None = None,
|
| 221 |
+
tokenize: bool = False,
|
| 222 |
+
return_dict: bool = True,
|
| 223 |
+
add_generation_prompt: bool = False,
|
| 224 |
+
**kwargs,
|
| 225 |
+
):
|
| 226 |
+
"""
|
| 227 |
+
Renders one conversation, or a batch of them, and optionally tokenizes it together with its time series.
|
| 228 |
+
|
| 229 |
+
A time series is a content part `{"type": "timeseries", "timeseries": array}`. Template arguments (`tools`,
|
| 230 |
+
`documents`, `continue_final_message`, `reasoning_effort`, `enable_thinking`, `preserve_thinking`) are
|
| 231 |
+
forwarded to the template; other `kwargs` go to `__call__` when `tokenize=True`.
|
| 232 |
+
"""
|
| 233 |
+
is_batched = isinstance(conversation, (list, tuple)) and isinstance(conversation[0], (list, tuple))
|
| 234 |
+
conversations = conversation if is_batched else [conversation]
|
| 235 |
+
|
| 236 |
+
timeseries, rendered_conversations = [], []
|
| 237 |
+
for messages in conversations:
|
| 238 |
+
rendered_messages = []
|
| 239 |
+
for message in messages:
|
| 240 |
+
content = message.get("content")
|
| 241 |
+
if isinstance(content, (list, tuple)):
|
| 242 |
+
parts = []
|
| 243 |
+
for part in content:
|
| 244 |
+
if isinstance(part, dict) and part.get("type") == "timeseries":
|
| 245 |
+
timeseries.append(part["timeseries"])
|
| 246 |
+
part = {"type": "timeseries"}
|
| 247 |
+
parts.append(part)
|
| 248 |
+
message = {**message, "content": parts}
|
| 249 |
+
rendered_messages.append(message)
|
| 250 |
+
rendered_conversations.append(rendered_messages)
|
| 251 |
+
|
| 252 |
+
template_keys = (
|
| 253 |
+
"tools",
|
| 254 |
+
"documents",
|
| 255 |
+
"continue_final_message",
|
| 256 |
+
"reasoning_effort",
|
| 257 |
+
"enable_thinking",
|
| 258 |
+
"preserve_thinking",
|
| 259 |
+
)
|
| 260 |
+
template_kwargs = {key: kwargs.pop(key) for key in template_keys if key in kwargs}
|
| 261 |
+
prompts = [
|
| 262 |
+
self.tokenizer.apply_chat_template(
|
| 263 |
+
messages,
|
| 264 |
+
chat_template=chat_template or self.chat_template,
|
| 265 |
+
tokenize=False,
|
| 266 |
+
add_generation_prompt=add_generation_prompt,
|
| 267 |
+
**template_kwargs,
|
| 268 |
+
)
|
| 269 |
+
for messages in rendered_conversations
|
| 270 |
+
]
|
| 271 |
+
if not tokenize:
|
| 272 |
+
return prompts if is_batched else prompts[0]
|
| 273 |
+
|
| 274 |
+
# the template writes the special tokens itself
|
| 275 |
+
kwargs.setdefault("add_special_tokens", False)
|
| 276 |
+
inputs = self(text=prompts, timeseries=timeseries or None, **kwargs)
|
| 277 |
+
return inputs if return_dict else inputs["input_ids"]
|
| 278 |
+
|
| 279 |
+
@property
|
| 280 |
+
def model_input_names(self):
|
| 281 |
+
names = self.tokenizer.model_input_names + self.feature_extractor.model_input_names
|
| 282 |
+
return list(dict.fromkeys(names))
|
| 283 |
+
|
| 284 |
+
|
| 285 |
+
__all__ = ["Lfm3TsProcessor", "add_timeseries_to_chat_template"]
|
processor_config.json
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"auto_map": {
|
| 3 |
+
"AutoProcessor": "processing_lfm3_ts.Lfm3TsProcessor"
|
| 4 |
+
},
|
| 5 |
+
"feature_extractor": {
|
| 6 |
+
"auto_map": {
|
| 7 |
+
"AutoFeatureExtractor": "feature_extraction_lfm3_ts.Lfm3TsFeatureExtractor",
|
| 8 |
+
"AutoProcessor": "processing_lfm3_ts.Lfm3TsProcessor"
|
| 9 |
+
},
|
| 10 |
+
"eps": 1e-06,
|
| 11 |
+
"feature_extractor_type": "Lfm3TsFeatureExtractor",
|
| 12 |
+
"patch_size": 8
|
| 13 |
+
},
|
| 14 |
+
"position_marker_every": 0,
|
| 15 |
+
"processor_class": "Lfm3TsProcessor",
|
| 16 |
+
"value_ruler": false
|
| 17 |
+
}
|
tokenizer.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:753a4f0070f119707185658554bcedf8eb43d05cc952b5f670c562de8cfd0aee
|
| 3 |
+
size 17908817
|
tokenizer_config.json
ADDED
|
@@ -0,0 +1,69 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"audio_out_token": "<|audio_out|>",
|
| 3 |
+
"audio_token": "<|audio|>",
|
| 4 |
+
"auto_map": {
|
| 5 |
+
"AutoProcessor": "processing_lfm3_ts.Lfm3TsProcessor"
|
| 6 |
+
},
|
| 7 |
+
"backend": "tokenizers",
|
| 8 |
+
"bos_token": "<|startoftext|>",
|
| 9 |
+
"eos_token": "<|im_end|>",
|
| 10 |
+
"image_token": "<|image|>",
|
| 11 |
+
"is_local": true,
|
| 12 |
+
"local_files_only": false,
|
| 13 |
+
"model_max_length": 1000000000000000019884624838656,
|
| 14 |
+
"model_specific_special_tokens": {
|
| 15 |
+
"audio_out_token": "<|audio_out|>",
|
| 16 |
+
"audio_token": "<|audio|>",
|
| 17 |
+
"image_token": "<|image|>",
|
| 18 |
+
"ts_token": "<|ts|>",
|
| 19 |
+
"video_token": "<|video|>",
|
| 20 |
+
"vision_end_token": "<|vision_end|>",
|
| 21 |
+
"vision_start_token": "<|vision_start|>"
|
| 22 |
+
},
|
| 23 |
+
"pad_token": "<|pad|>",
|
| 24 |
+
"processor_class": "Lfm3TsProcessor",
|
| 25 |
+
"response_template": {
|
| 26 |
+
"defaults": {
|
| 27 |
+
"role": "assistant"
|
| 28 |
+
},
|
| 29 |
+
"fields": {
|
| 30 |
+
"content": {
|
| 31 |
+
"close_pattern": "<\\|im_end\\|>\\s*",
|
| 32 |
+
"content": "text"
|
| 33 |
+
},
|
| 34 |
+
"reasoning_content": {
|
| 35 |
+
"close_pattern": "</think>\\s*",
|
| 36 |
+
"content": "text",
|
| 37 |
+
"open_pattern": "<think>\\n?"
|
| 38 |
+
},
|
| 39 |
+
"tool_calls": {
|
| 40 |
+
"close_pattern": "</tool_call>\\s*",
|
| 41 |
+
"content": "xml-inline",
|
| 42 |
+
"content_args": {
|
| 43 |
+
"tag_pattern": "<parameter=(?P<key>[^>\\n]+)>\\n?(?P<value>.*?)\\n?</parameter>(?=\\s*(?:<parameter=|</function>\\s*\\Z|\\Z))",
|
| 44 |
+
"value_parser": {
|
| 45 |
+
"args": {
|
| 46 |
+
"strip": false
|
| 47 |
+
},
|
| 48 |
+
"name": "text"
|
| 49 |
+
}
|
| 50 |
+
},
|
| 51 |
+
"open_pattern": "<tool_call>\\s*<function=(?P<name>[^\\n>]+)>",
|
| 52 |
+
"repeats": true,
|
| 53 |
+
"transform": {
|
| 54 |
+
"function": {
|
| 55 |
+
"arguments": "{content}",
|
| 56 |
+
"name": "{name}"
|
| 57 |
+
},
|
| 58 |
+
"type": "function"
|
| 59 |
+
}
|
| 60 |
+
}
|
| 61 |
+
},
|
| 62 |
+
"start_anchor_pattern": "<\\|im_start\\|>assistant\\n"
|
| 63 |
+
},
|
| 64 |
+
"tokenizer_class": "TokenizersBackend",
|
| 65 |
+
"ts_token": "<|ts|>",
|
| 66 |
+
"video_token": "<|video|>",
|
| 67 |
+
"vision_end_token": "<|vision_end|>",
|
| 68 |
+
"vision_start_token": "<|vision_start|>"
|
| 69 |
+
}
|