Add files using upload-large-folder tool

README.md

@@ -0,0 +1,150 @@
+<div align="center">
+
+# ChatTime: A Multimodal Time Series Foundation Model
+
+<a href='https://ojs.aaai.org/index.php/AAAI/article/view/33384'><img src='https://img.shields.io/badge/Paper-PDF-orange'></a> 
+[![huggingface](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Datasets-FFD21E)](https://huggingface.co/collections/ChengsenWang/chattime-datasets-6731b504efecc8a6e439741c)
+[![huggingface](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Models-FFD21E)](https://huggingface.co/collections/ChengsenWang/chattime-models-6731b650cb98bc7842713fde)
+
+</div>
+
+## ✨ Introduction
+
+Human experts typically integrate numerical and textual multimodal information to analyze time series. However, most traditional deep learning predictors rely solely on unimodal numerical data, using a fixed-length window for training and prediction on a single dataset, and cannot adapt to different scenarios. The powered pre-trained large language model has introduced new opportunities for time series analysis. Yet, existing methods are either inefficient in training, incapable of handling textual information, or lack zero-shot forecasting capability. In this paper, we innovatively model time series as a foreign language and construct ChatTime, a unified framework for time series and text processing. As an out-of-the-box multimodal time series foundation model, ChatTime provides zero-shot forecasting capability and supports bimodal input/output for both time series and text. We design a series of experiments to verify the superior performance of ChatTime across multiple tasks and scenarios, and create four multimodal datasets to address data gaps. The experimental results demonstrate the potential and utility of ChatTime. Specifically, in the unimodal zero-shot forecasting task, ChatTime achieves 99.9% of the previous state-of-the-art using only 4% of the pre-training data. In the multimodal tasks of context-guided forecasting and time series question answering, ChatTime outperforms the previous state-of-the-art by 3.7% and 36.6%, respectively.
+
+![](./img/architecture.png)
+
+## 📈 Usage
+
+We present three minimal examples showing how to perform the multimodal time series analysis using the ChatTime model. The code and corresponding results are available in the [notebook](./demo.ipynb).
+
+### Zero-Shot Time Series Forecasting
+
+```python
+
+import numpy as np
+import pandas as pd
+import matplotlib.pyplot as plt
+from model.model import ChatTime
+
+dataset = "Traffic"
+hist_len = 120
+pred_len = 24
+model_path = "ChengsenWang/ChatTime-1-7B-Chat"
+
+df = pd.read_csv(f"./dataset/{dataset}.csv")
+hist_data = np.array(df["Hist"].apply(eval).values.tolist())[:, -hist_len:][0]
+pred_data = np.array(df["Pred"].apply(eval).values.tolist())[:, :pred_len][0]
+
+model = ChatTime(hist_len=hist_len, pred_len=pred_len, model_path=model_path)
+
+out = model.predict(hist_data)
+
+hist_x = np.linspace(0, hist_len-1, hist_len)
+pred_x = np.linspace(hist_len, hist_len+pred_len-1, pred_len)
+
+plt.figure(figsize=(8, 2), dpi=500)
+plt.plot(hist_x, hist_data, color='#000000')
+plt.plot(pred_x, pred_data, color='#000000', label='true')
+plt.plot(pred_x, out, color='#FF7F0E', label='pred')
+plt.axvline(hist_len, color='red')
+plt.legend(loc="upper left")
+plt.show()
+
+```
+
+### Context-Guided Time Series Forecasting
+
+```python
+
+import numpy as np
+import pandas as pd
+import matplotlib.pyplot as plt
+from model.model import ChatTime
+
+dataset = "PTF"
+hist_len = 120
+pred_len = 24
+model_path = "ChengsenWang/ChatTime-1-7B-Chat"
+
+df = pd.read_csv(f"./dataset/{dataset}.csv")
+hist_data = np.array(df["Hist"].apply(eval).values.tolist())[:, -hist_len:][0]
+pred_data = np.array(df["Pred"].apply(eval).values.tolist())[:, :pred_len][0]
+context = df["Text"].values[0]
+
+model = ChatTime(hist_len=hist_len, pred_len=pred_len, model_path=model_path)
+
+out_text = model.predict(hist_data, context)
+out = model.predict(hist_data)
+
+hist_x = np.linspace(0, hist_len-1, hist_len)
+pred_x = np.linspace(hist_len, hist_len+pred_len-1, pred_len)
+
+plt.figure(figsize=(8, 2), dpi=500)
+plt.plot(hist_x, hist_data, color='#000000')
+plt.plot(pred_x, pred_data, color='#000000', label='true')
+plt.plot(pred_x, out_text, color='#FF7F0E', label='pred_text')
+plt.plot(pred_x, out, color='#1F77B4', label='pred')
+plt.axvline(hist_len, color='red')
+plt.legend(loc="upper left")
+plt.show()
+
+```
+
+### Time Series Question Answering
+
+```python
+
+import numpy as np
+import pandas as pd
+import matplotlib.pyplot as plt
+from model.model import ChatTime
+
+dataset = "TSQA"
+model_path = "ChengsenWang/ChatTime-1-7B-Chat"
+
+df = pd.read_csv(f"./dataset/{dataset}.csv")
+series = np.array(df["Series"].apply(eval).values.tolist())[0]
+question = df["Question"].values[0]
+answer = df["Answer"].values[0]
+
+model = ChatTime(model_path=model_path)
+
+out = model.analyze(question, series)
+
+plt.figure(figsize=(8, 2), dpi=500)
+plt.plot(series, color='#000000')
+plt.show()
+
+print(question)
+print(f"\n{out} / {answer}\n")
+
+```
+
+### Continuous Pre-Training and Instruction Fine-Tuning
+
+The code for Continuous Pre-Training and Instruction Fine-Tuning for ChatTime is located within the [training](./training/) folder, while the model weights are available at [ChengsenWang/ChatTime-1-7B-Base](https://huggingface.co/ChengsenWang/ChatTime-1-7B-Base) and [ChengsenWang/ChatTime-1-7B-Chat](https://huggingface.co/ChengsenWang/ChatTime-1-7B-Chat), respectively.
+
+## :floppy_disk: Datasets
+
+Refer to following repositories for instructions on downloading and utilizing the datasets.
+
+- The datasets used in the ChatTime paper for Continuous Pre-Training and Instruction Fine-Tuning are available in the HuggingFace repositories: [ChengsenWang/ChatTime-1-Pretrain-1M](https://huggingface.co/datasets/ChengsenWang/ChatTime-1-Pretrain-1M) and [ChengsenWang/ChatTime-1-Finetune-100K](https://huggingface.co/datasets/ChengsenWang/ChatTime-1-Finetune-100K).
+- Unimodal zero-shot forecasting datasets can be accessed via the previous [Google Drive](https://drive.google.com/drive/folders/1S7u4exc5NkKRWfdgqYBZ-VqSz9XfrEDV?usp=sharing), while multimodal datasets for context-guided forecasting and time series question-answering tasks are available on HuggingFace at [ChengsenWang/CGTSF](https://huggingface.co/datasets/ChengsenWang/CGTSF) and [ChengsenWang/TSQA](https://huggingface.co/datasets/ChengsenWang/TSQA).
+
+## 📝 Citation
+
+If you find this repo or our work useful for your research, please consider citing the [paper](https://arxiv.org/abs/2412.11376):
+
+```tex
+@inproceedings{
+  author    = {Chengsen Wang and Qi Qi and Jingyu Wang and Haifeng Sun and Zirui Zhuang and Jinming Wu and Lei Zhang and Jianxin Liao},
+  title     = {ChatTime: A Unified Multimodal Time Series Foundation Model Bridging Numerical and Textual Data},
+  booktitle = {AAAI Conference on Artificial Intelligence},
+  year      = {2025},
+}
+```
+
+## 📪 Contact
+
+If you have any question, please contact [cswang@bupt.edu.cn]().

dataset/PTF.csv

@@ -0,0 +1,2 @@
+Idx,Date,Hist,Pred,Text
+1,2022/1/10,"[91.0, 93.0, 70.0, 54.0, 25.0, 26.0, 54.0, 88.0, 181.0, 324.0, 324.0, 307.0, 273.0, 264.0, 262.0, 295.0, 301.0, 226.0, 250.0, 215.0, 252.0, 153.0, 139.0, 158.0, 120.0, 94.0, 72.0, 48.0, 37.0, 31.0, 56.0, 92.0, 229.0, 341.0, 331.421686746988, 278.0, 257.0, 256.0, 269.0, 232.0, 291.0, 290.0, 266.0, 234.0, 273.0, 167.0, 131.0, 141.0, 128.0, 120.0, 103.0, 65.0, 38.0, 36.0, 62.0, 104.0, 205.0, 352.0, 300.0, 318.0, 285.0, 290.0, 311.0, 294.0, 280.0, 289.0, 232.0, 254.0, 243.0, 213.0, 151.0, 155.0, 128.0, 147.0, 148.0, 150.0, 118.0, 85.0, 75.0, 57.0, 103.0, 96.0, 150.0, 205.0, 269.0, 226.0, 224.0, 297.0, 260.0, 286.0, 240.0, 244.0, 256.0, 208.0, 153.0, 134.0, 138.0, 145.0, 153.0, 164.0, 87.0, 99.0, 67.0, 43.0, 56.0, 75.0, 102.0, 155.0, 208.0, 219.0, 217.0, 221.0, 231.0, 232.0, 213.0, 212.0, 196.0, 148.0, 118.0, 100.0]","[98.0, 68.0, 66.0, 43.0, 31.0, 30.0, 44.0, 96.0, 228.0, 351.0, 334.0, 253.0, 268.0, 254.0, 271.0, 267.0, 260.0, 211.0, 246.0, 228.0, 209.0, 159.0, 154.0, 108.0]","This sequence records traffic flow at a highway in Paris, France, with a collection granularity of 1 hour. The target date for prediction is Monday, January 10, 2022. It is a weekday with overcast and light breeze. The minimum temperature is 1 degrees, and the maximum temperature is 4 degrees. The sun will rise at 9:41 and set at 18:14."

dataset/TSQA.csv

@@ -0,0 +1,10 @@
+Task,Size,Question,Answer,Label,Series
+Outliers,512,"Given the following definitions:
+Outlier: The data point that significantly differs from other observations in a time series.
+Sudden spike: The rapid and significant increase in the value of a variable over a short period, followed by a return to the original baseline.
+Level shift: The significant and sustained change in the average level of a time series.
+Select one of the following answers that best describes the provided time series:
+(a) This time series has no obvious outlier.
+(b) This time series has a sudden spike.
+(c) This time series has a level shift.
+Only answer (a), (b), or (c).",(c),level shift,"[0.7184792798575108, 0.5890992628753317, 0.720379226692959, 0.7371817534266486, 0.8218986846647067, 0.8831912599405045, 0.924568275381516, 1.105450126561668, 1.104915171493406, 1.105075077212364, 1.172752127438253, 1.1017339697399184, 1.1885389876081582, 1.0229032988686375, 1.0173509225272503, 0.9778380959636221, 0.9905061842429381, 0.9192487309319277, 0.955159815166365, 0.9386546300931191, 1.0365586552042338, 0.9848060010135675, 0.9031910437874903, 0.9490722460363644, 0.7736690375506368, 0.7973083717713468, 0.6195164348946642, 0.4505078990428173, 0.5258964311649057, 0.41695857834045796, 0.40094083046178974, 0.4074777885743717, 0.4898508983261235, 0.5204220132901043, 0.6147843655529912, 0.7104068479820054, 0.6888432643084303, 0.7093100484404489, 0.9203034882926628, 0.9595492099364219, 1.0524983080812496, 1.0540235235303996, 1.0545795711233983, 1.0504204593712156, 1.0150624010493834, 0.9674438806947031, 0.9480584014886574, 0.8387509457584394, 0.756715885697077, 0.6876310725765916, 0.6741053133041839, 0.5765459677947272, 0.8150541280061177, 0.8010140835380615, 0.8017261681809998, 0.8522744537466967, 0.997448921515707, 1.0254380922548936, 1.0854623631951241, 1.1810641178273915, 1.1917727690594564, 1.156212392946056, 1.0032228674436472, 1.0097138291132903, 0.9644046450619518, 0.9786099232959252, 0.9015820258468665, 0.9600845692636649, 0.9254786568752694, 1.0275123161011555, 1.0439750545550874, 0.9781885938000409, 0.9794484252067047, 0.8996776075842985, 0.8288172338731659, 0.7377456987757804, 0.6666415913497841, 0.4954252371332561, 0.3871874702376371, 0.39201341491430786, 0.4691953202649433, 0.32902849388076194, 0.5139388807727633, 0.6274850412452753, 0.5259500940219756, 0.6234727127754345, 0.7352204681274915, 0.8887733771487537, 0.8788745652679044, 0.8321993334089987, 0.9906538408377465, 1.0884153923116429, 1.0419565131523847, 1.0989571442112747, 1.0968068649354548, 0.9917228083304463, 0.9140857106015, 0.8268703694816582, 0.7841652686982843, 0.6563931372557668, 0.704137218562753, 0.6463851364825651, 0.7506367510170829, 0.7554979826318322, 0.9260291687003874, 0.9089546087562222, 0.9693357182731545, 1.1030492035662054, 1.2037368783717812, 1.1546801365968427, 1.1832562640065416, 1.2227138316112929, 1.0586537993195908, 1.1019346318306265, 1.0957716907972106, 1.0435114304268116, 0.9432916345837352, 0.9631075724933503, 0.9909790612209175, 0.902075105613618, 1.038934325071035, 0.9956296391923886, 0.9671387170483486, 0.9262196734645687, 0.960294897621253, 0.7043583357537013, 0.5390751263748786, 0.5065810639874693, 0.42986439305645174, 0.45645353834092917, 0.4199892080755012, 0.44424909650077626, 0.4999027503106028, 0.5419331322097575, 0.5928239178075897, 0.7700799966350905, 0.7657828913531622, 0.8342811572897805, 0.8853906370913285, 0.920301247256349, 1.0980213455034573, 1.026281140574831, 1.0574832942885364, 1.0565905379973175, 1.0225231206444396, 0.9959311509374223, 0.831378237673205, 0.8535538463246993, 0.6864405593142846, 0.6795595826003835, 0.6267738101529042, 0.6322335718113458, 0.7141720760554285, 0.7070980364621493, 0.9013901134388234, 0.9182868748935803, 0.9901188442633743, 1.0602930122437244, 1.1437313511943954, 1.2521610849514286, 1.1573637604198495, 1.1488324866917643, 1.001628768089996, 0.9539103891985391, 1.0421937873720968, 0.9169351532898107, 0.9181443865083075, 0.9870397344023188, 0.9743500980619964, 0.9278090932905554, 0.9869314160638195, 1.0825826963691016, 0.9183086209574945, 0.865278855396822, 0.9217381982644645, 0.7578601168462166, 0.6473126784348423, 0.49641592532600703, 0.4970296783009491, 0.488547021540167, 0.433373127695315, 0.47983095468527404, 0.5537629501662963, 0.5759671849077381, 0.6090143186894204, 0.6387543942622422, 0.7652722032542528, 0.77676853522667, 0.8821526879120217, 1.044484519382665, 1.109259890307919, 1.0820707143438995, 1.0370613573514775, 1.0957628304825053, 1.0580424077326653, 1.0143376747856974, 0.8715637233344328, 0.8556943613327225, 0.6927027258079334, 0.6766893669789288, 0.6184844783413905, 0.7306990192316035, 0.6672370783283702, 0.7944649559832869, 0.8326901828736859, 0.9656049723383584, 1.056679107562601, 1.0943165393078627, 1.189886820998889, 1.1271927149892658, 1.1977011068367385, 1.155952373861953, 1.146964840734465, 1.0975866934126028, 0.9554315747980717, 1.011860502416628, 0.9022676830321746, 0.9262251486399553, 0.9745082808375313, 1.038931255746843, 0.9345750120587087, 0.8758563063048941, 0.9220831095921272, 0.842203342897673, 0.8026814852580891, 0.7262943394431286, 0.6914774930999312, 0.6369795018288573, 0.4672892885082276, 0.44424805787729965, 0.46526279273593074, 0.500063884334687, 0.5849212005758848, 0.5546111154221287, 0.6168837755228581, 0.657730971092552, 0.7936078881771494, 0.9071219549558203, 0.8848795364518045, 0.9504371476718672, 0.9855640423604446, 1.0823133640844074, 1.1432619064607117, 1.1330658313270692, 1.0361077987982434, 1.0065302028833885, 0.8992705087877719, 0.8473733436759884, 0.7801492907831172, 0.7784846618295218, 0.7820701339727955, 0.7214826681707797, 0.6764354492853714, 0.7458898813294413, 0.8580283210552919, 1.0190915082844239, 1.1499430819819825, 1.0546364390285556, 1.1545070867266525, 1.1532497822886496, 1.1305042496753939, 1.1374751941014465, 1.1451052573144964, 1.0984450376615762, 1.0007209086705193, 1.0750979382290524, 0.9965740706335955, 0.954945626857081, 0.9927980457832121, 0.9502067760650941, 0.9484691011475065, 0.969721516082676, 0.9095044917595535, 0.8942393452053652, 0.7287712533676985, 0.7777980882341948, 0.6176006587423752, 0.5124213502594732, 0.5528431556689122, 0.5088449418686641, 0.4093617807757338, 0.49731713222991464, 0.6206976435456033, 0.6294345145659748, 0.5547663124623439, 0.7297202990060477, 0.7950380657536229, 0.831397425599947, 0.9575384453262332, 1.0249258749103494, -0.8686413077216231, -0.9014042639103335, -0.9462897790105698, -0.9316112945419105, -0.9754026372137907, -1.077839091816972, -1.157768612443351, -1.1915681981266364, -1.2840146438930529, -1.252211472967503, -1.2552632923579257, -1.29625542078108, -1.231867952201975, -1.2033132751820963, -1.0883172493629345, -1.0111279420606132, -0.9992633330998815, -0.9131983391171828, -0.8964812349542782, -0.7614250697098287, -0.8925261308611362, -0.8094410002909738, -0.9102688878990173, -0.9733338417324072, -1.0751505114862114, -1.08965194203292, -1.1332817204148675, -1.0534382410528467, -1.0266754644760356, -1.0573439518397016, -1.05377027626065, -1.0147742871806245, -1.0408024164375704, -1.0856486422810674, -1.1869376217613805, -1.3950229148750681, -1.3792567179476216, -1.4717478634994605, -1.5317027101151302, -1.4264296167170638, -1.5201952762047963, -1.4922805074819232, -1.474855988413353, -1.3262341224940994, -1.3433518932104842, -1.148255309119935, -1.2108637890969522, -1.170299113941676, -1.1037421737466713, -1.0299891380609636, -0.9801008341287212, -0.9219634549696611, -0.9440461984330617, -0.949804377990094, -0.9885212669436836, -1.0256829385851371, -1.092873856822888, -1.2608438528577888, -1.3027736041696603, -1.3753573093523024, -1.3663938480689783, -1.1903586828226393, -1.2538803807814365, -1.2296561913245694, -1.1579183208404322, -0.9523518474970951, -0.9687948405224851, -0.9273789482728185, -0.8936287525613079, -0.8287063101123809, -0.89361260639647, -0.8566734799821512, -0.9633913927865125, -0.930982596691703, -1.0173425290824007, -1.0502330054462965, -1.0460491041715922, -1.1066482922541603, -1.0229610734038308, -0.946362419019918, -1.0331157273243892, -0.9874075984844329, -1.0337622602453653, -1.1325618354127178, -1.1113564082958935, -1.3232618273810468, -1.3439881342221736, -1.3713247605456274, -1.4038137962805697, -1.5491725053059193, -1.4841590287800477, -1.50520534277329, -1.355380088374908, -1.342253809326822, -1.26080369201156, -1.3406527075717005, -1.2320227367651089, -1.1141442807091895, -1.0909547976576186, -0.9540556636574512, -0.9860605117821389, -0.9356540439356138, -0.9377912768226226, -0.8857871716175939, -0.9384607110429724, -1.0733243486264663, -1.143056463993184, -1.2162740042438915, -1.2822887309171298, -1.2431976299079925, -1.3339066849209409, -1.3297762401849913, -1.2231484954481255, -1.0920329828785025, -1.1800048627095034, -1.0328385682751826, -0.9202726745569217, -0.7891465772196437, -0.8762666442708366, -0.8784231692731526, -0.8717536169965459, -0.8603014629464936, -0.9226663401142484, -0.9819918646207833, -0.9097043171805149, -1.0391453679781164, -0.9906326441860036, -1.0915032920557335, -1.0608382893948707, -0.9929730232149548, -1.0420644752097894, -0.9075342280312644, -1.0900297099874714, -1.158505114064349, -1.097432139767959, -1.3377460636392786, -1.3955892901697475, -1.4135665251561769, -1.4412389424090744, -1.547702582250635, -1.4959451430179962, -1.4829898408214726, -1.462563567091084, -1.41932614034306, -1.2829777790162316, -1.1146926650263587, -1.1547526038403548, -1.1381595627684011, -0.9968758247992633, -1.0262957815583647, -0.9749502120447681, -0.9474722009574156, -0.9230359787169641, -0.9228611755947898, -0.9665611614309083, -1.0332284786854837, -1.126236833497266, -1.2415048570179552, -1.202358663939116, -1.302625950830468, -1.2919299268612776, -1.2109660266822126, -1.223029696230018, -1.146203360911823, -1.04322395678094, -1.067441817682095, -0.9384951034814264, -0.8819257164076604, -0.7802247105535728, -0.8847861901734233, -0.8278844293526411, -0.9727255949214483, -0.9088699584498551, -0.909283992724327, -1.0602938994992108, -1.049115026219483, -1.0838396855648236, -1.099354029693472, -1.0991181806527086, -1.0352571864811166, -0.912042018895675, -1.0669180542157057, -1.0913646542328168, -1.1229398349215658, -1.3194876846639794, -1.3033349559935388, -1.4686132454704097, -1.5381050213529202, -1.4900032373003065, -1.4321854813962085, -1.532427222954205, -1.4976727140596693, -1.4382496653991432, -1.3442376786640833, -1.3336434067107883, -1.1632603878446588, -1.2172793937290909, -1.0604347948010637, -1.031032133803601, -0.9581722564315434, -1.0146383797910858, -0.944432859524993, -0.8691707703538908, -0.9225368218258704, -0.9795864659593785, -0.8815830883518108, -1.128362532866554, -1.169702435779416, -1.278496184783831, -1.332390242524425, -1.2435415920629027, -1.3418778988547932, -1.2541078864208102, -1.1018046739923812, -1.0157833995179433, -0.9593342499913821, -0.943846730987557, -0.8432493600910623, -0.8810471427656916, -0.8627731693147348, -0.8181214820983312, -0.9059203435094004]"

dataset/Traffic.csv

@@ -0,0 +1,2 @@
+Hist,Pred
+"[0.5481484285024462, 0.1368960204047248, -0.2229498366807815, -0.6393428998797244, -0.9117976202444646, -1.1482677549006544, -1.323050028342186, -1.3076280630385213, -1.323050028342186, -1.2510808569250849, -0.9323602406493506, -0.30005966319910415, 0.1625992959108324, 0.45047598157923735, 0.6149769448183257, 1.0416513182197116, 1.2884027630783443, 1.3809345549003318, 1.3449499691917812, 1.4888883120259835, 1.4580443814186546, 1.1084798345355915, 0.9645414917013887, 0.728071357045199, 0.39392877546580046, 0.0032389877729654547, -0.3977321101223129, -0.6650461753858319, -0.9632041712566799, -1.1893929957104266, -1.3281906834434074, -1.4001598548605088, -1.4104411650629518, -1.3744565793544012, -1.173971030406762, -0.9683448263579013, -0.5211078325516294, -0.1149960795551296, 0.26027174283404114, 0.5532890836036675, 0.6509615305268766, 1.067354593725819, 0.8103218386647435, 0.6715241509317624, 0.9285569059928381, 0.7589152876525282, 0.6201175999195473, 0.31167829384625634, -0.10985542445390811, -0.46456062643819274, -0.7678592774102622, -1.0762985834835532, -1.2819247875324138, -1.4515664058727238, -1.441285095670281, -1.364175269151958, -1.173971030406762, -0.7318746917017116, -0.1921059060734524, 0.35280353465602826, 0.8720096998794017, 0.589273669312218, 0.5430077734012244, 1.015948042713604, 1.2010116263575785, 1.1598863855478065, 1.6790925507711794, 1.8795780997188185, 1.7459210670870595, 2.3319557486263123, 1.643107965062629, 0.985104112106275, 0.15745864080961072, -0.2229498366807815, -0.7729999325114837, -1.0814392385847746, -1.2613621671275277, -1.364175269151958, -1.4258631303666163, -1.323050028342186, -1.0814392385847746, -0.695890105993161, -0.007042322429477873, 0.4453353264780157, 0.7897592182598572, 0.4710386019841233, 0.5173044978951169, 0.7126493917415347, 1.236996212066129, 1.3603719344954457, 1.8024682732004964, 2.1160482343750084, 2.0595010282615718, 2.095485613970123, 1.4374817610137682, 1.0879172141307052, 0.476179257085345, -0.01218297753069918, -0.6136396243736167, -0.9837667916615658, -1.1996743059128696, -1.3744565793544012, -1.3898785446580657, -1.3076280630385213, -1.1020018589896607, -0.6804681406894963, 0.08034881429128791, 0.38878812036457916, 1.062213938624598, 0.6252582550207689, 0.5070231876926738, 0.8463064243732941, 1.2112929365600218, 1.581420103847971, 1.5197322426333126, 1.84873416911149, 1.7099364813785085, 2.0646416833627934, 1.591701414050414, 1.1393237651429202]","[0.5944143244134397, 0.0032389877729654547, -0.6136396243736167, -0.9991887569652305, -1.1791116855079835, -1.3436126487470719, -1.4053005099617302, -1.3384719936458507, -1.1688303753055402, -0.567373728462623, -0.007042322429477873, 0.39906943056702215, 0.985104112106275, 0.7640559427537496, 0.7383526672476424, 0.8720096998794017, 1.4066378304064393, 1.365512589596667, 1.8898594099212618, 1.8744374446175975, 2.033797752755464, 2.1160482343750084, 1.6688112405687368, 1.1187611447380343]"

model/model.py

@@ -0,0 +1,124 @@
+import re
+from statistics import mode
+
+import numpy as np
+import torch
+from transformers import (
+    LlamaForCausalLM,
+    LlamaTokenizer,
+    pipeline,
+)
+
+from utils.prompt import getPrompt
+from utils.tools import Discretizer, Serializer
+
+
+class ChatTime:
+    def __init__(self, model_path, hist_len=None, pred_len=None,
+                 max_pred_len=16, num_samples=8, top_k=100, top_p=1.0, temperature=1.0):
+        self.model_path = model_path
+        self.hist_len = hist_len
+        self.pred_len = pred_len
+
+        self.max_pred_len = max_pred_len
+        self.num_samples = num_samples
+        self.top_k = top_k
+        self.top_p = top_p
+        self.temperature = temperature
+
+        self.discretizer = Discretizer()
+        self.serializer = Serializer()
+
+        self.model = LlamaForCausalLM.from_pretrained(
+            self.model_path,
+            low_cpu_mem_usage=True,
+            return_dict=True,
+            torch_dtype=torch.float16,
+            device_map="auto",
+        )
+
+        self.tokenizer = LlamaTokenizer.from_pretrained(self.model_path, trust_remote_code=True)
+        self.tokenizer.pad_token = self.tokenizer.eos_token
+        self.tokenizer.padding_side = "right"
+        self.eos_token_id = self.tokenizer.eos_token_id
+
+    def predict(self, hist_data, context=None):
+        if self.hist_len is None or self.pred_len is None:
+            raise ValueError("hist_len and pred_len must be specified before prediction")
+
+        series = hist_data
+        prediction_list = []
+        remaining = self.pred_len
+
+        while remaining > 0:
+            dispersed_series = self.discretizer.discretize(series)
+            serialized_series = self.serializer.serialize(dispersed_series)
+            serialized_series = getPrompt(flag="prediction", context=context, input=serialized_series)
+
+            pipe = pipeline(
+                task="text-generation",
+                model=self.model,
+                tokenizer=self.tokenizer,
+                min_new_tokens=2 * min(remaining, self.max_pred_len) + 8,
+                max_new_tokens=2 * min(remaining, self.max_pred_len) + 8,
+                do_sample=True,
+                num_return_sequences=self.num_samples,
+                top_k=self.top_k,
+                top_p=self.top_p,
+                temperature=self.temperature,
+                eos_token_id=self.eos_token_id,
+            )
+            samples = pipe(serialized_series)
+
+            pred_list = []
+            for sample in samples:
+                serialized_prediction = sample["generated_text"].split("### Response:\n")[1]
+                dispersed_prediction = self.serializer.inverse_serialize(serialized_prediction)
+                pred = self.discretizer.inverse_discretize(dispersed_prediction)
+
+                if len(pred) < min(remaining, self.max_pred_len):
+                    pred = np.concatenate([pred, np.full(min(remaining, self.max_pred_len) - len(pred), np.NaN)])
+
+                pred_list.append(pred[:min(remaining, self.max_pred_len)])
+
+            prediction = np.nanmedian(pred_list, axis=0)
+            prediction_list.append(prediction)
+            remaining -= prediction.shape[-1]
+
+            if remaining <= 0:
+                break
+
+            series = np.concatenate([series, prediction], axis=-1)
+
+        prediction = np.concatenate(prediction_list, axis=-1)
+
+        return prediction
+
+    def analyze(self, question, series):
+        dispersed_series = self.discretizer.discretize(series)
+        serialized_series = self.serializer.serialize(dispersed_series)
+        serialized_series = getPrompt(flag="analysis", instruction=question, input=serialized_series)
+
+        pipe = pipeline(
+            task="text-generation",
+            model=self.model,
+            tokenizer=self.tokenizer,
+            max_new_tokens=self.max_pred_len,
+            do_sample=True,
+            num_return_sequences=self.num_samples,
+            top_k=self.top_k,
+            top_p=self.top_p,
+            temperature=self.temperature,
+            eos_token_id=self.eos_token_id,
+        )
+        samples = pipe(serialized_series)
+
+        response_list = []
+        for sample in samples:
+            response = sample["generated_text"].split("### Response:\n")[1].split('.')[0] + "."
+            response = re.findall(r"\([abc]\)", response)[0]
+            response_list.append(response)
+
+        response = mode(response_list)
+
+        return response

rats40k_adapter/logs/pipeline_20260608_175250_zeroshot.log

@@ -0,0 +1,27 @@
+accelerate: 0.34.2
+Run id: 20260608_175250
+Python: /dev/shm/suiqk/conda_envs/scalerag-ts-v4/bin/python
+Accelerate: /dev/shm/suiqk/conda_envs/scalerag-ts-v4/bin/python -m accelerate.commands.accelerate_cli
+CUDA_VISIBLE_DEVICES: 0,1,2,3
+Model path: /mnt/share01/sqk/models/ChatTime-1-7B-Chat
+Data path: /mnt/share01/sqk/datasets/RATs40K/RATs-Uni-TSImage_Reason.json
+Output dir: /mnt/share01/sqk/ChatTime/rats40k_adapter/outputs/pipeline_20260608_175250/zeroshot
+Log file: /mnt/share01/sqk/ChatTime/rats40k_adapter/logs/pipeline_20260608_175250_zeroshot.log
+Dataset: /mnt/share01/sqk/datasets/RATs40K/RATs-Uni-TSImage_Reason.json
+Split: TSAD_test
+Total samples: 6034
+World size: 4
+Per-device eval batch size: 4
+Maximum global eval batch size: 16
+Output dir: /mnt/share01/sqk/ChatTime/rats40k_adapter/outputs/pipeline_20260608_175250/zeroshot
+
+
+
+
+
+
+
+
+[rank0]:[W608 19:20:59.606662159 ProcessGroupNCCL.cpp:4561] [PG ID 0 PG GUID 0 Rank 0]  using GPU 0 to perform barrier as devices used by this process are currently unknown. This can potentially cause a hang if this rank to GPU mapping is incorrect. Specify device_ids in barrier() to force use of a particular device, or call init_process_group() with a device_id.
+Saved predictions: /mnt/share01/sqk/ChatTime/rats40k_adapter/outputs/pipeline_20260608_175250/zeroshot/RATs-Uni-TSImage_Reason_Reason_by_chattime_zeroshot.json
+Saved metrics: /mnt/share01/sqk/ChatTime/rats40k_adapter/outputs/pipeline_20260608_175250/zeroshot/TSAD_test_metrics.json

rats40k_adapter/logs/sft_4gpu_20260608_172354.log

@@ -0,0 +1,148 @@
+accelerate: 0.34.2
+Run id: 20260608_172354
+Python: /dev/shm/suiqk/conda_envs/scalerag-ts-v4/bin/python
+Accelerate: /dev/shm/suiqk/conda_envs/scalerag-ts-v4/bin/python -m accelerate.commands.accelerate_cli
+CUDA_VISIBLE_DEVICES: 0,1,2,3
+Model path: /mnt/share01/sqk/models/ChatTime-1-7B-Chat
+Data path: /mnt/share01/sqk/datasets/RATs40K/RATs-Uni-TSImage_Reason.json
+Adapter output dir: /mnt/share01/sqk/ChatTime/rats40k_adapter/outputs/sft_20260608_172354/adapter
+Eval output dir: /mnt/share01/sqk/ChatTime/rats40k_adapter/outputs/sft_20260608_172354/eval
+Log file: /mnt/share01/sqk/ChatTime/rats40k_adapter/logs/sft_4gpu_20260608_172354.log
+⚙️  Running in WANDB offline mode
+⚙️  Running in WANDB offline mode
+⚙️  Running in WANDB offline mode
+⚙️  Running in WANDB offline mode
+SFT token length check: max_prompt_len=2818, max_total_len=3134, max_seq_length=4096
+Traceback (most recent call last):
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/importlib/metadata/__init__.py", line 563, in from_name
+    return next(cls.discover(name=name))
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+StopIteration
+
+During handling of the above exception, another exception occurred:
+
+Traceback (most recent call last):
+  File "/mnt/share01/sqk/ChatTime/rats40k_adapter/finetune_rats40k_lora.py", line 283, in <module>
+    main()
+  File "/mnt/share01/sqk/ChatTime/rats40k_adapter/finetune_rats40k_lora.py", line 249, in main
+    model = load_model(args)
+            ^^^^^^^^^^^^^^^^
+  File "/mnt/share01/sqk/ChatTime/rats40k_adapter/finetune_rats40k_lora.py", line 185, in load_model
+    quantization_config = BitsAndBytesConfig(
+                          ^^^^^^^^^^^^^^^^^^^
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/transformers/utils/quantization_config.py", line 509, in __init__
+    self.post_init()
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/transformers/utils/quantization_config.py", line 567, in post_init
+    if self.load_in_4bit and not version.parse(importlib.metadata.version("bitsandbytes")) >= version.parse(
+                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/importlib/metadata/__init__.py", line 1009, in version
+    return distribution(distribution_name).version
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/importlib/metadata/__init__.py", line 982, in distribution
+    return Distribution.from_name(distribution_name)
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/importlib/metadata/__init__.py", line 565, in from_name
+    raise PackageNotFoundError(name)
+importlib.metadata.PackageNotFoundError: No package metadata was found for bitsandbytes
+SFT token length check: max_prompt_len=2818, max_total_len=3134, max_seq_length=4096
+Traceback (most recent call last):
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/importlib/metadata/__init__.py", line 563, in from_name
+    return next(cls.discover(name=name))
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+StopIteration
+
+During handling of the above exception, another exception occurred:
+
+Traceback (most recent call last):
+  File "/mnt/share01/sqk/ChatTime/rats40k_adapter/finetune_rats40k_lora.py", line 283, in <module>
+    main()
+  File "/mnt/share01/sqk/ChatTime/rats40k_adapter/finetune_rats40k_lora.py", line 249, in main
+    model = load_model(args)
+            ^^^^^^^^^^^^^^^^
+  File "/mnt/share01/sqk/ChatTime/rats40k_adapter/finetune_rats40k_lora.py", line 185, in load_model
+    quantization_config = BitsAndBytesConfig(
+                          ^^^^^^^^^^^^^^^^^^^
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/transformers/utils/quantization_config.py", line 509, in __init__
+    self.post_init()
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/transformers/utils/quantization_config.py", line 567, in post_init
+    if self.load_in_4bit and not version.parse(importlib.metadata.version("bitsandbytes")) >= version.parse(
+                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/importlib/metadata/__init__.py", line 1009, in version
+    return distribution(distribution_name).version
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/importlib/metadata/__init__.py", line 982, in distribution
+    return Distribution.from_name(distribution_name)
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/importlib/metadata/__init__.py", line 565, in from_name
+    raise PackageNotFoundError(name)
+importlib.metadata.PackageNotFoundError: No package metadata was found for bitsandbytes
+SFT token length check: max_prompt_len=2818, max_total_len=3134, max_seq_length=4096
+Traceback (most recent call last):
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/importlib/metadata/__init__.py", line 563, in from_name
+    return next(cls.discover(name=name))
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+StopIteration
+
+During handling of the above exception, another exception occurred:
+
+Traceback (most recent call last):
+  File "/mnt/share01/sqk/ChatTime/rats40k_adapter/finetune_rats40k_lora.py", line 283, in <module>
+    main()
+  File "/mnt/share01/sqk/ChatTime/rats40k_adapter/finetune_rats40k_lora.py", line 249, in main
+    model = load_model(args)
+            ^^^^^^^^^^^^^^^^
+  File "/mnt/share01/sqk/ChatTime/rats40k_adapter/finetune_rats40k_lora.py", line 185, in load_model
+    quantization_config = BitsAndBytesConfig(
+                          ^^^^^^^^^^^^^^^^^^^
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/transformers/utils/quantization_config.py", line 509, in __init__
+    self.post_init()
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/transformers/utils/quantization_config.py", line 567, in post_init
+    if self.load_in_4bit and not version.parse(importlib.metadata.version("bitsandbytes")) >= version.parse(
+                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/importlib/metadata/__init__.py", line 1009, in version
+    return distribution(distribution_name).version
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/importlib/metadata/__init__.py", line 982, in distribution
+    return Distribution.from_name(distribution_name)
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/importlib/metadata/__init__.py", line 565, in from_name
+    raise PackageNotFoundError(name)
+importlib.metadata.PackageNotFoundError: No package metadata was found for bitsandbytes
+W0608 17:25:51.569000 4051992 site-packages/torch/distributed/elastic/multiprocessing/api.py:897] Sending process 4052417 closing signal SIGTERM
+W0608 17:25:51.569000 4051992 site-packages/torch/distributed/elastic/multiprocessing/api.py:897] Sending process 4052418 closing signal SIGTERM
+W0608 17:25:51.570000 4051992 site-packages/torch/distributed/elastic/multiprocessing/api.py:897] Sending process 4052419 closing signal SIGTERM
+E0608 17:25:51.748000 4051992 site-packages/torch/distributed/elastic/multiprocessing/api.py:869] failed (exitcode: 1) local_rank: 3 (pid: 4052420) of binary: /dev/shm/suiqk/conda_envs/scalerag-ts-v4/bin/python
+Traceback (most recent call last):
+  File "<frozen runpy>", line 198, in _run_module_as_main
+  File "<frozen runpy>", line 88, in _run_code
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/accelerate/commands/accelerate_cli.py", line 52, in <module>
+    main()
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/accelerate/commands/accelerate_cli.py", line 48, in main
+    args.func(args)
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/accelerate/commands/launch.py", line 1165, in launch_command
+    multi_gpu_launcher(args)
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/accelerate/commands/launch.py", line 799, in multi_gpu_launcher
+    distrib_run.run(args)
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/torch/distributed/run.py", line 909, in run
+    elastic_launch(
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/torch/distributed/launcher/api.py", line 138, in __call__
+    return launch_agent(self._config, self._entrypoint, list(args))
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/torch/distributed/launcher/api.py", line 269, in launch_agent
+    raise ChildFailedError(
+torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
+============================================================
+rats40k_adapter/finetune_rats40k_lora.py FAILED
+------------------------------------------------------------
+Failures:
+  <NO_OTHER_FAILURES>
+------------------------------------------------------------
+Root Cause (first observed failure):
+[0]:
+  time      : 2026-06-08_17:25:51
+  host      : simtech-gpuserver
+  rank      : 3 (local_rank: 3)
+  exitcode  : 1 (pid: 4052420)
+  error_file: <N/A>
+  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
+============================================================

rats40k_adapter/logs/sft_4gpu_20260608_173130.log

@@ -0,0 +1,138 @@
+accelerate: 0.34.2
+Run id: 20260608_173130
+Python: /dev/shm/suiqk/conda_envs/scalerag-ts-v4/bin/python
+Accelerate: /dev/shm/suiqk/conda_envs/scalerag-ts-v4/bin/python -m accelerate.commands.accelerate_cli
+CUDA_VISIBLE_DEVICES: 0,1,2,3
+Model path: /mnt/share01/sqk/models/ChatTime-1-7B-Chat
+Data path: /mnt/share01/sqk/datasets/RATs40K/RATs-Uni-TSImage_Reason.json
+Adapter output dir: /mnt/share01/sqk/ChatTime/rats40k_adapter/outputs/sft_20260608_173130/adapter
+Eval output dir: /mnt/share01/sqk/ChatTime/rats40k_adapter/outputs/sft_20260608_173130/eval
+Log file: /mnt/share01/sqk/ChatTime/rats40k_adapter/logs/sft_4gpu_20260608_173130.log
+⚙️  Running in WANDB offline mode
+⚙️  Running in WANDB offline mode
+⚙️  Running in WANDB offline mode
+⚙️  Running in WANDB offline mode
+SFT token length check: max_prompt_len=2818, max_total_len=3134, max_seq_length=4096
+SFT token length check: max_prompt_len=2818, max_total_len=3134, max_seq_length=4096
+SFT token length check: max_prompt_len=2818, max_total_len=3134, max_seq_length=4096
+
+
+
+
+
+trainable params: 39,976,960 || all params: 6,860,320,768 || trainable%: 0.5827
+trainable params: 39,976,960 || all params: 6,860,320,768 || trainable%: 0.5827
+/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/accelerate/accelerator.py:494: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.
+  self.scaler = torch.cuda.amp.GradScaler(**kwargs)
+No label_names provided for model class `PeftModelForCausalLM`. Since `PeftModel` hides base models input arguments, if label_names is not given, label_names can't be set automatically within `Trainer`. Note that empty label_names list will be used instead.
+/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/accelerate/accelerator.py:494: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.
+  self.scaler = torch.cuda.amp.GradScaler(**kwargs)
+No label_names provided for model class `PeftModelForCausalLM`. Since `PeftModel` hides base models input arguments, if label_names is not given, label_names can't be set automatically within `Trainer`. Note that empty label_names list will be used instead.
+trainable params: 39,976,960 || all params: 6,860,320,768 || trainable%: 0.5827
+/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/accelerate/accelerator.py:494: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.
+  self.scaler = torch.cuda.amp.GradScaler(**kwargs)
+Detected kernel version 5.4.0, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher.
+No label_names provided for model class `PeftModelForCausalLM`. Since `PeftModel` hides base models input arguments, if label_names is not given, label_names can't be set automatically within `Trainer`. Note that empty label_names list will be used instead.
+[rank1]: Traceback (most recent call last):
+[rank1]:   File "/mnt/share01/sqk/ChatTime/rats40k_adapter/finetune_rats40k_lora.py", line 283, in <module>
+[rank1]:     main()
+[rank1]:   File "/mnt/share01/sqk/ChatTime/rats40k_adapter/finetune_rats40k_lora.py", line 277, in main
+[rank1]:     trainer.train()
+[rank1]:   File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/transformers/trainer.py", line 2237, in train
+[rank1]:     return inner_training_loop(
+[rank1]:            ^^^^^^^^^^^^^^^^^^^^
+[rank1]:   File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/transformers/trainer.py", line 2361, in _inner_training_loop
+[rank1]:     model = self._wrap_model(self.model_wrapped)
+[rank1]:             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+[rank1]:   File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/transformers/trainer.py", line 1985, in _wrap_model
+[rank1]:     if self.accelerator.unwrap_model(model, keep_torch_compile=False) is not model:
+[rank1]:        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+[rank1]: TypeError: Accelerator.unwrap_model() got an unexpected keyword argument 'keep_torch_compile'
+[rank3]: Traceback (most recent call last):
+[rank3]:   File "/mnt/share01/sqk/ChatTime/rats40k_adapter/finetune_rats40k_lora.py", line 283, in <module>
+[rank3]:     main()
+[rank3]:   File "/mnt/share01/sqk/ChatTime/rats40k_adapter/finetune_rats40k_lora.py", line 277, in main
+[rank3]:     trainer.train()
+[rank3]:   File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/transformers/trainer.py", line 2237, in train
+[rank3]:     return inner_training_loop(
+[rank3]:            ^^^^^^^^^^^^^^^^^^^^
+[rank3]:   File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/transformers/trainer.py", line 2361, in _inner_training_loop
+[rank3]:     model = self._wrap_model(self.model_wrapped)
+[rank3]:             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+[rank3]:   File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/transformers/trainer.py", line 1985, in _wrap_model
+[rank3]:     if self.accelerator.unwrap_model(model, keep_torch_compile=False) is not model:
+[rank3]:        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+[rank3]: TypeError: Accelerator.unwrap_model() got an unexpected keyword argument 'keep_torch_compile'
+[rank0]: Traceback (most recent call last):
+[rank0]:   File "/mnt/share01/sqk/ChatTime/rats40k_adapter/finetune_rats40k_lora.py", line 283, in <module>
+[rank0]:     main()
+[rank0]:   File "/mnt/share01/sqk/ChatTime/rats40k_adapter/finetune_rats40k_lora.py", line 277, in main
+[rank0]:     trainer.train()
+[rank0]:   File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/transformers/trainer.py", line 2237, in train
+[rank0]:     return inner_training_loop(
+[rank0]:            ^^^^^^^^^^^^^^^^^^^^
+[rank0]:   File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/transformers/trainer.py", line 2361, in _inner_training_loop
+[rank0]:     model = self._wrap_model(self.model_wrapped)
+[rank0]:             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+[rank0]:   File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/transformers/trainer.py", line 1985, in _wrap_model
+[rank0]:     if self.accelerator.unwrap_model(model, keep_torch_compile=False) is not model:
+[rank0]:        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+[rank0]: TypeError: Accelerator.unwrap_model() got an unexpected keyword argument 'keep_torch_compile'
+trainable params: 39,976,960 || all params: 6,860,320,768 || trainable%: 0.5827
+/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/accelerate/accelerator.py:494: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.
+  self.scaler = torch.cuda.amp.GradScaler(**kwargs)
+No label_names provided for model class `PeftModelForCausalLM`. Since `PeftModel` hides base models input arguments, if label_names is not given, label_names can't be set automatically within `Trainer`. Note that empty label_names list will be used instead.
+[rank0]:[W608 17:36:00.577053648 ProcessGroupNCCL.cpp:1496] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())
+[rank2]: Traceback (most recent call last):
+[rank2]:   File "/mnt/share01/sqk/ChatTime/rats40k_adapter/finetune_rats40k_lora.py", line 283, in <module>
+[rank2]:     main()
+[rank2]:   File "/mnt/share01/sqk/ChatTime/rats40k_adapter/finetune_rats40k_lora.py", line 277, in main
+[rank2]:     trainer.train()
+[rank2]:   File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/transformers/trainer.py", line 2237, in train
+[rank2]:     return inner_training_loop(
+[rank2]:            ^^^^^^^^^^^^^^^^^^^^
+[rank2]:   File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/transformers/trainer.py", line 2361, in _inner_training_loop
+[rank2]:     model = self._wrap_model(self.model_wrapped)
+[rank2]:             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+[rank2]:   File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/transformers/trainer.py", line 1985, in _wrap_model
+[rank2]:     if self.accelerator.unwrap_model(model, keep_torch_compile=False) is not model:
+[rank2]:        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+[rank2]: TypeError: Accelerator.unwrap_model() got an unexpected keyword argument 'keep_torch_compile'
+W0608 17:36:00.830000 4087788 site-packages/torch/distributed/elastic/multiprocessing/api.py:897] Sending process 4088224 closing signal SIGTERM
+W0608 17:36:00.831000 4087788 site-packages/torch/distributed/elastic/multiprocessing/api.py:897] Sending process 4088226 closing signal SIGTERM
+W0608 17:36:00.832000 4087788 site-packages/torch/distributed/elastic/multiprocessing/api.py:897] Sending process 4088227 closing signal SIGTERM
+E0608 17:36:01.029000 4087788 site-packages/torch/distributed/elastic/multiprocessing/api.py:869] failed (exitcode: 1) local_rank: 1 (pid: 4088225) of binary: /dev/shm/suiqk/conda_envs/scalerag-ts-v4/bin/python
+Traceback (most recent call last):
+  File "<frozen runpy>", line 198, in _run_module_as_main
+  File "<frozen runpy>", line 88, in _run_code
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/accelerate/commands/accelerate_cli.py", line 52, in <module>
+    main()
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/accelerate/commands/accelerate_cli.py", line 48, in main
+    args.func(args)
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/accelerate/commands/launch.py", line 1165, in launch_command
+    multi_gpu_launcher(args)
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/accelerate/commands/launch.py", line 799, in multi_gpu_launcher
+    distrib_run.run(args)
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/torch/distributed/run.py", line 909, in run
+    elastic_launch(
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/torch/distributed/launcher/api.py", line 138, in __call__
+    return launch_agent(self._config, self._entrypoint, list(args))
+           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/torch/distributed/launcher/api.py", line 269, in launch_agent
+    raise ChildFailedError(
+torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
+============================================================
+rats40k_adapter/finetune_rats40k_lora.py FAILED
+------------------------------------------------------------
+Failures:
+  <NO_OTHER_FAILURES>
+------------------------------------------------------------
+Root Cause (first observed failure):
+[0]:
+  time      : 2026-06-08_17:36:00
+  host      : simtech-gpuserver
+  rank      : 1 (local_rank: 1)
+  exitcode  : 1 (pid: 4088225)
+  error_file: <N/A>
+  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
+============================================================

rats40k_adapter/logs/sft_4gpu_20260608_174152.log

@@ -0,0 +1,48 @@
+accelerate: 0.34.2
+Run id: 20260608_174152
+Python: /dev/shm/suiqk/conda_envs/scalerag-ts-v4/bin/python
+Accelerate: /dev/shm/suiqk/conda_envs/scalerag-ts-v4/bin/python -m accelerate.commands.accelerate_cli
+CUDA_VISIBLE_DEVICES: 0,1,2,3
+Model path: /mnt/share01/sqk/models/ChatTime-1-7B-Chat
+Data path: /mnt/share01/sqk/datasets/RATs40K/RATs-Uni-TSImage_Reason.json
+Adapter output dir: /mnt/share01/sqk/ChatTime/rats40k_adapter/outputs/sft_20260608_174152/adapter
+Eval output dir: /mnt/share01/sqk/ChatTime/rats40k_adapter/outputs/sft_20260608_174152/eval
+Log file: /mnt/share01/sqk/ChatTime/rats40k_adapter/logs/sft_4gpu_20260608_174152.log
+⚙️  Running in WANDB offline mode
+⚙️  Running in WANDB offline mode
+⚙️  Running in WANDB offline mode
+⚙️  Running in WANDB offline mode
+Applied accelerate compatibility patch: Accelerator.unwrap_model accepts keep_torch_compile.
+Applied accelerate compatibility patch: Accelerator.unwrap_model accepts keep_torch_compile.
+Applied accelerate compatibility patch: Accelerator.unwrap_model accepts keep_torch_compile.
+Applied accelerate compatibility patch: Accelerator.unwrap_model accepts keep_torch_compile.
+SFT token length check: max_prompt_len=2818, max_total_len=3134, max_seq_length=4096
+
+SFT token length check: max_prompt_len=2818, max_total_len=3134, max_seq_length=4096
+
+
+
+
+
+trainable params: 39,976,960 || all params: 6,860,320,768 || trainable%: 0.5827
+/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/accelerate/accelerator.py:494: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.
+  self.scaler = torch.cuda.amp.GradScaler(**kwargs)
+No label_names provided for model class `PeftModelForCausalLM`. Since `PeftModel` hides base models input arguments, if label_names is not given, label_names can't be set automatically within `Trainer`. Note that empty label_names list will be used instead.
+trainable params: 39,976,960 || all params: 6,860,320,768 || trainable%: 0.5827
+trainable params: 39,976,960 || all params: 6,860,320,768 || trainable%: 0.5827
+/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/accelerate/accelerator.py:494: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.
+  self.scaler = torch.cuda.amp.GradScaler(**kwargs)
+No label_names provided for model class `PeftModelForCausalLM`. Since `PeftModel` hides base models input arguments, if label_names is not given, label_names can't be set automatically within `Trainer`. Note that empty label_names list will be used instead.
+/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/accelerate/accelerator.py:494: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.
+  self.scaler = torch.cuda.amp.GradScaler(**kwargs)
+No label_names provided for model class `PeftModelForCausalLM`. Since `PeftModel` hides base models input arguments, if label_names is not given, label_names can't be set automatically within `Trainer`. Note that empty label_names list will be used instead.
+trainable params: 39,976,960 || all params: 6,860,320,768 || trainable%: 0.5827
+/dev/shm/suiqk/conda_envs/scalerag-ts-v4/lib/python3.11/site-packages/accelerate/accelerator.py:494: FutureWarning: `torch.cuda.amp.GradScaler(args...)` is deprecated. Please use `torch.amp.GradScaler('cuda', args...)` instead.
+  self.scaler = torch.cuda.amp.GradScaler(**kwargs)
+Detected kernel version 5.4.0, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher.
+No label_names provided for model class `PeftModelForCausalLM`. Since `PeftModel` hides base models input arguments, if label_names is not given, label_names can't be set automatically within `Trainer`. Note that empty label_names list will be used instead.
+
  0%|          | 0/946 [00:00<?, ?it/s]`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.
+`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.
+`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.
+`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.
+
  0%|          | 1/946 [00:41<11:01:04, 41.97s/it]
  0%|          | 2/946 [01:24<11:08:02, 42.46s/it]
  0%|          | 3/946 [02:07<11:08:46, 42.55s/it]
  0%|          | 4/946 [02:49<11:08:05, 42.55s/it]
  1%|          | 5/946 [03:32<11:05:36, 42.44s/it]
  1%|          | 6/946 [04:14<11:04:02, 42.39s/it]
  1%|          | 7/946 [04:57<11:04:53, 42.49s/it]
  1%|          | 8/946 [05:39<11:03:06, 42.42s/it]

rats40k_adapter/outputs/pipeline_20260608_175250/sft/adapter/checkpoint-800/README.md

@@ -0,0 +1,202 @@
+---
+base_model: /mnt/share01/sqk/models/ChatTime-1-7B-Chat
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

rats40k_adapter/outputs/pipeline_20260608_175250/sft/adapter/checkpoint-800/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/mnt/share01/sqk/models/ChatTime-1-7B-Chat",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "o_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

rats40k_adapter/outputs/pipeline_20260608_175250/sft/adapter/checkpoint-800/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

rats40k_adapter/outputs/pipeline_20260608_175250/sft/adapter/checkpoint-800/trainer_state.json

@@ -0,0 +1,594 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.6914232853178275,
+  "eval_steps": 500,
+  "global_step": 800,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.021144442976080348,
+      "grad_norm": 6.2254319190979,
+      "learning_rate": 3.3333333333333335e-05,
+      "loss": 3.0149,
+      "step": 10
+    },
+    {
+      "epoch": 0.042288885952160696,
+      "grad_norm": 0.7327573895454407,
+      "learning_rate": 7.500000000000001e-05,
+      "loss": 1.623,
+      "step": 20
+    },
+    {
+      "epoch": 0.06343332892824105,
+      "grad_norm": 0.5618261098861694,
+      "learning_rate": 0.00011250000000000001,
+      "loss": 1.1099,
+      "step": 30
+    },
+    {
+      "epoch": 0.08457777190432139,
+      "grad_norm": 0.48980122804641724,
+      "learning_rate": 0.00015416666666666668,
+      "loss": 0.9131,
+      "step": 40
+    },
+    {
+      "epoch": 0.10572221488040175,
+      "grad_norm": 0.488565593957901,
+      "learning_rate": 0.00019583333333333334,
+      "loss": 0.7716,
+      "step": 50
+    },
+    {
+      "epoch": 0.1268666578564821,
+      "grad_norm": 0.4297373592853546,
+      "learning_rate": 0.0001979955456570156,
+      "loss": 0.723,
+      "step": 60
+    },
+    {
+      "epoch": 0.14801110083256244,
+      "grad_norm": 0.4536869525909424,
+      "learning_rate": 0.0001957683741648107,
+      "loss": 0.6879,
+      "step": 70
+    },
+    {
+      "epoch": 0.16915554380864278,
+      "grad_norm": 0.41550788283348083,
+      "learning_rate": 0.0001935412026726058,
+      "loss": 0.6586,
+      "step": 80
+    },
+    {
+      "epoch": 0.19029998678472315,
+      "grad_norm": 0.38494592905044556,
+      "learning_rate": 0.0001913140311804009,
+      "loss": 0.6324,
+      "step": 90
+    },
+    {
+      "epoch": 0.2114444297608035,
+      "grad_norm": 0.3633956015110016,
+      "learning_rate": 0.000189086859688196,
+      "loss": 0.631,
+      "step": 100
+    },
+    {
+      "epoch": 0.23258887273688383,
+      "grad_norm": 0.3775959312915802,
+      "learning_rate": 0.0001868596881959911,
+      "loss": 0.6103,
+      "step": 110
+    },
+    {
+      "epoch": 0.2537333157129642,
+      "grad_norm": 0.35080480575561523,
+      "learning_rate": 0.0001846325167037862,
+      "loss": 0.6159,
+      "step": 120
+    },
+    {
+      "epoch": 0.2748777586890445,
+      "grad_norm": 0.4399946928024292,
+      "learning_rate": 0.0001824053452115813,
+      "loss": 0.5983,
+      "step": 130
+    },
+    {
+      "epoch": 0.2960222016651249,
+      "grad_norm": 0.4049876928329468,
+      "learning_rate": 0.0001801781737193764,
+      "loss": 0.5881,
+      "step": 140
+    },
+    {
+      "epoch": 0.31716664464120525,
+      "grad_norm": 0.3834834396839142,
+      "learning_rate": 0.0001779510022271715,
+      "loss": 0.5703,
+      "step": 150
+    },
+    {
+      "epoch": 0.33831108761728557,
+      "grad_norm": 0.3201199471950531,
+      "learning_rate": 0.0001757238307349666,
+      "loss": 0.5777,
+      "step": 160
+    },
+    {
+      "epoch": 0.35945553059336594,
+      "grad_norm": 0.3475135564804077,
+      "learning_rate": 0.0001734966592427617,
+      "loss": 0.5627,
+      "step": 170
+    },
+    {
+      "epoch": 0.3805999735694463,
+      "grad_norm": 0.3944849371910095,
+      "learning_rate": 0.0001712694877505568,
+      "loss": 0.569,
+      "step": 180
+    },
+    {
+      "epoch": 0.4017444165455266,
+      "grad_norm": 0.3674592673778534,
+      "learning_rate": 0.0001690423162583519,
+      "loss": 0.5621,
+      "step": 190
+    },
+    {
+      "epoch": 0.422888859521607,
+      "grad_norm": 0.3651335835456848,
+      "learning_rate": 0.000166815144766147,
+      "loss": 0.5597,
+      "step": 200
+    },
+    {
+      "epoch": 0.4440333024976873,
+      "grad_norm": 0.3435162901878357,
+      "learning_rate": 0.0001645879732739421,
+      "loss": 0.5538,
+      "step": 210
+    },
+    {
+      "epoch": 0.46517774547376767,
+      "grad_norm": 0.3870578408241272,
+      "learning_rate": 0.0001623608017817372,
+      "loss": 0.5574,
+      "step": 220
+    },
+    {
+      "epoch": 0.48632218844984804,
+      "grad_norm": 0.40335071086883545,
+      "learning_rate": 0.0001601336302895323,
+      "loss": 0.5394,
+      "step": 230
+    },
+    {
+      "epoch": 0.5074666314259284,
+      "grad_norm": 0.3105282187461853,
+      "learning_rate": 0.0001579064587973274,
+      "loss": 0.5403,
+      "step": 240
+    },
+    {
+      "epoch": 0.5286110744020087,
+      "grad_norm": 0.3729188144207001,
+      "learning_rate": 0.00015567928730512252,
+      "loss": 0.5466,
+      "step": 250
+    },
+    {
+      "epoch": 0.549755517378089,
+      "grad_norm": 0.3619287312030792,
+      "learning_rate": 0.0001534521158129176,
+      "loss": 0.5305,
+      "step": 260
+    },
+    {
+      "epoch": 0.5708999603541695,
+      "grad_norm": 0.34232136607170105,
+      "learning_rate": 0.0001512249443207127,
+      "loss": 0.5319,
+      "step": 270
+    },
+    {
+      "epoch": 0.5920444033302498,
+      "grad_norm": 0.38660332560539246,
+      "learning_rate": 0.0001489977728285078,
+      "loss": 0.5242,
+      "step": 280
+    },
+    {
+      "epoch": 0.6131888463063301,
+      "grad_norm": 0.35314109921455383,
+      "learning_rate": 0.0001467706013363029,
+      "loss": 0.5255,
+      "step": 290
+    },
+    {
+      "epoch": 0.6343332892824105,
+      "grad_norm": 0.3418401777744293,
+      "learning_rate": 0.00014454342984409802,
+      "loss": 0.5357,
+      "step": 300
+    },
+    {
+      "epoch": 0.6554777322584908,
+      "grad_norm": 0.357149213552475,
+      "learning_rate": 0.0001423162583518931,
+      "loss": 0.5131,
+      "step": 310
+    },
+    {
+      "epoch": 0.6766221752345711,
+      "grad_norm": 0.3720100224018097,
+      "learning_rate": 0.0001400890868596882,
+      "loss": 0.5072,
+      "step": 320
+    },
+    {
+      "epoch": 0.6977666182106516,
+      "grad_norm": 0.342650443315506,
+      "learning_rate": 0.0001378619153674833,
+      "loss": 0.5194,
+      "step": 330
+    },
+    {
+      "epoch": 0.7189110611867319,
+      "grad_norm": 0.34781211614608765,
+      "learning_rate": 0.00013563474387527841,
+      "loss": 0.5094,
+      "step": 340
+    },
+    {
+      "epoch": 0.7400555041628122,
+      "grad_norm": 0.3401576280593872,
+      "learning_rate": 0.00013340757238307352,
+      "loss": 0.5192,
+      "step": 350
+    },
+    {
+      "epoch": 0.7611999471388926,
+      "grad_norm": 0.3490856885910034,
+      "learning_rate": 0.0001311804008908686,
+      "loss": 0.5045,
+      "step": 360
+    },
+    {
+      "epoch": 0.7823443901149729,
+      "grad_norm": 0.3488720655441284,
+      "learning_rate": 0.0001289532293986637,
+      "loss": 0.502,
+      "step": 370
+    },
+    {
+      "epoch": 0.8034888330910532,
+      "grad_norm": 0.37278613448143005,
+      "learning_rate": 0.00012672605790645878,
+      "loss": 0.5038,
+      "step": 380
+    },
+    {
+      "epoch": 0.8246332760671337,
+      "grad_norm": 0.3677748441696167,
+      "learning_rate": 0.00012449888641425391,
+      "loss": 0.505,
+      "step": 390
+    },
+    {
+      "epoch": 0.845777719043214,
+      "grad_norm": 0.3815574049949646,
+      "learning_rate": 0.00012227171492204902,
+      "loss": 0.4997,
+      "step": 400
+    },
+    {
+      "epoch": 0.8669221620192943,
+      "grad_norm": 0.37245893478393555,
+      "learning_rate": 0.0001200445434298441,
+      "loss": 0.4989,
+      "step": 410
+    },
+    {
+      "epoch": 0.8880666049953746,
+      "grad_norm": 0.3642374277114868,
+      "learning_rate": 0.0001178173719376392,
+      "loss": 0.4992,
+      "step": 420
+    },
+    {
+      "epoch": 0.909211047971455,
+      "grad_norm": 0.32838189601898193,
+      "learning_rate": 0.0001155902004454343,
+      "loss": 0.4947,
+      "step": 430
+    },
+    {
+      "epoch": 0.9303554909475353,
+      "grad_norm": 0.36527854204177856,
+      "learning_rate": 0.00011336302895322941,
+      "loss": 0.4952,
+      "step": 440
+    },
+    {
+      "epoch": 0.9514999339236156,
+      "grad_norm": 0.3686304986476898,
+      "learning_rate": 0.0001111358574610245,
+      "loss": 0.4964,
+      "step": 450
+    },
+    {
+      "epoch": 0.9726443768996961,
+      "grad_norm": 0.3496793210506439,
+      "learning_rate": 0.0001089086859688196,
+      "loss": 0.4827,
+      "step": 460
+    },
+    {
+      "epoch": 0.9937888198757764,
+      "grad_norm": 0.3722958266735077,
+      "learning_rate": 0.0001066815144766147,
+      "loss": 0.4838,
+      "step": 470
+    },
+    {
+      "epoch": 1.0148011100832564,
+      "grad_norm": 0.3902372717857361,
+      "learning_rate": 0.00010445434298440981,
+      "loss": 0.4696,
+      "step": 480
+    },
+    {
+      "epoch": 1.0359455530593367,
+      "grad_norm": 0.3780229687690735,
+      "learning_rate": 0.00010222717149220491,
+      "loss": 0.4686,
+      "step": 490
+    },
+    {
+      "epoch": 1.057089996035417,
+      "grad_norm": 0.3552299737930298,
+      "learning_rate": 0.0001,
+      "loss": 0.457,
+      "step": 500
+    },
+    {
+      "epoch": 1.0782344390114973,
+      "grad_norm": 0.3887428045272827,
+      "learning_rate": 9.77728285077951e-05,
+      "loss": 0.4735,
+      "step": 510
+    },
+    {
+      "epoch": 1.0993788819875776,
+      "grad_norm": 0.3928622603416443,
+      "learning_rate": 9.55456570155902e-05,
+      "loss": 0.4675,
+      "step": 520
+    },
+    {
+      "epoch": 1.120523324963658,
+      "grad_norm": 0.3686327636241913,
+      "learning_rate": 9.331848552338531e-05,
+      "loss": 0.4804,
+      "step": 530
+    },
+    {
+      "epoch": 1.1416677679397385,
+      "grad_norm": 0.35772374272346497,
+      "learning_rate": 9.109131403118041e-05,
+      "loss": 0.4609,
+      "step": 540
+    },
+    {
+      "epoch": 1.1628122109158188,
+      "grad_norm": 0.35283800959587097,
+      "learning_rate": 8.88641425389755e-05,
+      "loss": 0.4693,
+      "step": 550
+    },
+    {
+      "epoch": 1.183956653891899,
+      "grad_norm": 0.37653160095214844,
+      "learning_rate": 8.663697104677061e-05,
+      "loss": 0.4551,
+      "step": 560
+    },
+    {
+      "epoch": 1.2051010968679794,
+      "grad_norm": 0.35314637422561646,
+      "learning_rate": 8.44097995545657e-05,
+      "loss": 0.4539,
+      "step": 570
+    },
+    {
+      "epoch": 1.2262455398440597,
+      "grad_norm": 0.35260340571403503,
+      "learning_rate": 8.21826280623608e-05,
+      "loss": 0.4531,
+      "step": 580
+    },
+    {
+      "epoch": 1.24738998282014,
+      "grad_norm": 0.3616096079349518,
+      "learning_rate": 7.995545657015591e-05,
+      "loss": 0.4645,
+      "step": 590
+    },
+    {
+      "epoch": 1.2685344257962203,
+      "grad_norm": 0.3933924436569214,
+      "learning_rate": 7.7728285077951e-05,
+      "loss": 0.4469,
+      "step": 600
+    },
+    {
+      "epoch": 1.2896788687723006,
+      "grad_norm": 0.3878353536128998,
+      "learning_rate": 7.550111358574611e-05,
+      "loss": 0.467,
+      "step": 610
+    },
+    {
+      "epoch": 1.3108233117483812,
+      "grad_norm": 0.41165846586227417,
+      "learning_rate": 7.32739420935412e-05,
+      "loss": 0.4504,
+      "step": 620
+    },
+    {
+      "epoch": 1.3319677547244615,
+      "grad_norm": 0.36190614104270935,
+      "learning_rate": 7.10467706013363e-05,
+      "loss": 0.4517,
+      "step": 630
+    },
+    {
+      "epoch": 1.3531121977005418,
+      "grad_norm": 0.3983185887336731,
+      "learning_rate": 6.881959910913141e-05,
+      "loss": 0.444,
+      "step": 640
+    },
+    {
+      "epoch": 1.3742566406766221,
+      "grad_norm": 0.38672661781311035,
+      "learning_rate": 6.659242761692652e-05,
+      "loss": 0.4488,
+      "step": 650
+    },
+    {
+      "epoch": 1.3954010836527027,
+      "grad_norm": 0.36232879757881165,
+      "learning_rate": 6.436525612472161e-05,
+      "loss": 0.4371,
+      "step": 660
+    },
+    {
+      "epoch": 1.416545526628783,
+      "grad_norm": 0.40571126341819763,
+      "learning_rate": 6.21380846325167e-05,
+      "loss": 0.4427,
+      "step": 670
+    },
+    {
+      "epoch": 1.4376899696048633,
+      "grad_norm": 0.36234796047210693,
+      "learning_rate": 5.9910913140311805e-05,
+      "loss": 0.4439,
+      "step": 680
+    },
+    {
+      "epoch": 1.4588344125809436,
+      "grad_norm": 0.4014786183834076,
+      "learning_rate": 5.7683741648106904e-05,
+      "loss": 0.4548,
+      "step": 690
+    },
+    {
+      "epoch": 1.479978855557024,
+      "grad_norm": 0.3884125053882599,
+      "learning_rate": 5.545657015590201e-05,
+      "loss": 0.4531,
+      "step": 700
+    },
+    {
+      "epoch": 1.5011232985331042,
+      "grad_norm": 0.3621061146259308,
+      "learning_rate": 5.322939866369711e-05,
+      "loss": 0.4407,
+      "step": 710
+    },
+    {
+      "epoch": 1.5222677415091845,
+      "grad_norm": 0.3601549565792084,
+      "learning_rate": 5.100222717149221e-05,
+      "loss": 0.439,
+      "step": 720
+    },
+    {
+      "epoch": 1.5434121844852648,
+      "grad_norm": 0.37766754627227783,
+      "learning_rate": 4.8775055679287305e-05,
+      "loss": 0.4397,
+      "step": 730
+    },
+    {
+      "epoch": 1.5645566274613452,
+      "grad_norm": 0.38728606700897217,
+      "learning_rate": 4.654788418708241e-05,
+      "loss": 0.4455,
+      "step": 740
+    },
+    {
+      "epoch": 1.5857010704374257,
+      "grad_norm": 0.3532933294773102,
+      "learning_rate": 4.432071269487751e-05,
+      "loss": 0.4375,
+      "step": 750
+    },
+    {
+      "epoch": 1.606845513413506,
+      "grad_norm": 0.37484633922576904,
+      "learning_rate": 4.209354120267261e-05,
+      "loss": 0.4386,
+      "step": 760
+    },
+    {
+      "epoch": 1.6279899563895863,
+      "grad_norm": 0.40252485871315,
+      "learning_rate": 3.986636971046771e-05,
+      "loss": 0.4394,
+      "step": 770
+    },
+    {
+      "epoch": 1.6491343993656669,
+      "grad_norm": 0.3895283043384552,
+      "learning_rate": 3.7639198218262804e-05,
+      "loss": 0.4356,
+      "step": 780
+    },
+    {
+      "epoch": 1.6702788423417472,
+      "grad_norm": 0.4058088958263397,
+      "learning_rate": 3.541202672605791e-05,
+      "loss": 0.4461,
+      "step": 790
+    },
+    {
+      "epoch": 1.6914232853178275,
+      "grad_norm": 0.40314358472824097,
+      "learning_rate": 3.318485523385301e-05,
+      "loss": 0.4311,
+      "step": 800
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 946,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.815045548784222e+18,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

rats40k_adapter/outputs/pipeline_20260608_175250/sft/adapter/checkpoint-946/README.md

@@ -0,0 +1,202 @@
+---
+base_model: /mnt/share01/sqk/models/ChatTime-1-7B-Chat
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

rats40k_adapter/outputs/pipeline_20260608_175250/sft/adapter/checkpoint-946/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/mnt/share01/sqk/models/ChatTime-1-7B-Chat",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "o_proj",
+    "v_proj",
+    "up_proj",
+    "q_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

requirements.md

@@ -0,0 +1,201 @@
+# Hardware
+
+- CPU: `AMD Ryzen 9 7950X 16-Core Processor`
+- GPU: `NVIDIA GeForce RTX 4090 24GB`
+
+# System
+
+- Kernel: `Linux-5.15.0-102-generic-x86_64-with-glibc2.35`
+- Distribution: `Ubuntu 22.04.3 LTS`
+- Python: `3.12.11`
+
+# Requirements
+
+```
+accelerate==1.8.1
+aiohappyeyeballs==2.6.1
+aiohttp==3.12.13
+aiosignal==1.3.2
+annotated-types==0.7.0
+anyio==4.9.0
+argon2-cffi==25.1.0
+argon2-cffi-bindings==21.2.0
+arrow==1.3.0
+asttokens==3.0.0
+async-lru==2.0.5
+attrs==25.3.0
+babel==2.17.0
+beautifulsoup4==4.13.4
+bleach==6.2.0
+boto3==1.38.46
+botocore==1.38.46
+brotlicffi==1.0.9.2
+certifi==2025.6.15
+cffi==1.17.1
+charset-normalizer==3.1.0
+click==8.2.1
+comm==0.2.2
+contourpy==1.3.2
+cycler==0.12.1
+datasets==3.6.0
+debugpy==1.8.14
+decorator==5.2.1
+defusedxml==0.7.1
+dill==0.3.8
+distro==1.9.0
+einops==0.8.1
+et_xmlfile==2.0.0
+executing==2.2.0
+fastjsonschema==2.21.1
+filelock==3.13.1
+fonttools==4.58.4
+fqdn==1.5.1
+frozenlist==1.7.0
+fsspec==2024.6.1
+h11==0.16.0
+hf-xet==1.1.5
+holidays==0.75
+httpcore==1.0.9
+httpx==0.28.1
+huggingface-hub==0.33.1
+idna==3.10
+ipykernel==6.29.5
+ipython==9.3.0
+ipython_pygments_lexers==1.1.1
+ipywidgets==8.1.7
+isoduration==20.11.0
+jedi==0.19.2
+Jinja2==3.1.4
+jiter==0.10.0
+jmespath==1.0.1
+joblib==1.5.1
+json5==0.12.0
+jsonpointer==3.0.0
+jsonschema==4.24.0
+jsonschema-specifications==2025.4.1
+jupyter==1.1.1
+jupyter_client==8.6.3
+jupyter-console==6.6.3
+jupyter_core==5.8.1
+jupyter-events==0.12.0
+jupyter-lsp==2.2.5
+jupyter_server==2.16.0
+jupyter_server_terminals==0.5.3
+jupyterlab==4.4.4
+jupyterlab_pygments==0.3.0
+jupyterlab_server==2.27.3
+jupyterlab_widgets==3.0.15
+kiwisolver==1.4.8
+MarkupSafe==2.1.5
+matplotlib==3.9.0
+matplotlib-inline==0.1.7
+mistune==3.1.3
+mpmath==1.3.0
+multidict==6.6.2
+multiprocess==0.70.16
+nbclient==0.10.2
+nbconvert==7.16.6
+nbformat==5.10.4
+nest-asyncio==1.6.0
+networkx==3.3
+notebook==7.4.4
+notebook_shim==0.2.4
+numpy==2.1.2
+nvidia-cublas-cu11==11.11.3.6
+nvidia-cuda-cupti-cu11==11.8.87
+nvidia-cuda-nvrtc-cu11==11.8.89
+nvidia-cuda-runtime-cu11==11.8.89
+nvidia-cudnn-cu11==9.1.0.70
+nvidia-cufft-cu11==10.9.0.58
+nvidia-curand-cu11==10.3.0.86
+nvidia-cusolver-cu11==11.4.1.48
+nvidia-cusparse-cu11==11.7.5.86
+nvidia-ml-py==12.575.51
+nvidia-nccl-cu11==2.21.5
+nvidia-nvtx-cu11==11.8.86
+openai==1.93.0
+openpyxl==3.1.5
+overrides==7.7.0
+packaging==25.0
+pandas==2.3.0
+pandocfilters==1.5.1
+parso==0.8.4
+patsy==1.0.1
+peft==0.15.2
+pexpect==4.9.0
+pillow==11.0.0
+pip==25.1
+platformdirs==4.3.8
+prettytable==3.16.0
+prometheus_client==0.22.1
+prompt_toolkit==3.0.51
+propcache==0.3.2
+protobuf==6.31.1
+psutil==7.0.0
+ptyprocess==0.7.0
+pure_eval==0.2.3
+pyarrow==20.0.0
+pycparser==2.21
+pydantic==2.11.7
+pydantic_core==2.33.2
+pyecharts==2.0.8
+Pygments==2.19.2
+pynvml==12.0.0
+pyparsing==3.2.3
+PySocks==1.7.1
+python-dateutil==2.9.0.post0
+python-json-logger==3.3.0
+pytz==2025.2
+PyYAML==6.0.2
+pyzmq==27.0.0
+referencing==0.36.2
+regex==2024.11.6
+requests==2.32.4
+rfc3339-validator==0.1.4
+rfc3986-validator==0.1.1
+rpds-py==0.25.1
+s3transfer==0.13.0
+safetensors==0.5.3
+scikit-learn==1.7.0
+scipy==1.16.0
+seaborn==0.13.2
+Send2Trash==1.8.3
+sentencepiece==0.2.0
+setuptools==78.1.1
+simplejson==3.20.1
+six==1.17.0
+sniffio==1.3.1
+soupsieve==2.7
+stack-data==0.6.3
+statsmodels==0.14.4
+swankit==0.2.4
+swanlab==0.6.4
+sympy==1.13.3
+terminado==0.18.1
+threadpoolctl==3.6.0
+tinycss2==1.4.0
+tokenizers==0.21.2
+torch==2.7.1+cu118
+torchaudio==2.7.1+cu118
+torchvision==0.22.1+cu118
+tornado==6.5.1
+tqdm==4.67.1
+traitlets==5.14.3
+transformers==4.53.0
+triton==3.3.1
+types-python-dateutil==2.9.0.20250516
+typing_extensions==4.12.2
+typing-inspection==0.4.1
+tzdata==2025.2
+uri-template==1.3.0
+urllib3==2.5.0
+wcwidth==0.2.13
+webcolors==24.11.1
+webencodings==0.5.1
+websocket-client==1.8.0
+wheel==0.45.1
+widgetsnbextension==4.0.14
+wrapt==1.17.2
+xxhash==3.5.0
+yarl==1.20.1
+```

requirements.txt

@@ -0,0 +1,8 @@
+accelerate==1.8.1
+datasets==3.6.0
+huggingface-hub==0.33.1
+numpy==2.1.2
+pandas==2.3.0
+scikit-learn==1.7.0
+torch==2.7.1+cu118
+transformers==4.53.0

tsqa_adapter/README.md

@@ -0,0 +1,73 @@
+# TSQA Adapter for ChatTime
+
+This folder adapts ChatTime to the **Time-MQA TSQA** multi-task time-series QA
+benchmark (`/mnt/share01/sqk/datasets/Time-MQA_TSQA/tmp`). It is the TSQA analog
+of `rats40k_adapter` and is meant to fill in the **ChatTime TSQA baseline**.
+
+The dataset has four source groups with free-text answers:
+`anomaly_detection`, `classification`, `forecasting`, `open_ended` (the
+`open_ended` group mixes `multiple_choice`, `true_false` and open questions).
+
+## How ChatTime is fed
+
+ChatTime has no chat template, so prompts use ChatTime's native analysis
+template (`utils.prompt.getPrompt`):
+
+* `### Instruction` = the TSQA `question`, kept **verbatim** (including any
+  numbers embedded in the text, so forecasting keeps its absolute scale).
+* `### Input` = the `time_series` field discretised + serialised into ChatTime's
+  native time-series tokens (`###value###`).
+* `### Response` = the free-text `answer` (the SFT target).
+
+Some context-enhanced forecasting questions are much longer than ChatTime's
+4096-token context (e.g. an earnings-call transcript with a tiny series).
+Prompts are therefore **left-truncated** — keeping the serialized series, the
+trailing ask and (in training) the full response — instead of failing.
+
+## Metrics
+
+Evaluation reuses the **canonical Time-MQA evaluator**
+(`MQA/data_utils.compute_group_metrics`), so the numbers are directly comparable
+to the other TSQA baselines (MQA / ITFormer). Per group:
+
+* `anomaly_detection`: accuracy (normal vs anomaly label parsing)
+* `classification`: accuracy (class-label parsing)
+* `forecasting`: MSE / MAE over parsed forecast values
+* `open_ended`: accuracy (multiple-choice / true-false / semantic / numeric)
+
+`eval_tsqa.py` writes `predictions.jsonl` and `metrics.json` (with a `by_group`
+block) under the eval output directory.
+
+## Required inputs
+
+- `MODEL_PATH`: local ChatTime model dir. Default
+  `/mnt/share01/sqk/models/ChatTime-1-7B-Chat`.
+- `PYTHON_BIN`: Python executable. Defaults to
+  `/dev/shm/suiqk/conda_envs/scalerag-ts-v4/bin/python` (override if your env
+  lives elsewhere, e.g. `/home/suiqk/anaconda3/envs/scalerag-ts-v4/bin/python`).
+- `MQA_DIR`: defaults to `/mnt/share01/sqk/MQA` (provides the shared evaluator).
+- `DATA_ROOT`: defaults to `/mnt/share01/sqk/datasets/Time-MQA_TSQA/tmp`.
+
+The defaults target the **4× V100** machine: FP16 LoRA with `LOAD_IN_4BIT=0`,
+`PER_DEVICE_TRAIN_BATCH_SIZE=1`, `GRADIENT_ACCUMULATION_STEPS=16`, the 4-GPU
+accelerate config at `/mnt/share01/sqk/ITFormer/accelerate_config.yaml`.
+Generation defaults: `MAX_NEW_TOKENS=256`, `MAX_INPUT_TOKENS=3840`,
+`MAX_SEQ_LENGTH=4096`.
+
+## SFT + Eval
+
+```bash
+cd /mnt/share01/sqk/ChatTime
+MODEL_PATH=/mnt/share01/sqk/models/ChatTime-1-7B-Chat \
+bash tsqa_adapter/run_sft_4gpu.sh
+```
+
+## Smoke test
+
+```bash
+cd /mnt/share01/sqk/ChatTime
+MAX_TRAIN_SAMPLES=128 MAX_EVAL_SAMPLES=64 bash tsqa_adapter/run_sft_4gpu.sh
+```
+
+Outputs land in `tsqa_adapter/outputs/sft_<RUN_ID>/{adapter,eval}` and logs in
+`tsqa_adapter/logs/`.

tsqa_adapter/eval_tsqa.py

@@ -0,0 +1,228 @@
+"""Evaluate ChatTime on the Time-MQA TSQA benchmark.
+
+Adapted from ``rats40k_adapter/eval_rats40k.py``. Generation is sharded across
+ranks; each rank writes a JSONL shard, then rank 0 merges, computes per-group
+metrics with the canonical Time-MQA evaluator and writes ``predictions.jsonl``
+plus ``metrics.json``.
+
+Over-long prompts are LEFT-truncated to ``--max_input_tokens`` so the series and
+the trailing ask survive (ChatTime's context is only 4096 tokens while some
+context-enhanced forecasting questions are much longer).
+"""
+
+import argparse
+import copy
+import json
+import os
+from pathlib import Path
+
+import torch
+import torch.distributed as dist
+from tqdm import tqdm
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+from tsqa_common import (
+    atomic_write_json,
+    balanced_limit,
+    build_prompt,
+    build_result_row,
+    compute_tsqa_metrics,
+    load_tsqa_records,
+    resolve_data_file,
+    write_jsonl,
+)
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(description="Evaluate ChatTime on Time-MQA TSQA.")
+    parser.add_argument("--model_path", required=True)
+    parser.add_argument("--adapter_path", default=None)
+    parser.add_argument("--data_root", default="/mnt/share01/sqk/datasets/Time-MQA_TSQA/tmp")
+    parser.add_argument("--eval_file", default="eval.jsonl")
+    parser.add_argument("--output_dir", required=True)
+    parser.add_argument("--max_eval_samples", type=int, default=None)
+    parser.add_argument("--eval_batch_size", type=int, default=4)
+    parser.add_argument("--max_input_tokens", type=int, default=3840)
+    parser.add_argument("--max_new_tokens", type=int, default=256)
+    parser.add_argument("--temperature", type=float, default=0.0)
+    parser.add_argument("--top_p", type=float, default=1.0)
+    parser.add_argument("--top_k", type=int, default=50)
+    parser.add_argument("--repetition_penalty", type=float, default=1.0)
+    parser.add_argument("--torch_dtype", choices=["auto", "bf16", "fp16", "fp32"], default="fp16")
+    parser.add_argument("--allow_hf_download", action="store_true")
+    return parser.parse_args()
+
+
+def init_distributed():
+    world_size = int(os.environ.get("WORLD_SIZE", "1"))
+    rank = int(os.environ.get("RANK", "0"))
+    local_rank = int(os.environ.get("LOCAL_RANK", "0"))
+    if torch.cuda.is_available():
+        torch.cuda.set_device(local_rank)
+    if world_size > 1 and not dist.is_initialized():
+        dist.init_process_group(backend="nccl")
+    return rank, local_rank, world_size
+
+
+def dtype_from_arg(value):
+    if value == "auto":
+        return "auto"
+    if value == "bf16":
+        return torch.bfloat16
+    if value == "fp16":
+        return torch.float16
+    return torch.float32
+
+
+def load_model_and_tokenizer(args, local_rank):
+    local_files_only = not args.allow_hf_download
+    tokenizer = AutoTokenizer.from_pretrained(
+        args.model_path,
+        trust_remote_code=True,
+        local_files_only=local_files_only,
+    )
+    if tokenizer.pad_token is None:
+        tokenizer.pad_token = tokenizer.eos_token
+    tokenizer.padding_side = "left"
+    # Keep the series and the trailing ask when a prompt is too long.
+    tokenizer.truncation_side = "left"
+
+    device = f"cuda:{local_rank}" if torch.cuda.is_available() else "cpu"
+    model = AutoModelForCausalLM.from_pretrained(
+        args.model_path,
+        trust_remote_code=True,
+        torch_dtype=dtype_from_arg(args.torch_dtype),
+        low_cpu_mem_usage=True,
+        device_map={"": device} if torch.cuda.is_available() else None,
+        local_files_only=local_files_only,
+    )
+
+    if args.adapter_path:
+        from peft import PeftModel
+
+        model = PeftModel.from_pretrained(
+            model,
+            args.adapter_path,
+            local_files_only=local_files_only,
+        )
+
+    model.eval()
+    return model, tokenizer, device
+
+
+def generate_responses(model, tokenizer, device, prompts, args):
+    model_context = getattr(model.config, "max_position_embeddings", None)
+    max_input_tokens = args.max_input_tokens
+    if model_context and max_input_tokens + args.max_new_tokens > model_context:
+        max_input_tokens = max(1, model_context - args.max_new_tokens)
+
+    inputs = tokenizer(
+        prompts,
+        return_tensors="pt",
+        padding=True,
+        truncation=True,
+        max_length=max_input_tokens,
+    )
+    inputs = {key: value.to(device) for key, value in inputs.items()}
+
+    do_sample = args.temperature > 0
+    generation_config = copy.deepcopy(model.generation_config)
+    generation_config.do_sample = do_sample
+    generation_config.pad_token_id = tokenizer.pad_token_id
+    generation_config.eos_token_id = tokenizer.eos_token_id
+    if do_sample:
+        generation_config.temperature = args.temperature
+        generation_config.top_p = args.top_p
+        generation_config.top_k = args.top_k
+    else:
+        generation_config.temperature = None
+        generation_config.top_p = None
+        generation_config.top_k = None
+
+    with torch.inference_mode():
+        output = model.generate(
+            **inputs,
+            generation_config=generation_config,
+            max_new_tokens=args.max_new_tokens,
+            repetition_penalty=args.repetition_penalty,
+        )
+    new_tokens = output[:, inputs["input_ids"].shape[-1]:]
+    return [
+        response.strip()
+        for response in tokenizer.batch_decode(new_tokens, skip_special_tokens=True)
+    ]
+
+
+def main():
+    args = parse_args()
+    rank, local_rank, world_size = init_distributed()
+    output_dir = Path(args.output_dir)
+    shard_dir = output_dir / "shards"
+    shard_dir.mkdir(parents=True, exist_ok=True)
+
+    data_path = resolve_data_file(args.data_root, args.eval_file)
+    records = load_tsqa_records(data_path)
+    records = balanced_limit(records, args.max_eval_samples)
+    shard_records = records[rank::world_size]
+
+    if rank == 0:
+        print(f"Dataset: {data_path}")
+        print(f"Total samples: {len(records)}")
+        print(f"World size: {world_size}")
+        print(f"Per-device eval batch size: {args.eval_batch_size}")
+        print(f"Maximum global eval batch size: {args.eval_batch_size * world_size}")
+        print(f"Output dir: {output_dir}")
+
+    model, tokenizer, device = load_model_and_tokenizer(args, local_rank)
+
+    results = []
+    batch_size = max(1, args.eval_batch_size)
+    batch_starts = range(0, len(shard_records), batch_size)
+    for start in tqdm(
+        batch_starts,
+        total=(len(shard_records) + batch_size - 1) // batch_size,
+        desc=f"rank {rank}",
+        disable=rank != 0,
+    ):
+        batch = shard_records[start:start + batch_size]
+        prompts = [
+            build_prompt(record["question"], record.get("time_series") or [])
+            for record in batch
+        ]
+        responses = generate_responses(model, tokenizer, device, prompts, args)
+        for record, response in zip(batch, responses):
+            results.append(build_result_row(record, response))
+
+    shard_path = shard_dir / f"predictions.rank{rank}.jsonl"
+    write_jsonl(results, shard_path)
+
+    if world_size > 1:
+        dist.barrier()
+
+    if rank == 0:
+        merged = []
+        for shard_rank in range(world_size):
+            path = shard_dir / f"predictions.rank{shard_rank}.jsonl"
+            with open(path, "r", encoding="utf-8") as handle:
+                for line in handle:
+                    line = line.strip()
+                    if line:
+                        merged.append(json.loads(line))
+        merged.sort(key=lambda row: str(row.get("id")))
+
+        predictions_path = output_dir / "predictions.jsonl"
+        metrics_path = output_dir / "metrics.json"
+        write_jsonl(merged, predictions_path)
+        metrics = compute_tsqa_metrics(merged)
+        atomic_write_json(metrics, metrics_path)
+        print(json.dumps(metrics, ensure_ascii=False, indent=2))
+        print(f"Saved predictions: {predictions_path}")
+        print(f"Saved metrics: {metrics_path}")
+
+    if world_size > 1:
+        dist.barrier()
+        dist.destroy_process_group()
+
+
+if __name__ == "__main__":
+    main()

tsqa_adapter/finetune_tsqa_lora.py

@@ -0,0 +1,351 @@
+"""LoRA SFT for ChatTime on the Time-MQA TSQA multi-task QA benchmark.
+
+Adapted from ``rats40k_adapter/finetune_rats40k_lora.py``. The main differences:
+
+* Data comes from TSQA JSONL (``train.jsonl``) with free-text answers.
+* Prompts use ChatTime's native analysis template with the question in
+  ``### Instruction`` and the serialised series in ``### Input``.
+* Over-long rows (prompt+response beyond ``max_seq_length``) are SKIPPED by
+  default (``--skip_overlong``) so they never reach the GPU -- this prevents OOM
+  and avoids training on truncated junk, since some context-enhanced forecasting
+  questions exceed ChatTime's 4096-token context. Pass ``--no_skip_overlong`` to
+  fall back to left-truncation (keeping the series, the trailing ask and the full
+  response) instead.
+"""
+
+import argparse
+import inspect
+import os
+from pathlib import Path
+
+import torch
+from torch.utils.data import Dataset
+from transformers import AutoModelForCausalLM, AutoTokenizer, Trainer, TrainingArguments
+
+from tsqa_common import (
+    balanced_limit,
+    build_prompt,
+    load_tsqa_records,
+    resolve_data_file,
+)
+
+
+def patch_accelerate_compatibility():
+    from accelerate import Accelerator
+
+    signature = inspect.signature(Accelerator.unwrap_model)
+    if "keep_torch_compile" in signature.parameters:
+        return
+
+    original_unwrap_model = Accelerator.unwrap_model
+
+    def unwrap_model_compat(self, model, keep_fp32_wrapper=True, keep_torch_compile=True):
+        del keep_torch_compile
+        return original_unwrap_model(self, model, keep_fp32_wrapper=keep_fp32_wrapper)
+
+    Accelerator.unwrap_model = unwrap_model_compat
+    print(
+        "Applied accelerate compatibility patch: "
+        "Accelerator.unwrap_model accepts keep_torch_compile."
+    )
+
+
+class PromptResponseDataset(Dataset):
+    def __init__(self, rows, tokenizer, max_seq_length, skip_overlong=True):
+        self.tokenizer = tokenizer
+        self.max_seq_length = max_seq_length
+        self.skip_overlong = skip_overlong
+        # Filtering is deterministic (same rows, tokenizer and threshold on every
+        # rank), so all DDP ranks agree on the kept set and dataset length.
+        self.rows = self._prepare_rows(rows)
+
+    def __len__(self):
+        return len(self.rows)
+
+    def _raw_encode(self, prompt, response):
+        prompt_ids = self.tokenizer(
+            prompt,
+            add_special_tokens=True,
+            truncation=False,
+        )["input_ids"]
+        response_ids = self.tokenizer(
+            response + self.tokenizer.eos_token,
+            add_special_tokens=False,
+            truncation=False,
+        )["input_ids"]
+        return prompt_ids, response_ids
+
+    def _fit(self, prompt_ids, response_ids):
+        # Always keep the full response; left-truncate the prompt to fit so the
+        # series, the trailing ask and the ### Response marker survive.
+        max_prompt = self.max_seq_length - len(response_ids)
+        truncated = False
+        if max_prompt < 1:
+            # Pathological: response alone exceeds the budget. Keep the response
+            # tail and a single prompt token so the example stays well-formed.
+            response_ids = response_ids[-(self.max_seq_length - 1):]
+            prompt_ids = prompt_ids[-1:]
+            truncated = True
+        elif len(prompt_ids) > max_prompt:
+            prompt_ids = prompt_ids[-max_prompt:]
+            truncated = True
+        return prompt_ids, response_ids, truncated
+
+    def _prepare_rows(self, rows):
+        kept = []
+        skipped = 0
+        truncated = 0
+        max_prompt_len = 0
+        max_total_len = 0
+        for row in rows:
+            _, prompt, response = row
+            prompt_ids, response_ids = self._raw_encode(prompt, response)
+            total = len(prompt_ids) + len(response_ids)
+            if total > self.max_seq_length:
+                if self.skip_overlong:
+                    # Drop over-long examples outright so they never reach the
+                    # GPU (prevents OOM and avoids training on truncated junk).
+                    skipped += 1
+                    continue
+                prompt_ids, response_ids, _ = self._fit(prompt_ids, response_ids)
+                total = len(prompt_ids) + len(response_ids)
+                truncated += 1
+            max_prompt_len = max(max_prompt_len, len(prompt_ids))
+            max_total_len = max(max_total_len, total)
+            kept.append(row)
+        print(
+            "SFT token length check: "
+            f"input_rows={len(rows)}, "
+            f"kept_rows={len(kept)}, "
+            f"skipped_overlong={skipped}, "
+            f"left_truncated_rows={truncated}, "
+            f"max_prompt_len={max_prompt_len}, "
+            f"max_total_len={max_total_len}, "
+            f"max_seq_length={self.max_seq_length}, "
+            f"skip_overlong={self.skip_overlong}"
+        )
+        return kept
+
+    def __getitem__(self, index):
+        _, prompt, response = self.rows[index]
+        prompt_ids, response_ids = self._raw_encode(prompt, response)
+        # Safety net: kept rows already fit when skip_overlong is on, but clamp
+        # anyway so a stray over-long row can never blow up the batch.
+        prompt_ids, response_ids, _ = self._fit(prompt_ids, response_ids)
+        input_ids = prompt_ids + response_ids
+        labels = [-100] * len(prompt_ids) + response_ids
+        attention_mask = [1] * len(input_ids)
+        return {
+            "input_ids": input_ids,
+            "attention_mask": attention_mask,
+            "labels": labels,
+        }
+
+
+class CausalLMCollator:
+    def __init__(self, tokenizer):
+        self.tokenizer = tokenizer
+
+    def __call__(self, features):
+        max_len = max(len(feature["input_ids"]) for feature in features)
+        input_ids = []
+        attention_mask = []
+        labels = []
+        for feature in features:
+            pad_len = max_len - len(feature["input_ids"])
+            input_ids.append(feature["input_ids"] + [self.tokenizer.pad_token_id] * pad_len)
+            attention_mask.append(feature["attention_mask"] + [0] * pad_len)
+            labels.append(feature["labels"] + [-100] * pad_len)
+        return {
+            "input_ids": torch.tensor(input_ids, dtype=torch.long),
+            "attention_mask": torch.tensor(attention_mask, dtype=torch.long),
+            "labels": torch.tensor(labels, dtype=torch.long),
+        }
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(
+        description="LoRA SFT for ChatTime on Time-MQA TSQA multi-task QA."
+    )
+    parser.add_argument("--model_path", required=True)
+    parser.add_argument("--data_root", default="/mnt/share01/sqk/datasets/Time-MQA_TSQA/tmp")
+    parser.add_argument("--train_file", default="train.jsonl")
+    parser.add_argument("--output_dir", required=True)
+    parser.add_argument("--max_train_samples", type=int, default=None)
+    parser.add_argument("--max_seq_length", type=int, default=4096)
+    parser.add_argument(
+        "--skip_overlong",
+        dest="skip_overlong",
+        action="store_true",
+        default=True,
+        help="Skip training rows whose prompt+response exceeds max_seq_length (default).",
+    )
+    parser.add_argument(
+        "--no_skip_overlong",
+        dest="skip_overlong",
+        action="store_false",
+        help="Left-truncate over-long rows instead of skipping them.",
+    )
+    parser.add_argument("--per_device_train_batch_size", type=int, default=2)
+    parser.add_argument("--gradient_accumulation_steps", type=int, default=8)
+    parser.add_argument("--num_train_epochs", type=float, default=2.0)
+    parser.add_argument("--learning_rate", type=float, default=2e-4)
+    parser.add_argument("--warmup_ratio", type=float, default=0.05)
+    parser.add_argument("--weight_decay", type=float, default=0.01)
+    parser.add_argument("--logging_steps", type=int, default=10)
+    parser.add_argument("--save_steps", type=int, default=200)
+    parser.add_argument("--save_total_limit", type=int, default=2)
+    parser.add_argument("--dataloader_num_workers", type=int, default=4)
+    parser.add_argument("--lora_rank", type=int, default=16)
+    parser.add_argument("--lora_alpha", type=int, default=32)
+    parser.add_argument("--lora_dropout", type=float, default=0.05)
+    parser.add_argument("--load_in_4bit", action="store_true")
+    parser.add_argument("--gradient_checkpointing", action="store_true")
+    parser.add_argument("--torch_dtype", choices=["bf16", "fp16", "fp32"], default="fp16")
+    parser.add_argument("--allow_hf_download", action="store_true")
+    return parser.parse_args()
+
+
+def dtype_from_arg(value):
+    if value == "bf16":
+        return torch.bfloat16
+    if value == "fp16":
+        return torch.float16
+    return torch.float32
+
+
+def local_rank():
+    return int(os.environ.get("LOCAL_RANK", "0"))
+
+
+def build_rows(data_root, train_file, max_samples):
+    path = resolve_data_file(data_root, train_file)
+    records = load_tsqa_records(path)
+    records = balanced_limit(records, max_samples)
+    rows = []
+    for index, record in enumerate(records):
+        prompt = build_prompt(record["question"], record.get("time_series") or [])
+        response = str(record["answer"]).strip()
+        row_id = str(record.get("id") or index)
+        rows.append((row_id, prompt, response))
+    return rows
+
+
+def load_model(args):
+    try:
+        from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training
+    except Exception as exc:
+        raise RuntimeError(
+            "peft is required for SFT. Install peft in the selected Python environment."
+        ) from exc
+
+    dtype = dtype_from_arg(args.torch_dtype)
+    local_files_only = not args.allow_hf_download
+    device = f"cuda:{local_rank()}" if torch.cuda.is_available() else "cpu"
+
+    quantization_config = None
+    if args.load_in_4bit:
+        from transformers import BitsAndBytesConfig
+
+        quantization_config = BitsAndBytesConfig(
+            load_in_4bit=True,
+            bnb_4bit_compute_dtype=dtype,
+            bnb_4bit_quant_type="nf4",
+            bnb_4bit_use_double_quant=True,
+        )
+
+    model = AutoModelForCausalLM.from_pretrained(
+        args.model_path,
+        trust_remote_code=True,
+        torch_dtype=dtype,
+        low_cpu_mem_usage=True,
+        quantization_config=quantization_config,
+        device_map={"": device} if torch.cuda.is_available() and args.load_in_4bit else None,
+        local_files_only=local_files_only,
+    )
+    if not args.load_in_4bit and torch.cuda.is_available():
+        model.to(device)
+    if args.load_in_4bit:
+        model = prepare_model_for_kbit_training(
+            model,
+            use_gradient_checkpointing=args.gradient_checkpointing,
+        )
+    elif args.gradient_checkpointing:
+        model.gradient_checkpointing_enable()
+
+    lora_config = LoraConfig(
+        r=args.lora_rank,
+        lora_alpha=args.lora_alpha,
+        lora_dropout=args.lora_dropout,
+        bias="none",
+        task_type="CAUSAL_LM",
+        target_modules=[
+            "q_proj",
+            "k_proj",
+            "v_proj",
+            "o_proj",
+            "gate_proj",
+            "up_proj",
+            "down_proj",
+        ],
+    )
+    model = get_peft_model(model, lora_config)
+    model.print_trainable_parameters()
+    return model
+
+
+def main():
+    args = parse_args()
+    patch_accelerate_compatibility()
+    if torch.cuda.is_available():
+        torch.cuda.set_device(local_rank())
+
+    local_files_only = not args.allow_hf_download
+    tokenizer = AutoTokenizer.from_pretrained(
+        args.model_path,
+        trust_remote_code=True,
+        local_files_only=local_files_only,
+    )
+    if tokenizer.pad_token is None:
+        tokenizer.pad_token = tokenizer.eos_token
+    tokenizer.padding_side = "right"
+
+    rows = build_rows(args.data_root, args.train_file, args.max_train_samples)
+    train_dataset = PromptResponseDataset(
+        rows, tokenizer, args.max_seq_length, skip_overlong=args.skip_overlong
+    )
+    model = load_model(args)
+
+    bf16 = args.torch_dtype == "bf16"
+    fp16 = args.torch_dtype == "fp16"
+    training_args = TrainingArguments(
+        output_dir=args.output_dir,
+        per_device_train_batch_size=args.per_device_train_batch_size,
+        gradient_accumulation_steps=args.gradient_accumulation_steps,
+        num_train_epochs=args.num_train_epochs,
+        learning_rate=args.learning_rate,
+        warmup_ratio=args.warmup_ratio,
+        weight_decay=args.weight_decay,
+        logging_steps=args.logging_steps,
+        save_steps=args.save_steps,
+        save_total_limit=args.save_total_limit,
+        dataloader_num_workers=args.dataloader_num_workers,
+        bf16=bf16,
+        fp16=fp16,
+        report_to="none",
+        remove_unused_columns=False,
+        ddp_find_unused_parameters=False,
+    )
+    trainer = Trainer(
+        model=model,
+        args=training_args,
+        train_dataset=train_dataset,
+        data_collator=CausalLMCollator(tokenizer),
+    )
+    trainer.train()
+    trainer.save_model(args.output_dir)
+    tokenizer.save_pretrained(args.output_dir)
+
+
+if __name__ == "__main__":
+    main()

tsqa_adapter/outputs/sft_20260615_141604/adapter/README.md

@@ -0,0 +1,202 @@
+---
+base_model: /mnt/share01/sqk/models/ChatTime-1-7B-Chat
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

tsqa_adapter/outputs/sft_20260615_141604/adapter/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/mnt/share01/sqk/models/ChatTime-1-7B-Chat",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "o_proj",
+    "down_proj",
+    "v_proj",
+    "k_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

tsqa_adapter/outputs/sft_20260615_141604/adapter/checkpoint-250/README.md

@@ -0,0 +1,202 @@
+---
+base_model: /mnt/share01/sqk/models/ChatTime-1-7B-Chat
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

tsqa_adapter/outputs/sft_20260615_141604/adapter/checkpoint-250/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "/mnt/share01/sqk/models/ChatTime-1-7B-Chat",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "o_proj",
+    "down_proj",
+    "v_proj",
+    "k_proj",
+    "up_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

tsqa_adapter/outputs/sft_20260615_141604/adapter/checkpoint-250/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tsqa_adapter/outputs/sft_20260615_141604/adapter/checkpoint-250/trainer_state.json

@@ -0,0 +1,209 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.0,
+  "eval_steps": 500,
+  "global_step": 250,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.08012018027040561,
+      "grad_norm": 0.9694206714630127,
+      "learning_rate": 0.0001230769230769231,
+      "loss": 2.1733,
+      "step": 10
+    },
+    {
+      "epoch": 0.16024036054081123,
+      "grad_norm": 0.4126811921596527,
+      "learning_rate": 0.0001957805907172996,
+      "loss": 0.9926,
+      "step": 20
+    },
+    {
+      "epoch": 0.24036054081121683,
+      "grad_norm": 0.3288513422012329,
+      "learning_rate": 0.00018734177215189873,
+      "loss": 0.8508,
+      "step": 30
+    },
+    {
+      "epoch": 0.32048072108162245,
+      "grad_norm": 0.33170658349990845,
+      "learning_rate": 0.0001789029535864979,
+      "loss": 0.7701,
+      "step": 40
+    },
+    {
+      "epoch": 0.400600901352028,
+      "grad_norm": 0.2883685231208801,
+      "learning_rate": 0.00017046413502109706,
+      "loss": 0.7776,
+      "step": 50
+    },
+    {
+      "epoch": 0.48072108162243365,
+      "grad_norm": 0.22634504735469818,
+      "learning_rate": 0.0001620253164556962,
+      "loss": 0.7121,
+      "step": 60
+    },
+    {
+      "epoch": 0.5608412618928392,
+      "grad_norm": 0.22475452721118927,
+      "learning_rate": 0.00015358649789029536,
+      "loss": 0.7793,
+      "step": 70
+    },
+    {
+      "epoch": 0.6409614421632449,
+      "grad_norm": 0.3640509843826294,
+      "learning_rate": 0.00014514767932489453,
+      "loss": 0.757,
+      "step": 80
+    },
+    {
+      "epoch": 0.7210816224336505,
+      "grad_norm": 0.2446712851524353,
+      "learning_rate": 0.00013670886075949366,
+      "loss": 0.7813,
+      "step": 90
+    },
+    {
+      "epoch": 0.801201802704056,
+      "grad_norm": 0.21602191030979156,
+      "learning_rate": 0.00012827004219409285,
+      "loss": 0.7259,
+      "step": 100
+    },
+    {
+      "epoch": 0.8813219829744617,
+      "grad_norm": 0.2295820116996765,
+      "learning_rate": 0.00011983122362869199,
+      "loss": 0.7549,
+      "step": 110
+    },
+    {
+      "epoch": 0.9614421632448673,
+      "grad_norm": 0.2932703495025635,
+      "learning_rate": 0.00011139240506329114,
+      "loss": 0.7469,
+      "step": 120
+    },
+    {
+      "epoch": 1.0400600901352028,
+      "grad_norm": 0.24934373795986176,
+      "learning_rate": 0.00010295358649789031,
+      "loss": 0.7391,
+      "step": 130
+    },
+    {
+      "epoch": 1.1201802704056085,
+      "grad_norm": 0.19357512891292572,
+      "learning_rate": 9.451476793248946e-05,
+      "loss": 0.6932,
+      "step": 140
+    },
+    {
+      "epoch": 1.200300450676014,
+      "grad_norm": 0.20672819018363953,
+      "learning_rate": 8.607594936708861e-05,
+      "loss": 0.707,
+      "step": 150
+    },
+    {
+      "epoch": 1.2804206309464197,
+      "grad_norm": 0.40644571185112,
+      "learning_rate": 7.763713080168776e-05,
+      "loss": 0.6962,
+      "step": 160
+    },
+    {
+      "epoch": 1.3605408112168251,
+      "grad_norm": 0.2238582968711853,
+      "learning_rate": 6.919831223628693e-05,
+      "loss": 0.6914,
+      "step": 170
+    },
+    {
+      "epoch": 1.4406609914872308,
+      "grad_norm": 0.24433942139148712,
+      "learning_rate": 6.0759493670886084e-05,
+      "loss": 0.7065,
+      "step": 180
+    },
+    {
+      "epoch": 1.5207811717576365,
+      "grad_norm": 0.29082414507865906,
+      "learning_rate": 5.232067510548524e-05,
+      "loss": 0.708,
+      "step": 190
+    },
+    {
+      "epoch": 1.6009013520280422,
+      "grad_norm": 0.3576412796974182,
+      "learning_rate": 4.388185654008439e-05,
+      "loss": 0.6934,
+      "step": 200
+    },
+    {
+      "epoch": 1.6810215322984476,
+      "grad_norm": 0.2905654013156891,
+      "learning_rate": 3.5443037974683544e-05,
+      "loss": 0.6848,
+      "step": 210
+    },
+    {
+      "epoch": 1.7611417125688533,
+      "grad_norm": 0.2700088620185852,
+      "learning_rate": 2.7004219409282698e-05,
+      "loss": 0.6854,
+      "step": 220
+    },
+    {
+      "epoch": 1.8412618928392588,
+      "grad_norm": 0.220729798078537,
+      "learning_rate": 1.856540084388186e-05,
+      "loss": 0.7243,
+      "step": 230
+    },
+    {
+      "epoch": 1.9213820731096645,
+      "grad_norm": 0.23472554981708527,
+      "learning_rate": 1.0126582278481012e-05,
+      "loss": 0.7209,
+      "step": 240
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.2952578067779541,
+      "learning_rate": 1.6877637130801686e-06,
+      "loss": 0.7225,
+      "step": 250
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 250,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 4.489394280737014e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

tsqa_adapter/outputs/sft_20260615_141604/adapter/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tsqa_adapter/outputs/sft_20260615_141604/eval/metrics.json

@@ -0,0 +1,118 @@
+{
+  "by_group": {
+    "anomaly_detection": {
+      "count": 200,
+      "accuracy": 0.82,
+      "correct": 164,
+      "parsed": 200
+    },
+    "classification": {
+      "count": 200,
+      "accuracy": 0.74,
+      "correct": 148,
+      "parsed": 200
+    },
+    "forecasting": {
+      "count": 200,
+      "valid_samples": 170,
+      "valid_points": 3164,
+      "mse": 825899.547396054,
+      "mae": 151.69107746555417
+    },
+    "open_ended": {
+      "count": 200,
+      "accuracy": 0.45,
+      "parsed_accuracy": 0.45685279187817257,
+      "parse_rate": 0.985,
+      "correct": 90,
+      "parsed": 197,
+      "unparsed": 3,
+      "by_format": {
+        "multiple_choice": {
+          "count": 67,
+          "accuracy": 0.2835820895522388,
+          "parsed_accuracy": 0.2878787878787879,
+          "correct": 19,
+          "parsed": 66,
+          "unparsed": 1
+        },
+        "open_ended_question": {
+          "count": 67,
+          "accuracy": 0.417910447761194,
+          "parsed_accuracy": 0.4307692307692308,
+          "correct": 28,
+          "parsed": 65,
+          "unparsed": 2
+        },
+        "true_false": {
+          "count": 66,
+          "accuracy": 0.6515151515151515,
+          "parsed_accuracy": 0.6515151515151515,
+          "correct": 43,
+          "parsed": 66,
+          "unparsed": 0
+        }
+      },
+      "by_method": {
+        "anomaly": {
+          "count": 1,
+          "accuracy": 1.0,
+          "correct": 1
+        },
+        "cyclical": {
+          "count": 4,
+          "accuracy": 0.75,
+          "correct": 3
+        },
+        "multiple_choice": {
+          "count": 64,
+          "accuracy": 0.296875,
+          "correct": 19
+        },
+        "numeric_scalar": {
+          "count": 31,
+          "accuracy": 0.25806451612903225,
+          "correct": 8
+        },
+        "numeric_sequence": {
+          "count": 1,
+          "accuracy": 0.0,
+          "correct": 0
+        },
+        "seasonality": {
+          "count": 3,
+          "accuracy": 1.0,
+          "correct": 3
+        },
+        "trend": {
+          "count": 20,
+          "accuracy": 0.45,
+          "correct": 9
+        },
+        "true_false": {
+          "count": 65,
+          "accuracy": 0.6461538461538462,
+          "correct": 42
+        },
+        "volatility": {
+          "count": 8,
+          "accuracy": 0.625,
+          "correct": 5
+        }
+      }
+    }
+  },
+  "text_overall": {
+    "count": 800,
+    "exact_match": 0.39,
+    "normalized_exact_match": 0.39,
+    "token_f1": 0.6366794082162608
+  },
+  "num_samples": 800,
+  "counts_by_group": {
+    "anomaly_detection": 200,
+    "classification": 200,
+    "forecasting": 200,
+    "open_ended": 200
+  }
+}

tsqa_adapter/run_sft_4gpu.sh

@@ -0,0 +1,163 @@
+#!/bin/bash
+set -Eeuo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
+cd "$PROJECT_DIR"
+
+RUN_ID="${RUN_ID:-$(date +%Y%m%d_%H%M%S)}"
+PYTHON_BIN="${PYTHON_BIN:-/dev/shm/suiqk/conda_envs/scalerag-ts-v4/bin/python}"
+ACCELERATE_CONFIG="${ACCELERATE_CONFIG:-/mnt/share01/sqk/ITFormer/accelerate_config.yaml}"
+CUDA_VISIBLE_DEVICES="${CUDA_VISIBLE_DEVICES:-0,1,2,3}"
+MODEL_PATH="${MODEL_PATH:-/mnt/share01/sqk/models/ChatTime-1-7B-Chat}"
+ALLOW_HF_DOWNLOAD="${ALLOW_HF_DOWNLOAD:-0}"
+MQA_DIR="${MQA_DIR:-/mnt/share01/sqk/MQA}"
+DATA_ROOT="${DATA_ROOT:-/mnt/share01/sqk/datasets/Time-MQA_TSQA/tmp}"
+TRAIN_FILE="${TRAIN_FILE:-train.jsonl}"
+EVAL_FILE="${EVAL_FILE:-eval.jsonl}"
+OUTPUT_ROOT="${OUTPUT_ROOT:-${PROJECT_DIR}/tsqa_adapter/outputs/sft_${RUN_ID}}"
+ADAPTER_OUTPUT_DIR="${ADAPTER_OUTPUT_DIR:-${OUTPUT_ROOT}/adapter}"
+EVAL_OUTPUT_DIR="${EVAL_OUTPUT_DIR:-${OUTPUT_ROOT}/eval}"
+RUN_EVAL_AFTER_SFT="${RUN_EVAL_AFTER_SFT:-1}"
+MAX_TRAIN_SAMPLES="${MAX_TRAIN_SAMPLES:-}"
+MAX_EVAL_SAMPLES="${MAX_EVAL_SAMPLES:-}"
+EVAL_BATCH_SIZE="${EVAL_BATCH_SIZE:-4}"
+MAX_SEQ_LENGTH="${MAX_SEQ_LENGTH:-4096}"
+SKIP_OVERLONG="${SKIP_OVERLONG:-1}"
+PER_DEVICE_TRAIN_BATCH_SIZE="${PER_DEVICE_TRAIN_BATCH_SIZE:-1}"
+GRADIENT_ACCUMULATION_STEPS="${GRADIENT_ACCUMULATION_STEPS:-16}"
+NUM_TRAIN_EPOCHS="${NUM_TRAIN_EPOCHS:-2}"
+LEARNING_RATE="${LEARNING_RATE:-2e-4}"
+LORA_RANK="${LORA_RANK:-16}"
+LORA_ALPHA="${LORA_ALPHA:-32}"
+LORA_DROPOUT="${LORA_DROPOUT:-0.05}"
+SAVE_STEPS="${SAVE_STEPS:-200}"
+LOGGING_STEPS="${LOGGING_STEPS:-10}"
+SAVE_TOTAL_LIMIT="${SAVE_TOTAL_LIMIT:-2}"
+DATALOADER_NUM_WORKERS="${DATALOADER_NUM_WORKERS:-4}"
+LOAD_IN_4BIT="${LOAD_IN_4BIT:-0}"
+GRADIENT_CHECKPOINTING="${GRADIENT_CHECKPOINTING:-1}"
+TORCH_DTYPE="${TORCH_DTYPE:-fp16}"
+MAX_NEW_TOKENS="${MAX_NEW_TOKENS:-256}"
+MAX_INPUT_TOKENS="${MAX_INPUT_TOKENS:-3840}"
+LOG_DIR="${LOG_DIR:-${PROJECT_DIR}/tsqa_adapter/logs}"
+LOG_FILE="${LOG_FILE:-${LOG_DIR}/sft_4gpu_${RUN_ID}.log}"
+
+mkdir -p "$LOG_DIR" "$OUTPUT_ROOT"
+
+fail() {
+    echo "$*" >&2
+    exit 1
+}
+
+exec > >(tee -a "$LOG_FILE") 2>&1
+
+export CUDA_VISIBLE_DEVICES
+export PYTHONPATH="${PROJECT_DIR}:${MQA_DIR}:${PYTHONPATH:-}"
+export MQA_DIR
+export TOKENIZERS_PARALLELISM=false
+export PYTHONWARNINGS="ignore::FutureWarning:transformers.utils.hub"
+export WANDB_MODE=offline
+
+# Force a guaranteed-local, writable temp dir. The repo (cwd) sits on a
+# shared/NFS mount; if no standard temp dir is usable Python's tempfile falls
+# back to the cwd, and DataLoader workers then try to bind their AF_UNIX IPC
+# sockets there -- which NFS rejects with "Operation not permitted", crashing on
+# the first batch and littering the repo with tmpXXXX scratch dirs. /dev/shm is
+# local RAM, world-writable and supports unix sockets.
+TMPDIR="${TMPDIR:-/dev/shm/tsqa_sft_tmp_${RUN_ID}}"
+mkdir -p "$TMPDIR" || fail "Cannot create local TMPDIR: $TMPDIR (set TMPDIR=/path to a writable local dir)."
+export TMPDIR TMP="$TMPDIR" TEMP="$TMPDIR"
+trap 'rm -rf "$TMPDIR" 2>/dev/null || true' EXIT
+
+[ -x "$PYTHON_BIN" ] || fail "Python executable not found: $PYTHON_BIN. Set PYTHON_BIN=/path/to/bin/python."
+[ -f "$ACCELERATE_CONFIG" ] || fail "Accelerate config not found: $ACCELERATE_CONFIG"
+[ -f "${DATA_ROOT}/${TRAIN_FILE}" ] || fail "TSQA train file not found: ${DATA_ROOT}/${TRAIN_FILE}"
+[ -f "${DATA_ROOT}/${EVAL_FILE}" ] || fail "TSQA eval file not found: ${DATA_ROOT}/${EVAL_FILE}"
+[ -n "$MODEL_PATH" ] || fail "MODEL_PATH is required. Use a local ChatTime model path, or set ALLOW_HF_DOWNLOAD=1 with a HuggingFace model id."
+
+if [ ! -d "$MODEL_PATH" ] && [ "$ALLOW_HF_DOWNLOAD" != "1" ]; then
+    fail "MODEL_PATH is not a local directory: $MODEL_PATH. Set ALLOW_HF_DOWNLOAD=1 if you intentionally want HuggingFace downloads."
+fi
+
+"$PYTHON_BIN" -c "import accelerate; print('accelerate:', accelerate.__version__)" || \
+    fail "The selected Python cannot import accelerate: $PYTHON_BIN"
+ACCELERATE_CMD=("$PYTHON_BIN" -m accelerate.commands.accelerate_cli)
+
+if [ "$LOAD_IN_4BIT" = "1" ]; then
+    "$PYTHON_BIN" -c "import importlib.metadata as m; print('bitsandbytes:', m.version('bitsandbytes'))" || \
+        fail "LOAD_IN_4BIT=1 requires bitsandbytes in $PYTHON_BIN."
+fi
+
+TRAIN_EXTRA_ARGS=()
+if [ -n "$MAX_TRAIN_SAMPLES" ]; then
+    TRAIN_EXTRA_ARGS+=(--max_train_samples "$MAX_TRAIN_SAMPLES")
+fi
+if [ "$ALLOW_HF_DOWNLOAD" = "1" ]; then
+    TRAIN_EXTRA_ARGS+=(--allow_hf_download)
+fi
+if [ "$LOAD_IN_4BIT" = "1" ]; then
+    TRAIN_EXTRA_ARGS+=(--load_in_4bit)
+fi
+if [ "$GRADIENT_CHECKPOINTING" = "1" ]; then
+    TRAIN_EXTRA_ARGS+=(--gradient_checkpointing)
+fi
+if [ "$SKIP_OVERLONG" = "1" ]; then
+    TRAIN_EXTRA_ARGS+=(--skip_overlong)
+else
+    TRAIN_EXTRA_ARGS+=(--no_skip_overlong)
+fi
+
+echo "Run id: $RUN_ID"
+echo "Python: $PYTHON_BIN"
+echo "Accelerate: ${ACCELERATE_CMD[*]}"
+echo "CUDA_VISIBLE_DEVICES: $CUDA_VISIBLE_DEVICES"
+echo "Model path: $MODEL_PATH"
+echo "Data root: $DATA_ROOT (train=$TRAIN_FILE eval=$EVAL_FILE)"
+echo "Adapter output dir: $ADAPTER_OUTPUT_DIR"
+echo "Eval output dir: $EVAL_OUTPUT_DIR"
+echo "Log file: $LOG_FILE"
+
+"${ACCELERATE_CMD[@]}" launch --config_file "$ACCELERATE_CONFIG" \
+    tsqa_adapter/finetune_tsqa_lora.py \
+    --model_path "$MODEL_PATH" \
+    --data_root "$DATA_ROOT" \
+    --train_file "$TRAIN_FILE" \
+    --output_dir "$ADAPTER_OUTPUT_DIR" \
+    --max_seq_length "$MAX_SEQ_LENGTH" \
+    --per_device_train_batch_size "$PER_DEVICE_TRAIN_BATCH_SIZE" \
+    --gradient_accumulation_steps "$GRADIENT_ACCUMULATION_STEPS" \
+    --num_train_epochs "$NUM_TRAIN_EPOCHS" \
+    --learning_rate "$LEARNING_RATE" \
+    --lora_rank "$LORA_RANK" \
+    --lora_alpha "$LORA_ALPHA" \
+    --lora_dropout "$LORA_DROPOUT" \
+    --save_steps "$SAVE_STEPS" \
+    --logging_steps "$LOGGING_STEPS" \
+    --save_total_limit "$SAVE_TOTAL_LIMIT" \
+    --dataloader_num_workers "$DATALOADER_NUM_WORKERS" \
+    --torch_dtype "$TORCH_DTYPE" \
+    "${TRAIN_EXTRA_ARGS[@]}"
+
+if [ "$RUN_EVAL_AFTER_SFT" = "1" ]; then
+    EVAL_EXTRA_ARGS=()
+    if [ -n "$MAX_EVAL_SAMPLES" ]; then
+        EVAL_EXTRA_ARGS+=(--max_eval_samples "$MAX_EVAL_SAMPLES")
+    fi
+    if [ "$ALLOW_HF_DOWNLOAD" = "1" ]; then
+        EVAL_EXTRA_ARGS+=(--allow_hf_download)
+    fi
+
+    "${ACCELERATE_CMD[@]}" launch --config_file "$ACCELERATE_CONFIG" \
+        tsqa_adapter/eval_tsqa.py \
+        --model_path "$MODEL_PATH" \
+        --adapter_path "$ADAPTER_OUTPUT_DIR" \
+        --data_root "$DATA_ROOT" \
+        --eval_file "$EVAL_FILE" \
+        --output_dir "$EVAL_OUTPUT_DIR" \
+        --eval_batch_size "$EVAL_BATCH_SIZE" \
+        --max_new_tokens "$MAX_NEW_TOKENS" \
+        --max_input_tokens "$MAX_INPUT_TOKENS" \
+        --torch_dtype "$TORCH_DTYPE" \
+        "${EVAL_EXTRA_ARGS[@]}"
+fi

tsqa_adapter/tsqa_common.py

@@ -0,0 +1,197 @@
+"""Time-MQA TSQA dataset adapter for ChatTime.
+
+This mirrors ``rats40k_adapter`` but targets the multi-task Time-MQA TSQA
+benchmark (``/mnt/share01/sqk/datasets/Time-MQA_TSQA/tmp``), which contains four
+source groups -- anomaly_detection, classification, forecasting and open_ended
+-- with free-text answers.
+
+Design choices:
+
+* ChatTime has no chat template, so prompts use ChatTime's native analysis
+  template (``utils.prompt.getPrompt``): the natural-language ``question`` goes
+  into ``### Instruction`` (kept verbatim, including any embedded numbers so
+  forecasting keeps its absolute scale) and the ChatTime-serialised series goes
+  into ``### Input`` so the model still receives its native time-series tokens.
+* The free-text ``answer`` is the ``### Response`` target.
+* Some context-enhanced forecasting questions are far longer than ChatTime's
+  4096-token context. Callers left-truncate the prompt (keeping the series, the
+  trailing ask and the full response) rather than failing.
+* Evaluation reuses the canonical Time-MQA evaluator
+  (``MQA/data_utils.compute_group_metrics``) so ChatTime numbers are directly
+  comparable to the other TSQA baselines.
+"""
+
+import json
+import os
+import sys
+from collections import defaultdict
+from pathlib import Path
+
+import numpy as np
+
+
+CHAT_TIME_DIR = Path(__file__).resolve().parents[1]
+MQA_DIR = Path(os.environ.get("MQA_DIR", "/mnt/share01/sqk/MQA"))
+if str(CHAT_TIME_DIR) not in sys.path:
+    sys.path.insert(0, str(CHAT_TIME_DIR))
+
+from utils.prompt import getPrompt  # noqa: E402
+from utils.tools import Discretizer, Serializer  # noqa: E402
+
+
+DEFAULT_DATA_ROOT = "/mnt/share01/sqk/datasets/Time-MQA_TSQA/tmp"
+SOURCE_ORDER = ("anomaly_detection", "classification", "forecasting", "open_ended")
+
+# Fields carried into the saved prediction rows so the Time-MQA evaluator and
+# manual inspection have everything they need.
+RESULT_FIELDS = (
+    "id",
+    "figure_path",
+    "application_domain",
+    "task_type",
+    "source_type",
+    "question_format",
+)
+
+
+def resolve_data_file(data_root, data_file):
+    """Resolve a split file: absolute override, or relative to ``data_root``."""
+    path = Path(data_file)
+    if not path.is_absolute():
+        path = Path(data_root) / path
+    if not path.is_file():
+        raise FileNotFoundError(f"TSQA JSONL not found: {path}")
+    return str(path)
+
+
+def load_tsqa_records(path):
+    """Load TSQA JSONL records, dropping rows without both a question and answer."""
+    records = []
+    with open(path, "r", encoding="utf-8") as handle:
+        for line in handle:
+            line = line.strip()
+            if not line:
+                continue
+            record = json.loads(line)
+            if not str(record.get("question") or "").strip():
+                continue
+            if not str(record.get("answer") or "").strip():
+                continue
+            records.append(record)
+    return records
+
+
+def balanced_limit(records, max_samples):
+    """Round-robin a balanced subset across source groups (for smoke tests)."""
+    if not max_samples or max_samples <= 0 or len(records) <= max_samples:
+        return records
+
+    grouped = defaultdict(list)
+    for record in records:
+        grouped[str(record.get("source_type") or "unknown")].append(record)
+
+    names = [name for name in SOURCE_ORDER if grouped.get(name)]
+    names.extend(sorted(name for name in grouped if name not in names))
+
+    selected = []
+    offsets = defaultdict(int)
+    while len(selected) < max_samples:
+        added = False
+        for name in names:
+            offset = offsets[name]
+            if offset < len(grouped[name]):
+                selected.append(grouped[name][offset])
+                offsets[name] += 1
+                added = True
+                if len(selected) == max_samples:
+                    break
+        if not added:
+            break
+    return selected
+
+
+def _to_float_array(series):
+    arr = np.asarray(series, dtype=float).reshape(-1)
+    if arr.size == 0:
+        return arr
+    if not np.isfinite(arr).all():
+        finite = np.isfinite(arr)
+        fill = float(np.median(arr[finite])) if finite.any() else 0.0
+        arr = np.where(np.isfinite(arr), arr, fill)
+    return arr
+
+
+def serialize_series(series):
+    """Discretise + serialise a univariate series into ChatTime time tokens."""
+    arr = _to_float_array(series)
+    if arr.size == 0:
+        return ""
+    discretizer = Discretizer()
+    serializer = Serializer()
+    return serializer.serialize(discretizer.discretize(arr))
+
+
+def build_prompt(question, series, response=None):
+    """Build a ChatTime prompt: question -> Instruction, series -> Input."""
+    question = str(question or "").strip()
+    response = "" if response is None else str(response)
+    serialized = serialize_series(series)
+    if serialized:
+        return getPrompt(
+            flag="analysis",
+            instruction=question,
+            input=serialized,
+            response=response,
+        )
+    # No usable series: fall back to a plain instruction prompt.
+    return getPrompt(
+        flag="general",
+        instruction=question,
+        input="",
+        response=response,
+    )
+
+
+def build_result_row(record, prediction):
+    """Assemble a prediction row in the Time-MQA evaluator schema."""
+    row = {field: str(record.get(field) or "") for field in RESULT_FIELDS}
+    row["question"] = str(record.get("question") or "")
+    row["answer"] = str(record.get("answer") or "")
+    row["prediction"] = "" if prediction is None else str(prediction).strip()
+    return row
+
+
+def compute_tsqa_metrics(rows):
+    """Per-group TSQA metrics, reusing the canonical Time-MQA evaluator."""
+    if str(MQA_DIR) not in sys.path:
+        sys.path.insert(0, str(MQA_DIR))
+    from data_utils import compute_group_metrics, compute_text_metrics
+
+    rows = list(rows)
+    metrics = {"by_group": compute_group_metrics(rows)}
+    metrics["text_overall"] = compute_text_metrics(rows)
+    counts = defaultdict(int)
+    for row in rows:
+        counts[str(row.get("source_type") or "unknown")] += 1
+    metrics["num_samples"] = len(rows)
+    metrics["counts_by_group"] = dict(counts)
+    return metrics
+
+
+def atomic_write_json(obj, path):
+    path = Path(path)
+    path.parent.mkdir(parents=True, exist_ok=True)
+    tmp = path.with_suffix(path.suffix + f".tmp.{os.getpid()}")
+    with open(tmp, "w", encoding="utf-8") as handle:
+        json.dump(obj, handle, indent=2, ensure_ascii=False)
+    os.replace(tmp, path)
+
+
+def write_jsonl(rows, path):
+    path = Path(path)
+    path.parent.mkdir(parents=True, exist_ok=True)
+    tmp = path.with_suffix(path.suffix + f".tmp.{os.getpid()}")
+    with open(tmp, "w", encoding="utf-8") as handle:
+        for row in rows:
+            handle.write(json.dumps(row, ensure_ascii=False) + "\n")
+    os.replace(tmp, path)

utils/prompt.py

@@ -0,0 +1,57 @@
+GEN_SYS_PROMPT = "Below is an instruction that describes a task. Write a response that appropriately completes the request."
+PRE_SYS_PROMPT = "You are a helpful assistant that performs time series prediction. The user will provide a sequence and you will predict the sequence."
+ANA_SYS_PROMPT = "You are a helpful assistant that performs time series analysis. The user will provide a sequence and you will respond to the questions based on this sequence."
+
+PRE_INST_PROMPT = "Please predict the following sequence carefully."
+PRE_INST_PROMPT_TEXT = "Please predict the following sequence carefully. Context knowledge you may consider: {}"
+ANA_INST_PROMPT_TEXT = "Please answer the following question carefully after analyzing the sequence: {}"
+
+TEMPLATE = """{}
+
+### Instruction:
+{}
+
+### Input:
+{}
+
+### Response:
+{}"""
+
+
+def getPrompt(flag, instruction=None, input=None, response=None, context=None):
+    if flag == "general":  # instruction, [input], [response]
+        system = GEN_SYS_PROMPT
+        if instruction is None:
+            raise ValueError("Instruction must be provided for general tasks.")
+        else:
+            instruction = instruction
+        input = "" if input is None else input
+        response = "" if response is None else response
+
+    elif flag == "prediction":  # [context], input, [response]
+        system = PRE_SYS_PROMPT
+        instruction = PRE_INST_PROMPT if context is None else PRE_INST_PROMPT_TEXT.format(context)
+        if input is None:
+            raise ValueError("Input must be provided for prediction tasks.")
+        else:
+            input = input
+        response = "" if response is None else response
+
+    elif flag == "analysis":  # instruction, input, [response]
+        system = ANA_SYS_PROMPT
+        if instruction is None:
+            raise ValueError("Instruction must be provided for analysis tasks.")
+        else:
+            instruction = ANA_INST_PROMPT_TEXT.format(instruction)
+        if input is None:
+            raise ValueError("Input must be provided for analysis tasks.")
+        else:
+            input = input
+        response = "" if response is None else response
+
+    else:
+        raise ValueError("Flag must be one of 'general', 'prediction', or 'analysis'.")
+
+    prompt = TEMPLATE.format(system, instruction, input, response)
+
+    return prompt

utils/tools.py

@@ -0,0 +1,64 @@
+import re
+
+import numpy as np
+from sklearn.preprocessing import MinMaxScaler
+
+
+class Discretizer:
+    def __init__(self, low_limit=-1, high_limit=1, n_tokens=10002):
+        self.scaler = MinMaxScaler()
+
+        self.boundaries = np.linspace(low_limit, high_limit, n_tokens - 1)
+        self.centers = (self.boundaries[1:] + self.boundaries[:-1]) / 2
+        self.centers = np.concatenate((self.centers[:1], self.centers, self.centers[-1:]))
+
+    def get_centers(self):
+        return self.centers
+
+    def discretize(self, context, fit_length=None):
+        fit_length = len(context) if fit_length is None else fit_length
+        self.scaler.fit(context[:fit_length].reshape(-1, 1))
+        scaled_context = self.scaler.transform(context.reshape(-1, 1)).reshape(-1) - 0.5
+
+        bin_ids = np.digitize(x=scaled_context, bins=self.boundaries, right=True)
+        dispersed_context = self.centers[bin_ids]
+
+        dispersed_context[np.isnan(context)] = np.nan
+
+        return dispersed_context
+
+    def inverse_discretize(self, scaled_context):
+        context = self.scaler.inverse_transform(scaled_context.reshape(-1, 1) + 0.5).reshape(-1)
+
+        return context
+
+
+class Serializer:
+    def __init__(self, prec=4, time_sep=" ", time_flag="###", nan_flag="Nan"):
+        self.prec = prec
+        self.time_sep = time_sep
+        self.time_flag = time_flag
+        self.nan_flag = nan_flag
+
+    def serialize(self, context):
+        serialized_context = np.array([f"{self.time_flag}{i:.{self.prec}f}{self.time_flag}" for i in context])
+        serialized_context[np.isnan(context)] = f"{self.time_flag}{self.nan_flag}{self.time_flag}"
+        serialized_context = self.time_sep.join(serialized_context)
+
+        return serialized_context
+
+    def inverse_serialize(self, serialized_context):
+        pattern = rf"{self.time_flag}(.*?){self.time_flag}"
+        matches = re.findall(pattern, serialized_context)
+
+        context = []
+        for num in matches:
+            try:
+                context.append(float(num))
+            except ValueError as e:
+                print(e)
+                context.append(np.NaN)
+
+        context = np.array(context)
+
+        return context