File size: 83,852 Bytes
22087d0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
"""
Context Management for AgentPress Threads.

This module handles token counting and thread summarization to prevent
reaching the context window limitations of LLM models.
"""

import json
import os
import asyncio
from typing import List, Dict, Any, Optional, Union

from litellm.utils import token_counter
from anthropic import Anthropic
from core.services.supabase import DBConnection
from core.utils.logger import logger
from core.ai_models import model_manager
from core.agentpress.prompt_caching import apply_anthropic_caching_strategy, supports_prompt_caching

DEFAULT_TOKEN_THRESHOLD = 120000

# Module-level singleton clients for memory efficiency
# These are lazily initialized once and reused across all ContextManager instances
_anthropic_client = None
_bedrock_client = None
_clients_initialized = False


def _get_anthropic_client_singleton():
    """Module-level lazy initialization of Anthropic client (singleton)."""
    global _anthropic_client, _clients_initialized
    if _anthropic_client is None and not _clients_initialized:
        api_key = os.environ.get("ANTHROPIC_API_KEY")
        if api_key:
            _anthropic_client = Anthropic(api_key=api_key)
        _clients_initialized = True
    return _anthropic_client


def _get_bedrock_client_singleton():
    """Module-level lazy initialization of Bedrock client (singleton)."""
    global _bedrock_client
    if _bedrock_client is None:
        try:
            import boto3
            _bedrock_client = boto3.client('bedrock-runtime', region_name='us-west-2')
        except Exception as e:
            logger.debug(f"Could not initialize Bedrock client: {e}")
    return _bedrock_client


class ContextManager:
    """Manages thread context including token counting and summarization."""
    
    def __init__(self, token_threshold: int = DEFAULT_TOKEN_THRESHOLD, db=None):
        """Initialize the ContextManager.
        
        Args:
            token_threshold: Token count threshold to trigger summarization
            db: Optional DBConnection instance to reuse (avoids creating new ones)
        """
        self.db = db if db is not None else DBConnection()
        self.token_threshold = token_threshold
        # Tool output management
        self.keep_recent_tool_outputs = 5  # Number of recent tool outputs to preserve
        # Compression strategy
        self.compression_target_ratio = 0.6  # Compress to 60% of max tokens (hysteresis)
        self.keep_recent_user_messages = 10  # Number of recent user messages to keep uncompressed
        self.keep_recent_assistant_messages = 10  # Number of recent assistant messages to keep uncompressed

    def _get_anthropic_client(self):
        """Get the singleton Anthropic client."""
        return _get_anthropic_client_singleton()
    
    async def save_compressed_messages(self, compressed_messages: List[Dict[str, Any]]) -> int:
        """Save compressed message content to database metadata.
        
        Updates messages in DB with:
        - metadata.compressed = True
        - metadata.compressed_content = the compressed content
        
        This allows future reads to use compressed content directly without re-compressing.
        Always updates compressed_content when provided - allows tiered compression to
        upgrade already-compressed messages to more aggressive compression.
        
        Args:
            compressed_messages: List of dicts with 'message_id' and 'compressed_content'
            
        Returns:
            Number of messages successfully saved
        """
        if not compressed_messages:
            return 0
        
        from core.threads import repo as threads_repo
        
        saved_count = await threads_repo.save_compressed_messages_batch(compressed_messages)
        
        if saved_count > 0:
            logger.info(f"💾 Compression save: {saved_count} messages saved")
        
        return saved_count
    
    def _get_bedrock_client(self):
        """Get the singleton Bedrock client."""
        return _get_bedrock_client_singleton()

    async def count_tokens(self, model: str, messages: List[Dict[str, Any]], system_prompt: Optional[Dict[str, Any]] = None, apply_caching: bool = True) -> int:
        """Count tokens using the correct tokenizer for the model.
        
        For Anthropic/Claude models: Uses Anthropic's official tokenizer
        For Bedrock models: Uses Bedrock's count_tokens API
        For other models: Uses LiteLLM's token_counter
        
        IMPORTANT: By default, applies caching transformation before counting to match
        the actual token count that will be sent to the API.
        
        Args:
            model: Model name
            messages: List of messages
            system_prompt: Optional system prompt
            apply_caching: If True, temporarily apply caching transformation before counting
            
        Returns:
            Token count (with caching overhead if apply_caching=True)
        """
        # Apply caching transformation if requested (to match API reality)
        messages_to_count = messages
        system_to_count = system_prompt
        
        if apply_caching and supports_prompt_caching(model):
            try:
                # Temporarily apply caching transformation
                prepared = await apply_anthropic_caching_strategy(
                    system_prompt, messages, model, thread_id=None, force_recalc=False
                )
                # Separate system from messages
                system_to_count = None
                messages_to_count = []
                for msg in prepared:
                    if msg.get('role') == 'system':
                        system_to_count = msg
                    else:
                        messages_to_count.append(msg)
            except Exception as e:
                logger.debug(f"Failed to apply caching for counting: {e}")
                # Continue with uncached messages
        
        # Check if this is an Anthropic model
        if 'claude' in model.lower() or 'anthropic' in model.lower():
            # Use Anthropic's official tokenizer
            try:
                client = self._get_anthropic_client()
                if client:
                    # Strip provider prefix
                    clean_model = model.split('/')[-1] if '/' in model else model
                    
                    # Clean messages - only role and content
                    clean_messages = []
                    for msg in messages_to_count:
                        if msg.get('role') == 'system':
                            continue  # System passed separately
                        clean_messages.append({
                            'role': msg.get('role'),
                            'content': msg.get('content')
                        })
                    
                    # Extract system content
                    system_content = None
                    if system_to_count and isinstance(system_to_count, dict):
                        system_content = system_to_count.get('content')
                    
                    # Build parameters
                    count_params = {'model': clean_model, 'messages': clean_messages}
                    if system_content:
                        count_params['system'] = system_content
                    
                    result = client.messages.count_tokens(**count_params)
                    return result.input_tokens
            except Exception as e:
                logger.debug(f"Anthropic token counting failed, falling back to LiteLLM: {e}")
        
        # Check if this is a Bedrock model
        elif 'bedrock' in model.lower():
            try:
                bedrock_client = self._get_bedrock_client()
                if bedrock_client:
                    # Import profile IDs
                    from core.ai_models.registry import KIMI_K2_PROFILE_ID, SONNET_4_5_PROFILE_ID, HAIKU_4_5_PROFILE_ID, MINIMAX_M2_PROFILE_ID
                    
                    model_id_mapping = {
                        KIMI_K2_PROFILE_ID: "moonshot.kimi-k2-thinking",  # Kimi K2 (Basic Mode)
                        SONNET_4_5_PROFILE_ID: "us.anthropic.claude-sonnet-4-5-20250929-v1:0",  # Sonnet 4.5 (Power Mode)
                        "tyj1ks3nj9qf": "anthropic.claude-sonnet-4-20250514-v1:0",  # Sonnet 4 (no constant defined)
                        HAIKU_4_5_PROFILE_ID: "anthropic.claude-haiku-4-5-20251001-v1:0",  # HAIKU 4.5 (Legacy - replaced by Kimi K2)
                        MINIMAX_M2_PROFILE_ID: "minimax.minimax-m2",  # MiniMax M2
                    }
                    
                    # Extract profile ID from ARN
                    bedrock_model_id = None
                    if "application-inference-profile" in model:
                        profile_id = model.split("/")[-1]
                        bedrock_model_id = model_id_mapping.get(profile_id)
                    
                    if not bedrock_model_id:
                        # Import Kimi K2 profile ID for default
                        from core.ai_models.registry import KIMI_K2_PROFILE_ID
                        bedrock_model_id = model_id_mapping.get(KIMI_K2_PROFILE_ID, "moonshot.kimi-k2-thinking")  # Default to Kimi K2
                    
                    # Clean content blocks for Bedrock Converse API
                    def clean_content_for_bedrock(content):
                        """
                        Convert Anthropic format to Bedrock Converse API format.
                        Converts cache_control -> cachePoint to preserve cache overhead in token counts.
                        """
                        if isinstance(content, str):
                            return [{'text': content}]
                        elif isinstance(content, list):
                            cleaned = []
                            for block in content:
                                if isinstance(block, dict):
                                    # Extract text
                                    if 'text' in block:
                                        cleaned.append({'text': block['text']})
                                        # Convert cache_control to cachePoint (separate block)
                                        if 'cache_control' in block:
                                            cleaned.append({'cachePoint': {'type': 'default'}})
                            return cleaned if cleaned else [{'text': str(content)}]
                        return [{'text': str(content)}]
                    
                    # Format messages for Bedrock
                    bedrock_messages = []
                    system_content = None
                    
                    for msg in messages_to_count:
                        if msg.get('role') == 'system':
                            system_content = clean_content_for_bedrock(msg.get('content'))
                            continue
                        
                        bedrock_messages.append({
                            'role': msg.get('role'),
                            'content': clean_content_for_bedrock(msg.get('content'))
                        })
                    
                    # Build input
                    input_to_count = {'messages': bedrock_messages}
                    if system_content:
                        input_to_count['system'] = system_content
                    elif system_to_count:
                        input_to_count['system'] = clean_content_for_bedrock(system_to_count.get('content'))
                    
                    # Call Bedrock count_tokens API
                    response = bedrock_client.count_tokens(
                        modelId=bedrock_model_id,
                        input={'converse': input_to_count}
                    )
                    
                    return response['inputTokens']
            except Exception as e:
                logger.debug(f"Bedrock token counting failed, falling back to LiteLLM: {e}")
        
        # Fallback to LiteLLM token_counter (wrap in thread pool - CPU-heavy tiktoken operation)
        if system_to_count:
            return await asyncio.to_thread(token_counter, model=model, messages=[system_to_count] + messages_to_count)
        else:
            return await asyncio.to_thread(token_counter, model=model, messages=messages_to_count)

    async def estimate_token_usage(self, prompt_messages: List[Dict[str, Any]], completion_content: str, model: str) -> Dict[str, Any]:
        """
        Estimate token usage for billing when exact usage is unavailable.
        This is critical for billing on timeouts, crashes, disconnects, etc.
        
        Uses provider-specific APIs (Anthropic/Bedrock) when available for accuracy,
        with fallbacks to LiteLLM token_counter and word count estimation.
        
        Args:
            prompt_messages: The prompt messages sent to the LLM
            completion_content: The accumulated completion text
            model: Model name
            
        Returns:
            Dict with prompt_tokens, completion_tokens, total_tokens, estimated=True
        """
        try:
            # Count prompt tokens using accurate provider APIs
            prompt_tokens = await self.count_tokens(model, prompt_messages, apply_caching=False)
            
            # Count completion tokens (just the text) - wrap in thread pool (CPU-heavy tiktoken operation)
            completion_tokens = 0
            if completion_content:
                completion_tokens = await asyncio.to_thread(token_counter, model=model, text=completion_content)
            
            total_tokens = prompt_tokens + completion_tokens
            
            logger.warning(f"⚠️ ESTIMATED TOKEN USAGE: prompt={prompt_tokens}, completion={completion_tokens}, total={total_tokens}")
            
            return {
                "prompt_tokens": prompt_tokens,
                "completion_tokens": completion_tokens,
                "total_tokens": total_tokens,
                "estimated": True
            }
        except Exception as e:
            logger.error(f"Context manager estimation failed: {e}, falling back to LiteLLM")
            # Fallback to LiteLLM
            try:
                prompt_tokens = token_counter(model=model, messages=prompt_messages)
                completion_tokens = token_counter(model=model, text=completion_content) if completion_content else 0
                
                logger.warning(f"⚠️ ESTIMATED TOKEN USAGE (LiteLLM): prompt={prompt_tokens}, completion={completion_tokens}")
                
                return {
                    "prompt_tokens": prompt_tokens,
                    "completion_tokens": completion_tokens,
                    "total_tokens": prompt_tokens + completion_tokens,
                    "estimated": True
                }
            except Exception as e2:
                logger.error(f"LiteLLM estimation failed: {e2}, using word count fallback")
                # Final fallback to word count
                fallback_prompt = len(' '.join(str(m.get('content', '')) for m in prompt_messages).split()) * 1.3
                fallback_completion = len(completion_content.split()) * 1.3 if completion_content else 0
                
                logger.warning(f"⚠️ FALLBACK TOKEN ESTIMATION: prompt≈{int(fallback_prompt)}, completion≈{int(fallback_completion)}")
                
                return {
                    "prompt_tokens": int(fallback_prompt),
                    "completion_tokens": int(fallback_completion),
                    "total_tokens": int(fallback_prompt + fallback_completion),
                    "estimated": True,
                    "fallback": True
                }
    
    def is_tool_result_message(self, msg: Dict[str, Any]) -> bool:
        """Check if a message is a tool result message.

        Detects tool results from:
        1. Native tool calls: role="tool"
        2. Native tool calls: has tool_call_id field
        3. XML tool calls: role="user" ONLY if explicitly marked in metadata
        """
        if not isinstance(msg, dict):
            return False

        # Native tool calls have role="tool"
        if msg.get('role') == 'tool':
            return True

        # Native tool calls have tool_call_id at top level
        if 'tool_call_id' in msg:
            return True

        # CONSERVATIVE CHECK: For role="user", only consider it a tool result if:
        # 1. It has explicit tool_call_id in metadata (indicating it's a saved tool result)
        # This prevents false positives that break message validation
        if msg.get('role') == 'user':
            metadata = msg.get('metadata', {})
            if isinstance(metadata, dict) and 'tool_call_id' in metadata:
                return True

        return False
    
    def get_tool_call_ids_from_message(self, msg: Dict[str, Any]) -> List[str]:
        """Extract tool_call IDs from an assistant message with tool_calls.
        
        Returns list of tool_call IDs, or empty list if no tool_calls.
        """
        if not isinstance(msg, dict) or msg.get('role') != 'assistant':
            return []
        
        tool_calls = msg.get('tool_calls') or []
        if not tool_calls or not isinstance(tool_calls, list):
            return []
        
        ids = []
        for tc in tool_calls:
            if isinstance(tc, dict):
                tc_id = tc.get('id')
                if tc_id:
                    ids.append(tc_id)
        return ids
    
    def get_tool_call_id_from_result(self, msg: Dict[str, Any]) -> Optional[str]:
        """Extract the tool_call_id from a tool result message.

        Returns the tool_call_id, or None if not a tool result message.
        """
        if not isinstance(msg, dict):
            return None

        # Native tool results have tool_call_id directly at top level
        if 'tool_call_id' in msg:
            return msg.get('tool_call_id')

        # role="tool" messages should have tool_call_id
        if msg.get('role') == 'tool':
            return msg.get('tool_call_id')

        # Check metadata for tool_call_id (XML tool results or saved tool results)
        metadata = msg.get('metadata', {})
        if isinstance(metadata, dict) and 'tool_call_id' in metadata:
            return metadata.get('tool_call_id')

        return None
    
    def group_messages_by_tool_calls(self, messages: List[Dict[str, Any]]) -> List[List[Dict[str, Any]]]:
        """Group messages into atomic units respecting tool call pairing.
        
        CRITICAL: This ensures assistant messages with tool_calls are always grouped
        with their corresponding tool result messages. These groups must be treated
        as atomic units that cannot be split during compression or caching.
        
        Rules:
        - An assistant message with tool_calls + all following tool messages with 
          matching tool_call_ids = one atomic group
        - Regular messages (user, assistant without tool_calls) = standalone groups
        - Tool results without a preceding assistant (orphans) = standalone groups (with warning)
        
        Args:
            messages: List of conversation messages
            
        Returns:
            List of message groups, where each group is a list of messages
        """
        if not messages:
            return []
        
        groups: List[List[Dict[str, Any]]] = []
        current_group: List[Dict[str, Any]] = []
        expected_tool_call_ids: set = set()  # IDs we're waiting for results
        
        for msg in messages:
            role = msg.get('role', '')
            
            # Check if this is an assistant message with tool_calls
            tool_call_ids = self.get_tool_call_ids_from_message(msg)
            
            if tool_call_ids:
                # If we have a pending group, save it first
                if current_group:
                    groups.append(current_group)
                
                # Start a new group with this assistant message
                current_group = [msg]
                expected_tool_call_ids = set(tool_call_ids)
                
            elif self.is_tool_result_message(msg):
                # This is a tool result message
                tool_call_id = self.get_tool_call_id_from_result(msg)
                
                if tool_call_id and tool_call_id in expected_tool_call_ids:
                    # This tool result belongs to the current group
                    current_group.append(msg)
                    expected_tool_call_ids.discard(tool_call_id)
                    
                    # If we've received all expected tool results, close the group
                    if not expected_tool_call_ids:
                        groups.append(current_group)
                        current_group = []
                else:
                    # Orphaned tool result - doesn't match any expected ID
                    # Log warning and treat as standalone
                    if tool_call_id:
                        logger.warning(f"⚠️ Orphaned tool result detected: tool_call_id={tool_call_id} has no matching assistant message")
                    
                    # Close current group if any
                    if current_group:
                        groups.append(current_group)
                        current_group = []
                        expected_tool_call_ids = set()
                    
                    # Add orphan as standalone group
                    groups.append([msg])
            else:
                # Regular message (user or assistant without tool_calls)
                # Close current group if we have pending tool calls
                if current_group:
                    if expected_tool_call_ids:
                        logger.warning(f"⚠️ Closing tool call group with {len(expected_tool_call_ids)} missing tool results")
                    groups.append(current_group)
                    current_group = []
                    expected_tool_call_ids = set()
                
                # Add as standalone group
                groups.append([msg])
        
        # Don't forget the last group
        if current_group:
            if expected_tool_call_ids:
                logger.warning(f"⚠️ Final group has {len(expected_tool_call_ids)} missing tool results")
            groups.append(current_group)
        
        return groups
    
    def flatten_message_groups(self, groups: List[List[Dict[str, Any]]]) -> List[Dict[str, Any]]:
        """Flatten message groups back into a flat list of messages.
        
        Args:
            groups: List of message groups
            
        Returns:
            Flat list of messages preserving order
        """
        result = []
        for group in groups:
            result.extend(group)
        return result
    
    def validate_tool_call_pairing(self, messages: List[Dict[str, Any]]) -> tuple[bool, List[str], List[str]]:
        """Validate that tool calls and tool results are properly paired in BOTH directions.
        
        Bedrock requires:
        1. Every tool result must have a preceding assistant message with matching tool_call_id
        2. Every assistant tool_call must have a following tool result with matching tool_call_id
        
        This should be called before sending messages to the LLM to catch any
        issues that would cause Bedrock errors.
        
        Args:
            messages: List of messages to validate
            
        Returns:
            Tuple of (is_valid, orphaned_tool_result_ids, unanswered_tool_call_ids)
        """
        # Track all tool_call IDs from assistant messages
        all_tool_call_ids: set = set()
        # Track all tool_call_ids that have been answered with tool results
        answered_tool_call_ids: set = set()
        # Track orphaned tool results (results without matching assistant)
        orphaned_tool_result_ids: List[str] = []
        
        # First pass: collect all tool_call IDs from assistant messages
        for msg in messages:
            tool_call_ids = self.get_tool_call_ids_from_message(msg)
            all_tool_call_ids.update(tool_call_ids)
        
        # Second pass: check tool results and track which tool_calls are answered
        for msg in messages:
            if self.is_tool_result_message(msg):
                tool_call_id = self.get_tool_call_id_from_result(msg)
                if tool_call_id:
                    if tool_call_id not in all_tool_call_ids:
                        # Tool result without matching assistant
                        orphaned_tool_result_ids.append(tool_call_id)
                    else:
                        # This tool_call has been answered
                        answered_tool_call_ids.add(tool_call_id)
        
        # Find unanswered tool_calls (assistant tool_calls without matching tool results)
        unanswered_tool_call_ids = list(all_tool_call_ids - answered_tool_call_ids)
        
        is_valid = len(orphaned_tool_result_ids) == 0 and len(unanswered_tool_call_ids) == 0
        
        if orphaned_tool_result_ids:
            logger.error(f"🚨 VALIDATION FAILED: {len(orphaned_tool_result_ids)} orphaned tool results (no matching assistant): {orphaned_tool_result_ids}")
        if unanswered_tool_call_ids:
            logger.error(f"🚨 VALIDATION FAILED: {len(unanswered_tool_call_ids)} unanswered tool calls (no matching tool result): {unanswered_tool_call_ids}")
        
        return is_valid, orphaned_tool_result_ids, unanswered_tool_call_ids
    
    def needs_tool_ordering_repair(self, messages: List[Dict[str, Any]]) -> bool:
        """Fast O(n) check if messages have tool ordering issues. Exits early in common case.
        
        This is a lightweight pre-check that detects if user messages interrupt the
        tool_use -> tool_result flow. Only returns True if a problem is detected, allowing
        callers to skip expensive full validation in the common case.
        
        Args:
            messages: List of messages to check
            
        Returns:
            True if tool ordering issues detected (needs repair), False otherwise
        """
        for i, msg in enumerate(messages):
            if not isinstance(msg, dict):
                continue
            
            # Check if this is an assistant with tool_calls
            tool_call_ids = self.get_tool_call_ids_from_message(msg)
            if not tool_call_ids:
                continue
            
            # Check if immediately following messages are the expected tool results
            expected_count = len(tool_call_ids)
            found_tool_call_ids = set()
            
            for j in range(i + 1, min(i + 1 + expected_count + 1, len(messages))):
                next_msg = messages[j] if j < len(messages) else None
                if not next_msg:
                    # Missing tool results - needs repair
                    return True
                
                # If we hit a user message (not tool result) before all results, needs repair
                if next_msg.get('role') == 'user' and not self.is_tool_result_message(next_msg):
                    return True
                
                # Check if it's a tool result for one of our tool_calls
                tc_id = self.get_tool_call_id_from_result(next_msg)
                if tc_id and tc_id in tool_call_ids:
                    found_tool_call_ids.add(tc_id)
                elif not self.is_tool_result_message(next_msg):
                    # Non-tool-result message interrupting the flow
                    return True
            
            # If we didn't find all expected tool results, needs repair
            if len(found_tool_call_ids) < expected_count:
                return True
        
        return False  # No issues found
    
    def remove_orphaned_tool_results(self, messages: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
        """Remove orphaned tool results that have no matching assistant message.
        
        This is a repair function to fix invalid message structures before sending to LLM.
        
        Args:
            messages: List of messages
            
        Returns:
            Messages with orphaned tool results removed
        """
        # First pass: collect all valid tool_call IDs from assistant messages
        valid_tool_call_ids: set = set()
        for msg in messages:
            tool_call_ids = self.get_tool_call_ids_from_message(msg)
            valid_tool_call_ids.update(tool_call_ids)
        
        # Second pass: filter out orphaned tool results
        result = []
        removed_count = 0
        
        for msg in messages:
            if self.is_tool_result_message(msg):
                tool_call_id = self.get_tool_call_id_from_result(msg)
                if tool_call_id and tool_call_id not in valid_tool_call_ids:
                    logger.warning(f"🗑️ Removing orphaned tool result: tool_call_id={tool_call_id}")
                    removed_count += 1
                    continue
            result.append(msg)
        
        if removed_count > 0:
            logger.info(f"🔧 Removed {removed_count} orphaned tool results to fix message structure")
        
        return result
    
    def remove_unanswered_tool_calls(self, messages: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
        """Remove or fix assistant messages with tool_calls that have no matching tool results.
        
        This is a repair function to fix invalid message structures before sending to LLM.
        If an assistant message has ONLY unanswered tool_calls and no content, it's removed.
        If it has content, the tool_calls are removed but content is preserved.
        
        Args:
            messages: List of messages
            
        Returns:
            Messages with unanswered tool_calls fixed
        """
        # First pass: collect all tool_call_ids that have results
        answered_tool_call_ids: set = set()
        for msg in messages:
            if self.is_tool_result_message(msg):
                tool_call_id = self.get_tool_call_id_from_result(msg)
                if tool_call_id:
                    answered_tool_call_ids.add(tool_call_id)
        
        # Second pass: fix assistant messages with unanswered tool_calls
        result = []
        fixed_count = 0
        removed_count = 0
        
        for msg in messages:
            tool_call_ids = self.get_tool_call_ids_from_message(msg)
            
            if tool_call_ids:
                # Check which tool_calls are unanswered
                unanswered = [tc_id for tc_id in tool_call_ids if tc_id not in answered_tool_call_ids]
                
                if unanswered:
                    # Some tool_calls don't have results
                    answered = [tc_id for tc_id in tool_call_ids if tc_id in answered_tool_call_ids]
                    
                    content = msg.get('content', '')
                    has_content = bool(content and str(content).strip())
                    
                    if not answered and not has_content:
                        # All tool_calls are unanswered and no content - remove the message entirely
                        logger.warning(f"🗑️ Removing assistant message with {len(unanswered)} unanswered tool_calls and no content: {unanswered}")
                        removed_count += 1
                        continue
                    elif not answered and has_content:
                        # All tool_calls are unanswered but has content - keep content, remove tool_calls
                        fixed_msg = msg.copy()
                        fixed_msg.pop('tool_calls', None)
                        logger.warning(f"🔧 Removing {len(unanswered)} unanswered tool_calls from assistant message (keeping content): {unanswered}")
                        result.append(fixed_msg)
                        fixed_count += 1
                        continue
                    else:
                        # Some tool_calls are answered - keep only answered ones
                        fixed_msg = msg.copy()
                        original_tool_calls = fixed_msg.get('tool_calls') or []
                        fixed_msg['tool_calls'] = [
                            tc for tc in original_tool_calls 
                            if isinstance(tc, dict) and tc.get('id') in answered_tool_call_ids
                        ] if isinstance(original_tool_calls, list) else []
                        logger.warning(f"🔧 Removed {len(unanswered)} unanswered tool_calls from assistant message (kept {len(answered)}): {unanswered}")
                        result.append(fixed_msg)
                        fixed_count += 1
                        continue
            
            result.append(msg)
        
        if fixed_count > 0 or removed_count > 0:
            logger.info(f"🔧 Fixed {fixed_count} assistant messages, removed {removed_count} to fix unanswered tool_calls")
        
        return result
    
    def repair_tool_call_pairing(self, messages: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
        """Repair both directions of tool call pairing issues.

        This handles:
        1. Orphaned tool results (tool results without matching assistant) - removed
        2. Unanswered tool calls (assistant tool_calls without matching results) - removed/fixed

        Args:
            messages: List of messages

        Returns:
            Messages with all tool call pairing issues fixed
        """
        # Fix orphaned tool results first
        result = self.remove_orphaned_tool_results(messages)

        # Then fix unanswered tool calls
        result = self.remove_unanswered_tool_calls(result)

        # Validate the result
        is_valid, orphaned, unanswered = self.validate_tool_call_pairing(result)

        if not is_valid:
            logger.error(f"🚨 CRITICAL: Could not fully repair message structure. Orphaned: {len(orphaned)}, Unanswered: {len(unanswered)}")
        else:
            logger.info(f"✅ Message structure successfully repaired")

        return result

    def validate_tool_call_ordering(self, messages: List[Dict[str, Any]]) -> tuple[bool, List[str], List[str]]:
        """Validate that tool results immediately follow their assistant tool_calls.

        Some LLM providers (like Minimax) require tool results to appear right after
        the assistant message that made the tool_call, not separated by other messages.

        Args:
            messages: List of messages to validate

        Returns:
            Tuple of (is_valid, out_of_order_tool_call_ids, out_of_order_tool_result_ids)
        """
        out_of_order_tool_call_ids: List[str] = []
        out_of_order_tool_result_ids: List[str] = []

        for i, msg in enumerate(messages):
            if not isinstance(msg, dict):
                continue

            # Get tool_calls from this message
            tool_call_ids = self.get_tool_call_ids_from_message(msg)
            if not tool_call_ids:
                continue

            # Check that the next N messages are tool results for these tool_calls
            expected_count = len(tool_call_ids)
            found_tool_call_ids = set()

            for j in range(i + 1, min(i + 1 + expected_count, len(messages))):
                next_msg = messages[j]
                if not isinstance(next_msg, dict):
                    break

                tc_id = self.get_tool_call_id_from_result(next_msg)
                if tc_id and tc_id in tool_call_ids:
                    found_tool_call_ids.add(tc_id)
                else:
                    break  # Non-matching message found

            # Any tool_calls not found in the immediate next messages are out-of-order
            missing = set(tool_call_ids) - found_tool_call_ids
            if missing:
                out_of_order_tool_call_ids.extend(missing)

        # Now find the tool results for these out-of-order tool_calls
        out_of_order_set = set(out_of_order_tool_call_ids)
        for msg in messages:
            if not isinstance(msg, dict):
                continue
            tc_id = self.get_tool_call_id_from_result(msg)
            if tc_id and tc_id in out_of_order_set:
                out_of_order_tool_result_ids.append(tc_id)

        is_valid = len(out_of_order_tool_call_ids) == 0

        if not is_valid:
            logger.error(f"🚨 ORDERING VALIDATION FAILED: {len(out_of_order_tool_call_ids)} tool_calls have delayed results: {out_of_order_tool_call_ids}")

        return is_valid, out_of_order_tool_call_ids, out_of_order_tool_result_ids

    def remove_out_of_order_tool_pairs(self, messages: List[Dict[str, Any]], out_of_order_ids: List[str]) -> List[Dict[str, Any]]:
        """Remove assistant tool_calls and their delayed tool results that are out of order.

        Args:
            messages: List of messages
            out_of_order_ids: List of tool_call_ids that are out of order

        Returns:
            Messages with out-of-order tool pairs removed/fixed
        """
        if not out_of_order_ids:
            return messages

        out_of_order_set = set(out_of_order_ids)
        result = []
        removed_count = 0

        for msg in messages:
            if not isinstance(msg, dict):
                result.append(msg)
                continue

            # Remove tool results for out-of-order tool_calls
            tc_id = self.get_tool_call_id_from_result(msg)
            if tc_id and tc_id in out_of_order_set:
                logger.warning(f"🗑️ Removing out-of-order tool result: tool_call_id={tc_id}")
                removed_count += 1
                continue

            # Remove tool_calls from assistant messages that are out of order
            tool_call_ids = self.get_tool_call_ids_from_message(msg)
            if tool_call_ids:
                remaining_ids = [tc_id for tc_id in tool_call_ids if tc_id not in out_of_order_set]
                removed_ids = [tc_id for tc_id in tool_call_ids if tc_id in out_of_order_set]

                if removed_ids:
                    logger.warning(f"🗑️ Removing out-of-order tool_calls: {removed_ids}")
                    removed_count += len(removed_ids)

                    if not remaining_ids:
                        # All tool_calls were removed, check if message has content
                        content = msg.get('content')
                        if content and content != []:
                            # Keep the message but remove tool_calls
                            new_msg = {k: v for k, v in msg.items() if k != 'tool_calls'}
                            result.append(new_msg)
                        else:
                            # No content, skip the entire message
                            continue
                    else:
                        # Keep only the remaining tool_calls
                        new_msg = msg.copy()
                        new_tool_calls = [tc for tc in msg.get('tool_calls', []) if tc.get('id') not in out_of_order_set]
                        new_msg['tool_calls'] = new_tool_calls
                        result.append(new_msg)
                    continue

            result.append(msg)

        if removed_count > 0:
            logger.info(f"🔧 Removed {removed_count} out-of-order tool call/result pairs")

        return result

    def strip_all_tool_content_as_fallback(self, messages: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
        """EMERGENCY FALLBACK: Strip all tool-related content if repair completely fails.

        This is a last-resort method that removes:
        1. All tool result messages (role="tool")
        2. All tool_calls from assistant messages
        3. Any message with tool_call_id

        This ensures the LLM can at least continue with a clean message history,
        even if it means losing tool call context.

        Args:
            messages: List of messages

        Returns:
            Messages with all tool content stripped
        """
        logger.warning("🚨 EMERGENCY FALLBACK: Stripping all tool content from messages")

        result = []
        stripped_count = 0

        for msg in messages:
            if not isinstance(msg, dict):
                result.append(msg)
                continue

            # Skip tool result messages entirely
            if msg.get('role') == 'tool' or 'tool_call_id' in msg:
                stripped_count += 1
                continue

            # For assistant messages, remove tool_calls
            if msg.get('role') == 'assistant' and msg.get('tool_calls'):
                clean_msg = msg.copy()
                del clean_msg['tool_calls']
                # Keep the message if it has content
                if clean_msg.get('content'):
                    result.append(clean_msg)
                else:
                    stripped_count += 1
                continue

            # Keep all other messages
            result.append(msg)

        logger.warning(f"🚨 EMERGENCY FALLBACK: Stripped {stripped_count} tool-related messages")
        return result
    
    def remove_old_tool_outputs(
        self, 
        messages: List[Dict[str, Any]], 
        keep_last_n: int = 8
    ) -> List[Dict[str, Any]]:
        """Compress old tool output messages IN-MEMORY, keeping only the most recent N uncompressed.
        
        CRITICAL: This method compresses CONTENT only - it never removes messages.
        This preserves the tool_call_id field and maintains the assistant+tool_result pairing
        required by Bedrock.
        
        This is a pure in-memory operation with no side effects.
        
        Args:
            messages: List of conversation messages
            keep_last_n: Number of most recent tool outputs to preserve uncompressed
            
        Returns:
            Messages with old tool outputs' content replaced by summaries (structure intact)
        """
        if not messages:
            return messages
        
        # First, validate input has valid tool call pairing
        is_valid, orphaned_ids, unanswered_ids = self.validate_tool_call_pairing(messages)
        if not is_valid:
            logger.warning(f"⚠️ Input to remove_old_tool_outputs has pairing issues (orphaned: {len(orphaned_ids)}, unanswered: {len(unanswered_ids)}) - repairing first")
            messages = self.repair_tool_call_pairing(messages)
        
        # Group messages to understand which tool results belong to which assistant messages
        message_groups = self.group_messages_by_tool_calls(messages)
        
        # Flatten to get back to list form for processing
        # But now we understand the structure
        
        # First pass: identify tool result messages and their positions
        tool_result_positions = []
        for i, msg in enumerate(messages):
            if self.is_tool_result_message(msg):
                tool_result_positions.append(i)
        
        total_tool_results = len(tool_result_positions)
        
        if total_tool_results <= keep_last_n:
            # No compression needed
            logger.debug(f"Only {total_tool_results} tool outputs found, keeping all uncompressed (threshold: {keep_last_n})")
            return messages
        
        # Calculate how many to compress (oldest ones)
        num_to_compress = total_tool_results - keep_last_n
        positions_to_compress = set(tool_result_positions[:num_to_compress])
        
        logger.debug(f"Compressing {num_to_compress} old tool outputs in-memory (keeping last {keep_last_n} of {total_tool_results} uncompressed)")
        
        # Second pass: compress old tool outputs' CONTENT (keep structure intact)
        result = []
        for i, msg in enumerate(messages):
            if i in positions_to_compress:
                # Compress content but PRESERVE all other fields (especially tool_call_id!)
                message_id = msg.get('message_id', 'unknown')
                tool_call_id = self.get_tool_call_id_from_result(msg)
                
                summary_content = f"[Tool output compressed for token management] message_id: \"{message_id}\". Use expand-message tool to view full output."
                
                # Copy message and only replace content - preserve tool_call_id, role, name, etc.
                compressed_msg = msg.copy()
                compressed_msg['content'] = summary_content
                result.append(compressed_msg)
                
                logger.debug(f"Compressed tool output at position {i}, tool_call_id={tool_call_id}")
            else:
                result.append(msg)
        
        # Final validation - structure should still be valid
        is_valid_after, orphaned_after, unanswered_after = self.validate_tool_call_pairing(result)
        if not is_valid_after:
            logger.error(f"🚨 BUG: remove_old_tool_outputs broke tool call pairing! Orphaned: {orphaned_after}, Unanswered: {unanswered_after}")
        
        return result
    
    def compress_user_messages_in_memory(
        self,
        messages: List[Dict[str, Any]],
        keep_last_n: int = 10
    ) -> List[Dict[str, Any]]:
        """Compress user messages IN-MEMORY, keeping only the most recent N uncompressed.
        
        Args:
            messages: List of conversation messages
            keep_last_n: Number of most recent user messages to preserve
            
        Returns:
            Messages with old user messages compressed
        """
        if not messages:
            return messages
        
        # Find user message positions
        user_positions = []
        for i, msg in enumerate(messages):
            if isinstance(msg, dict) and msg.get('role') == 'user':
                user_positions.append(i)
        
        total_user_messages = len(user_positions)
        
        if total_user_messages <= keep_last_n:
            return messages
        
        # Positions to compress (all except last N)
        num_to_compress = total_user_messages - keep_last_n
        positions_to_compress = user_positions[:num_to_compress]
        
        logger.debug(f"Compressing {num_to_compress} user messages in-memory (keeping last {keep_last_n})")
        
        # Compress old user messages
        result = []
        for i, msg in enumerate(messages):
            if i in positions_to_compress:
                original_content = msg.get('content', '')
                if isinstance(original_content, str) and len(original_content) > 3000:
                    summary = original_content[:3000] + "... (truncated)"
                    compressed_msg = msg.copy()
                    compressed_msg['content'] = summary
                    result.append(compressed_msg)
                else:
                    result.append(msg)
            else:
                result.append(msg)
        
        return result
    
    def compress_assistant_messages_in_memory(
        self,
        messages: List[Dict[str, Any]],
        keep_last_n: int = 10
    ) -> List[Dict[str, Any]]:
        """Compress assistant messages IN-MEMORY, keeping only the most recent N uncompressed.
        
        Args:
            messages: List of conversation messages
            keep_last_n: Number of most recent assistant messages to preserve
            
        Returns:
            Messages with old assistant messages compressed
        """
        if not messages:
            return messages
        
        # Find assistant message positions
        assistant_positions = []
        for i, msg in enumerate(messages):
            if isinstance(msg, dict) and msg.get('role') == 'assistant':
                assistant_positions.append(i)
        
        total_assistant_messages = len(assistant_positions)
        
        if total_assistant_messages <= keep_last_n:
            return messages
        
        # Positions to compress (all except last N)
        num_to_compress = total_assistant_messages - keep_last_n
        positions_to_compress = assistant_positions[:num_to_compress]
        
        logger.debug(f"Compressing {num_to_compress} assistant messages in-memory (keeping last {keep_last_n})")
        
        # Compress old assistant messages
        result = []
        for i, msg in enumerate(messages):
            if i in positions_to_compress:
                original_content = msg.get('content', '')
                if isinstance(original_content, str) and len(original_content) > 3000:
                    summary = original_content[:3000] + "... (truncated)"
                    compressed_msg = msg.copy()
                    compressed_msg['content'] = summary
                    result.append(compressed_msg)
                else:
                    result.append(msg)
            else:
                result.append(msg)
        
        return result
    
    def compress_message(self, msg_content: Union[str, dict], message_id: Optional[str] = None, max_length: int = 3000) -> Union[str, dict]:
        """Compress the message content."""
        if isinstance(msg_content, str):
            if len(msg_content) > max_length:
                return msg_content[:max_length] + "... (truncated)" + f"\n\nmessage_id \"{message_id}\"\nUse expand-message tool to see contents"
            else:
                return msg_content
        
    def safe_truncate(self, msg_content: Union[str, dict], max_length: int = 100000) -> Union[str, dict]:
        """Truncate the message content safely by removing the middle portion."""
        max_length = min(max_length, 100000)
        if isinstance(msg_content, str):
            if len(msg_content) > max_length:
                # Calculate how much to keep from start and end
                keep_length = max_length - 150  # Reserve space for truncation message
                start_length = keep_length // 2
                end_length = keep_length - start_length
                
                start_part = msg_content[:start_length]
                end_part = msg_content[-end_length:] if end_length > 0 else ""
                
                return start_part + f"\n\n... (middle truncated) ...\n\n" + end_part + f"\n\nThis message is too long, repeat relevant information in your response to remember it"
            else:
                return msg_content
        elif isinstance(msg_content, dict):
            json_str = json.dumps(msg_content)
            if len(json_str) > max_length:
                # Calculate how much to keep from start and end
                keep_length = max_length - 150  # Reserve space for truncation message
                start_length = keep_length // 2
                end_length = keep_length - start_length
                
                start_part = json_str[:start_length]
                end_part = json_str[-end_length:] if end_length > 0 else ""
                
                return start_part + f"\n\n... (middle truncated) ...\n\n" + end_part + f"\n\nThis message is too long, repeat relevant information in your response to remember it"
            else:
                return msg_content
  
    async def compress_tool_result_messages(self, messages: List[Dict[str, Any]], llm_model: str, max_tokens: Optional[int], token_threshold: int = 1000, uncompressed_total_token_count: Optional[int] = None) -> List[Dict[str, Any]]:
        """Compress the tool result messages except the most recent N (configured by keep_recent_tool_outputs).
        
        Compression is deterministic (simple truncation), ensuring consistent results across requests.
        This allows prompt caching (applied later) to produce cache hits on identical compressed content.
        """
        if uncompressed_total_token_count is None:
            uncompressed_total_token_count = await self.count_tokens(llm_model, messages)

        max_tokens_value = max_tokens or (100 * 1000)

        if uncompressed_total_token_count > max_tokens_value:
            _i = 0  # Count the number of ToolResult messages
            for msg in reversed(messages):  # Start from the end and work backwards
                if not isinstance(msg, dict):
                    continue  # Skip non-dict messages
                if self.is_tool_result_message(msg):  # Only compress ToolResult messages
                    _i += 1  # Count the number of ToolResult messages
                    msg_token_count = token_counter(messages=[msg])  # Count the number of tokens in the message
                    if msg_token_count > token_threshold:  # If the message is too long
                        if _i > self.keep_recent_tool_outputs:  # If this is not one of the most recent N ToolResult messages
                            message_id = msg.get('message_id')  # Get the message_id
                            if message_id:
                                msg["content"] = self.compress_message(msg["content"], message_id, token_threshold * 3)
                            else:
                                logger.warning(f"UNEXPECTED: Message has no message_id {str(msg)[:100]}")
                        else:
                            msg["content"] = self.safe_truncate(msg["content"], int(max_tokens_value * 2))
        return messages

    async def compress_user_messages(self, messages: List[Dict[str, Any]], llm_model: str, max_tokens: Optional[int], token_threshold: int = 1000, uncompressed_total_token_count: Optional[int] = None) -> List[Dict[str, Any]]:
        """Compress the user messages except the most recent one.
        
        Compression is deterministic (simple truncation), ensuring consistent results across requests.
        This allows prompt caching (applied later) to produce cache hits on identical compressed content.
        """
        if uncompressed_total_token_count is None:
            uncompressed_total_token_count = await self.count_tokens(llm_model, messages)

        max_tokens_value = max_tokens or (100 * 1000)

        if uncompressed_total_token_count > max_tokens_value:
            _i = 0  # Count the number of User messages
            for msg in reversed(messages):  # Start from the end and work backwards
                if not isinstance(msg, dict):
                    continue  # Skip non-dict messages
                if msg.get('role') == 'user':  # Only compress User messages
                    _i += 1  # Count the number of User messages
                    msg_token_count = token_counter(messages=[msg])  # Count the number of tokens in the message
                    if msg_token_count > token_threshold:  # If the message is too long
                        if _i > self.keep_recent_user_messages:  # If this is not one of the most recent N User messages
                            message_id = msg.get('message_id')  # Get the message_id
                            if message_id:
                                # Secondary compression - just placeholder
                                msg["content"] = f"[Content compressed]\n\nmessage_id \"{message_id}\"\nUse expand-message tool to see contents"
                            else:
                                logger.warning(f"UNEXPECTED: Message has no message_id {str(msg)[:100]}")
                        else:
                            msg["content"] = self.safe_truncate(msg["content"], int(max_tokens_value * 2))
        return messages

    async def compress_assistant_messages(self, messages: List[Dict[str, Any]], llm_model: str, max_tokens: Optional[int], token_threshold: int = 1000, uncompressed_total_token_count: Optional[int] = None) -> List[Dict[str, Any]]:
        """Compress the assistant messages except the most recent one.
        
        Compression is deterministic (simple truncation), ensuring consistent results across requests.
        This allows prompt caching (applied later) to produce cache hits on identical compressed content.
        """
        if uncompressed_total_token_count is None:
            uncompressed_total_token_count = await self.count_tokens(llm_model, messages)

        max_tokens_value = max_tokens or (100 * 1000)
        
        if uncompressed_total_token_count > max_tokens_value:
            _i = 0  # Count the number of Assistant messages
            for msg in reversed(messages):  # Start from the end and work backwards
                if not isinstance(msg, dict):
                    continue  # Skip non-dict messages
                if msg.get('role') == 'assistant':  # Only compress Assistant messages
                    _i += 1  # Count the number of Assistant messages
                    msg_token_count = token_counter(messages=[msg])  # Count the number of tokens in the message
                    if msg_token_count > token_threshold:  # If the message is too long
                        if _i > self.keep_recent_assistant_messages:  # If this is not one of the most recent N Assistant messages
                            message_id = msg.get('message_id')  # Get the message_id
                            if message_id:
                                # Secondary compression - just placeholder
                                msg["content"] = f"[Content compressed]\n\nmessage_id \"{message_id}\"\nUse expand-message tool to see contents"
                            else:
                                logger.warning(f"UNEXPECTED: Message has no message_id {str(msg)[:100]}")
                        else:
                            msg["content"] = self.safe_truncate(msg["content"], int(max_tokens_value * 2))
                            
        return messages

    def remove_meta_messages(self, messages: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
        """Remove meta messages from the messages."""
        result: List[Dict[str, Any]] = []
        for msg in messages:
            msg_content = msg.get('content')
            # Try to parse msg_content as JSON if it's a string
            if isinstance(msg_content, str):
                try: 
                    msg_content = json.loads(msg_content)
                except json.JSONDecodeError: 
                    pass
            if isinstance(msg_content, dict):
                # Create a copy to avoid modifying the original
                msg_content_copy = msg_content.copy()
                if "tool_execution" in msg_content_copy:
                    tool_execution = msg_content_copy["tool_execution"].copy()
                    if "arguments" in tool_execution:
                        del tool_execution["arguments"]
                    msg_content_copy["tool_execution"] = tool_execution
                # Create a new message dict with the modified content
                new_msg = msg.copy()
                new_msg["content"] = json.dumps(msg_content_copy)
                result.append(new_msg)
            else:
                result.append(msg)
        return result

    async def compress_messages(self, messages: List[Dict[str, Any]], llm_model: str, max_tokens: Optional[int] = 41000, token_threshold: int = 4096, max_iterations: int = 5, actual_total_tokens: Optional[int] = None, system_prompt: Optional[Dict[str, Any]] = None, thread_id: Optional[str] = None) -> List[Dict[str, Any]]:
        """Compress the messages WITHOUT applying caching during iterations.
        
        Caching should be applied ONCE at the end by the caller, not during compression.
        Compressed messages are saved to the database for future reads.
        """
        # Capture original content for messages with message_id (for tracking compression)
        original_content_map: Dict[str, Any] = {}
        for msg in messages:
            message_id = msg.get('message_id')
            if message_id:
                original_content_map[message_id] = msg.get('content')
        
        # Get model-specific token limits from constants
        context_window = model_manager.get_context_window(llm_model)
        
        # Reserve tokens for output generation and safety margin
        if context_window >= 1_000_000:  # Very large context models (Gemini)
            max_tokens = context_window - 300_000  # Large safety margin for huge contexts
        elif context_window >= 400_000:  # Large context models (GPT-5)
            max_tokens = context_window - 64_000  # Reserve for output + margin
        elif context_window >= 200_000:  # Medium context models (Claude Sonnet)
            max_tokens = context_window - 32_000  # Reserve for output + margin
        elif context_window >= 100_000:  # Standard large context models
            max_tokens = context_window - 16_000  # Reserve for output + margin
        else:  # Smaller context models
            max_tokens = context_window - 8_000   # Reserve for output + margin
        
        # logger.debug(f"Model {llm_model}: context_window={context_window}, effective_limit={max_tokens}")

        result = messages
        result = self.remove_meta_messages(result)

        # Calculate initial token count with caching to match API reality
        if actual_total_tokens is not None:
            uncompressed_total_token_count = actual_total_tokens
        else:
            # Count conversation + system prompt WITH caching (to match API reality)
            uncompressed_total_token_count = await self.count_tokens(llm_model, result, system_prompt, apply_caching=True)
            logger.info(f"Initial token count (with caching): {uncompressed_total_token_count}")

        # Calculate target tokens (hysteresis: compress to 60% of max to avoid repeated compressions)
        target_tokens = int(max_tokens * self.compression_target_ratio)
        logger.info(f"Compression threshold: {max_tokens}, target: {target_tokens} (ratio: {self.compression_target_ratio})")
        
        # Check if we're already under threshold - no compression needed!
        if uncompressed_total_token_count <= max_tokens:
            logger.info(f"✅ Token count ({uncompressed_total_token_count}) under threshold ({max_tokens}), skipping compression")
            return await self.middle_out_messages(result)
        
        # PRIMARY STRATEGY: Remove old tool outputs if over threshold
        if uncompressed_total_token_count > max_tokens:
            logger.info(f"Context over limit ({uncompressed_total_token_count} > {max_tokens}), starting tiered compression...")
            
            # Tier 1: Compress old tool outputs in-memory
            result = self.remove_old_tool_outputs(result, keep_last_n=self.keep_recent_tool_outputs)
            
            # Recalculate WITH caching
            current_token_count = await self.count_tokens(llm_model, result, system_prompt, apply_caching=True)
            
            logger.info(f"After tool compression: {uncompressed_total_token_count} -> {current_token_count} tokens")
            
            # Tier 2: Compress user messages if still above target
            if current_token_count > target_tokens:
                logger.info(f"Still above target ({current_token_count} > {target_tokens}), compressing user messages...")
                
                # Compress in-memory for this request
                result = self.compress_user_messages_in_memory(result, keep_last_n=self.keep_recent_user_messages)
                
                # Recalculate with in-memory compressed messages WITH caching
                current_token_count = await self.count_tokens(llm_model, result, system_prompt, apply_caching=True)
                logger.info(f"After user compression: {current_token_count} tokens")
            
            # Tier 3: Compress assistant messages if still above target
            if current_token_count > target_tokens:
                logger.info(f"Still above target ({current_token_count} > {target_tokens}), compressing assistant messages...")
                
                # Compress in-memory for this request
                result = self.compress_assistant_messages_in_memory(result, keep_last_n=self.keep_recent_assistant_messages)
                
                # Recalculate with in-memory compressed messages WITH caching
                current_token_count = await self.count_tokens(llm_model, result, system_prompt, apply_caching=True)
                logger.info(f"After assistant compression: {current_token_count} tokens")
            
            logger.info(f"Tiered compression complete: {uncompressed_total_token_count} -> {current_token_count} tokens (target: {target_tokens})")
            uncompressed_total_token_count = current_token_count

        # SECONDARY STRATEGY: Apply compression to remaining messages if still above target
        # Use target_tokens as threshold to ensure we reach hysteresis goal
        if uncompressed_total_token_count > target_tokens:
            logger.info(f"Applying secondary compression to reach target ({uncompressed_total_token_count} > {target_tokens})")
            # Use lower token_threshold (500) to compress more messages, including smaller ones
            aggressive_threshold = 500  
            result = await self.compress_tool_result_messages(result, llm_model, target_tokens, aggressive_threshold, uncompressed_total_token_count)
            result = await self.compress_user_messages(result, llm_model, target_tokens, aggressive_threshold, uncompressed_total_token_count)
            result = await self.compress_assistant_messages(result, llm_model, target_tokens, aggressive_threshold, uncompressed_total_token_count)
        else:
            # Still run with original max_tokens in case there's any remaining content to compress
            result = await self.compress_tool_result_messages(result, llm_model, max_tokens, token_threshold, uncompressed_total_token_count)
            result = await self.compress_user_messages(result, llm_model, max_tokens, token_threshold, uncompressed_total_token_count)
            result = await self.compress_assistant_messages(result, llm_model, max_tokens, token_threshold, uncompressed_total_token_count)

        # Recalculate WITH caching (to match API reality)
        compressed_total = await self.count_tokens(llm_model, result, system_prompt, apply_caching=True)
        
        if compressed_total != uncompressed_total_token_count:
            logger.info(f"Context compression: {uncompressed_total_token_count} -> {compressed_total} tokens (saved {uncompressed_total_token_count - compressed_total})")
        else:
            logger.info(f"Context compression: {compressed_total} tokens (no compression needed, under threshold)")

        # Recurse if still too large
        if max_iterations <= 0:
            logger.warning(f"Max iterations reached, omitting messages to target ({target_tokens})")
            result = await self.compress_messages_by_omitting_messages(result, llm_model, target_tokens, system_prompt=system_prompt)
            compressed_total = await self.count_tokens(llm_model, result, system_prompt, apply_caching=True)
            # Fall through to last_usage update
        elif compressed_total > max_tokens:
            logger.warning(f"Further compression needed: {compressed_total} > {max_tokens}")
            # Recursive call - will handle its own last_usage update
            return await self.compress_messages(
                result, llm_model, max_tokens, 
                token_threshold // 2, max_iterations - 1, 
                compressed_total, system_prompt, thread_id=thread_id
            )
        elif compressed_total > target_tokens:
            # Still over target but under max_tokens - use omit_messages to reach target
            logger.info(f"Secondary compression didn't reach target ({compressed_total} > {target_tokens}). Using message omission to reach target.")
            result = await self.compress_messages_by_omitting_messages(result, llm_model, target_tokens, system_prompt=system_prompt)
            compressed_total = await self.count_tokens(llm_model, result, system_prompt, apply_caching=True)
            logger.info(f"After message omission to target: {compressed_total} tokens")

        logger.info(f"✨ Final compression complete: {compressed_total} tokens (target: {target_tokens}, max: {max_tokens})")
        
        # Identify messages that were compressed and save to database
        compressed_to_save: List[Dict[str, Any]] = []
        for msg in result:
            message_id = msg.get('message_id')
            if message_id and message_id in original_content_map:
                original_content = original_content_map[message_id]
                current_content = msg.get('content')
                
                # If content changed, it was compressed
                if current_content != original_content:
                    # Save the FULL message structure (matching DB content format)
                    # Remove message_id from the saved content as it's stored separately
                    msg_to_save = {k: v for k, v in msg.items() if k != 'message_id'}
                    compressed_to_save.append({
                        'message_id': message_id,
                        'compressed_content': json.dumps(msg_to_save)
                    })
        
        # Save compressed messages to database (fire and forget for performance)
        if compressed_to_save:
            try:
                await self.save_compressed_messages(compressed_to_save)
            except Exception as e:
                logger.warning(f"Failed to save compressed messages: {e}")
        
        return await self.middle_out_messages(result)
    
    async def compress_messages_by_omitting_messages(
            self, 
            messages: List[Dict[str, Any]], 
            llm_model: str, 
            max_tokens: Optional[int] = 41000,
            removal_batch_size: int = 3,  # Now operates on groups, not individual messages
            min_groups_to_keep: int = 5,  # Minimum number of groups to preserve
            system_prompt: Optional[Dict[str, Any]] = None
        ) -> List[Dict[str, Any]]:
        """Compress the messages by omitting message GROUPS from the middle.
        
        CRITICAL: This method operates on atomic message groups to preserve
        the assistant+tool_calls / tool_result pairing required by Bedrock.
        
        Args:
            messages: List of messages to compress
            llm_model: Model name for token counting
            max_tokens: Maximum allowed tokens
            removal_batch_size: Number of groups to remove per iteration
            min_groups_to_keep: Minimum number of groups to preserve
            system_prompt: Optional system prompt for token counting
        """
        if not messages:
            return messages
            
        result = messages
        result = self.remove_meta_messages(result)

        # Early exit if no compression needed - WITH caching
        initial_token_count = await self.count_tokens(llm_model, result, system_prompt, apply_caching=True)
        
        max_allowed_tokens = max_tokens or (100 * 1000)
        
        if initial_token_count <= max_allowed_tokens:
            return result

        # Group messages into atomic units (assistant+tool_calls grouped with their tool results)
        message_groups = self.group_messages_by_tool_calls(result)
        logger.info(f"📦 Grouped {len(result)} messages into {len(message_groups)} atomic groups for compression")
        
        system_message = system_prompt
        safety_limit = 500
        current_token_count = initial_token_count
        
        # Track all omitted messages to save them with placeholder content
        all_omitted_messages: List[Dict[str, Any]] = []
        
        while current_token_count > max_allowed_tokens and safety_limit > 0:
            safety_limit -= 1
            
            if len(message_groups) <= min_groups_to_keep:
                logger.warning(f"Cannot compress further: only {len(message_groups)} groups remain (min: {min_groups_to_keep})")
                break

            # Calculate removal strategy based on current group count
            if len(message_groups) > (removal_batch_size * 2):
                # Remove from middle, keeping recent and early context
                middle_start = len(message_groups) // 2 - (removal_batch_size // 2)
                middle_end = middle_start + removal_batch_size
                
                # Log what we're removing
                removed_groups = message_groups[middle_start:middle_end]
                removed_msg_count = sum(len(g) for g in removed_groups)
                logger.debug(f"Removing {len(removed_groups)} groups ({removed_msg_count} messages) from middle")
                
                # Track omitted messages
                for group in removed_groups:
                    all_omitted_messages.extend(group)
                
                message_groups = message_groups[:middle_start] + message_groups[middle_end:]
            else:
                # Remove from earlier groups, preserving recent context
                groups_to_remove = min(removal_batch_size, len(message_groups) // 2)
                if groups_to_remove > 0:
                    removed_groups = message_groups[:groups_to_remove]
                    removed_msg_count = sum(len(g) for g in removed_groups)
                    logger.debug(f"Removing {groups_to_remove} early groups ({removed_msg_count} messages)")
                    
                    # Track omitted messages
                    for group in removed_groups:
                        all_omitted_messages.extend(group)
                    
                    message_groups = message_groups[groups_to_remove:]
                else:
                    # Can't remove any more groups
                    break

            # Flatten groups back to messages for token counting
            conversation_messages = self.flatten_message_groups(message_groups)
            
            # Recalculate token count WITH caching
            current_token_count = await self.count_tokens(llm_model, conversation_messages, system_message, apply_caching=True)

        # Save omitted messages with placeholder content so they're read as compressed next time
        if all_omitted_messages:
            omitted_to_save: List[Dict[str, Any]] = []
            for msg in all_omitted_messages:
                message_id = msg.get('message_id')
                if message_id:
                    # Create placeholder content preserving message structure
                    placeholder_msg = {
                        'role': msg.get('role', 'user'),
                        'content': f"[Message omitted for context management. message_id: \"{message_id}\". Use expand-message tool to view full content.]"
                    }
                    # Preserve tool_call_id for tool messages
                    if msg.get('tool_call_id'):
                        placeholder_msg['tool_call_id'] = msg['tool_call_id']
                    # Preserve tool_calls for assistant messages (critical for pairing validation)
                    if msg.get('tool_calls'):
                        placeholder_msg['tool_calls'] = msg['tool_calls']
                    
                    omitted_to_save.append({
                        'message_id': message_id,
                        'compressed_content': json.dumps(placeholder_msg),
                        'is_omission': True  # Mark as omission so it can override compression
                    })
            
            if omitted_to_save:
                try:
                    await self.save_compressed_messages(omitted_to_save)
                    
                    # Also mark any tool results belonging to omitted assistant messages
                    # This handles the case where tool results were compressed separately
                    omitted_tool_call_ids = []
                    for msg in all_omitted_messages:
                        if msg.get('tool_calls'):
                            for tc in msg['tool_calls']:
                                tc_id = tc.get('id')
                                if tc_id:
                                    omitted_tool_call_ids.append(tc_id)
                    
                    if omitted_tool_call_ids and self.thread_id:
                        from core.threads import repo as threads_repo
                        marked_count = await threads_repo.mark_tool_results_as_omitted(
                            self.thread_id, 
                            omitted_tool_call_ids
                        )
                        if marked_count > 0:
                            logger.info(f"📝 Also marked {marked_count} orphaned tool results as omitted")
                except Exception as e:
                    logger.warning(f"Failed to save omitted messages: {e}")

        # Flatten final groups to messages
        final_messages = self.flatten_message_groups(message_groups)
        
        # Validate tool call pairing is intact
        is_valid, orphaned_ids, unanswered_ids = self.validate_tool_call_pairing(final_messages)
        if not is_valid:
            logger.warning(f"⚠️ Post-compression validation found pairing issues (orphaned: {len(orphaned_ids)}, unanswered: {len(unanswered_ids)}) - repairing")
            final_messages = self.repair_tool_call_pairing(final_messages)
        
        # Log with system prompt included for accurate token reporting WITH caching
        final_token_count = await self.count_tokens(llm_model, final_messages, system_message, apply_caching=True)
        
        logger.info(f"Context compression (omit): {initial_token_count} -> {final_token_count} tokens ({len(messages)} -> {len(final_messages)} messages, {len(message_groups)} groups)")
            
        return final_messages
    
    async def middle_out_messages(self, messages: List[Dict[str, Any]], max_messages: int = 320) -> List[Dict[str, Any]]:
        """Remove message GROUPS from the middle of the list, keeping approximately max_messages total.
        
        CRITICAL: This method operates on atomic message groups to preserve
        the assistant+tool_calls / tool_result pairing required by Bedrock.
        
        Args:
            messages: List of messages
            max_messages: Approximate maximum messages to keep (actual may vary due to group sizes)
            
        Returns:
            Messages with middle groups removed, preserving tool call pairing
        """
        if len(messages) <= max_messages:
            return messages
        
        # Group messages into atomic units
        message_groups = self.group_messages_by_tool_calls(messages)
        
        # If already few enough groups, return as-is
        total_messages = sum(len(g) for g in message_groups)
        if total_messages <= max_messages:
            return messages
        
        # Estimate how many groups we need to keep
        # Use average group size to estimate
        avg_group_size = total_messages / len(message_groups) if message_groups else 1
        target_groups = int(max_messages / avg_group_size)
        
        # Ensure we keep at least 4 groups (2 from start, 2 from end)
        target_groups = max(4, target_groups)
        
        if len(message_groups) <= target_groups:
            return messages
        
        # Keep half from the beginning and half from the end (by groups)
        keep_start_groups = target_groups // 2
        keep_end_groups = target_groups - keep_start_groups
        
        # Ensure we keep at least 1 group from each end
        keep_start_groups = max(1, keep_start_groups)
        keep_end_groups = max(1, keep_end_groups)
        
        # Build the result by keeping start and end groups
        kept_groups = message_groups[:keep_start_groups] + message_groups[-keep_end_groups:]
        
        # Track removed groups for saving as compressed
        removed_groups = message_groups[keep_start_groups:-keep_end_groups] if keep_end_groups > 0 else message_groups[keep_start_groups:]
        
        removed_count = len(message_groups) - len(kept_groups)
        if removed_count > 0:
            logger.info(f"📦 Middle-out: removed {removed_count} groups from middle ({len(message_groups)} -> {len(kept_groups)} groups)")
            
            # Save removed messages with placeholder content
            omitted_to_save: List[Dict[str, Any]] = []
            for group in removed_groups:
                for msg in group:
                    message_id = msg.get('message_id')
                    if message_id:
                        placeholder_msg = {
                            'role': msg.get('role', 'user'),
                            'content': f"[Message omitted for context management. message_id: \"{message_id}\". Use expand-message tool to view full content.]"
                        }
                        if msg.get('tool_call_id'):
                            placeholder_msg['tool_call_id'] = msg['tool_call_id']
                        # Preserve tool_calls for assistant messages (critical for pairing validation)
                        if msg.get('tool_calls'):
                            placeholder_msg['tool_calls'] = msg['tool_calls']
                        
                        omitted_to_save.append({
                            'message_id': message_id,
                            'compressed_content': json.dumps(placeholder_msg),
                            'is_omission': True  # Mark as omission so it can override compression
                        })
            
            if omitted_to_save:
                try:
                    await self.save_compressed_messages(omitted_to_save)
                    
                    # Also mark any tool results belonging to omitted assistant messages
                    omitted_tool_call_ids = []
                    for group in removed_groups:
                        for msg in group:
                            if msg.get('tool_calls'):
                                for tc in msg['tool_calls']:
                                    tc_id = tc.get('id')
                                    if tc_id:
                                        omitted_tool_call_ids.append(tc_id)
                    
                    if omitted_tool_call_ids and self.thread_id:
                        from core.threads import repo as threads_repo
                        marked_count = await threads_repo.mark_tool_results_as_omitted(
                            self.thread_id, 
                            omitted_tool_call_ids
                        )
                        if marked_count > 0:
                            logger.info(f"📝 Also marked {marked_count} orphaned tool results as omitted")
                except Exception as e:
                    logger.warning(f"Failed to save middle-out omitted messages: {e}")
        
        # Flatten groups back to messages
        result = self.flatten_message_groups(kept_groups)
        
        # Validate tool call pairing is intact
        is_valid, orphaned_ids, unanswered_ids = self.validate_tool_call_pairing(result)
        if not is_valid:
            logger.warning(f"⚠️ Middle-out validation found pairing issues (orphaned: {len(orphaned_ids)}, unanswered: {len(unanswered_ids)}) - repairing")
            result = self.repair_tool_call_pairing(result)
        
        return result