SamJoshua commited on
Commit
5979b3e
·
verified ·
1 Parent(s): a960ebc

Fix chat_template crash when tool-call arguments are an OpenAI-style JSON string

Browse files

`chat_template.jinja` crashes when an assistant tool call's `arguments` are a JSON string (the OpenAI Chat Completions shape that many harnesses, traces, and datasets replay into `apply_chat_template`). Dict arguments already render correctly.

## Repro (`transformers` 5.22.2, tokenizer only)

```python
from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("tencent/Hy3")
tools = [{"type": "function", "function": {
"name": "get_weather", "description": "Get weather",
"parameters": {"type": "object",
"properties": {"city": {"type": "string"}, "metric": {"type": "boolean"}},
"required": ["city"]}}}]

def history(args):
return [
{"role": "user", "content": "Weather in Paris?"},
{"role": "assistant", "content": "", "tool_calls": [
{"id": "c1", "type": "function",
"function": {"name": "get_weather", "arguments": args}}]},
{"role": "tool", "tool_call_id": "c1", "name": "get_weather", "content": '{"temp": 21}'},
]

tok.apply_chat_template(history({"city": "Paris", "metric": True}), tools=tools, tokenize=False)
# OK:
# <tool_call:opensource>get_weather<tool_sep:opensource>
# <arg_key:opensource>city</arg_key:opensource>
# <arg_value:opensource>Paris</arg_value:opensource>
# ...

tok.apply_chat_template(history('{"city": "Paris", "metric": true}'), tools=tools, tokenize=False)
# UndefinedError: 'str object' has no attribute 'items'
```

The template does `{% set arguments = tool['function']['arguments'] %}` then `{% for key, value in arguments.items() %}` with no mapping/string branch. I checked discussions on this repo and did not find an existing report of this crash.

## What changed

Before the existing `<arg_key>` / `<arg_value>` loop, string `arguments` are parsed into a mapping so the loop still emits the trained tags. A raw-string dump would skip those tags and break tool-call replay. `transformers` 5.22.2's chat-template environment has `tojson` but no `fromjson` filter, so the parse lives in the template (dict arguments pass through unchanged).

## Tests

On current `main` (`a960ebc3da`), with this template substituted:

| Case | Current | This PR |
|---|---|---|
| Dict `arguments` (`{"city": "Paris", "metric": True}`) | renders | unchanged (byte-identical) |
| OpenAI-style string `arguments` (`'{"city": "Paris", "metric": true}'`) | `UndefinedError: 'str object' has no attribute 'items'` | renders, byte-identical to the dict case |
| Empty dict / empty object string (`{}` / `'{}'`) | dict OK; string crashes | both render, identical |
| Nested dict / nested JSON string | dict OK; string crashes | both render, identical |
| Unicode dict / unicode JSON string (`París`, `東京の天気`) | dict OK; string crashes | both render, identical |
| Multiple dict tool calls | renders | unchanged (byte-identical) |

This does not change tool-name escaping.

Verified with `transformers` 5.22.2 via `tokenizer.apply_chat_template` (no weights).

Files changed (1) hide show
  1. chat_template.jinja +174 -1
chat_template.jinja CHANGED
@@ -1,3 +1,176 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  {#- ----------‑‑‑ special token variables ‑‑‑---------- -#}
2
  {%- set HYTK = ':opensource' %}
3
  {%- set eos_token = '<|hy_eos{}|>'.format(HYTK) %}
@@ -170,7 +343,7 @@
170
  {{- content }}
171
  {{- toolcalls_begin_token + '\n' }}
172
  {%- for tool in message['tool_calls'] %}
173
- {%- set arguments = tool['function']['arguments'] %}
174
  {{- toolcall_begin_token + tool['function']['name'] + toolsep_token + '\n' }}
175
  {%- for key, value in arguments.items() %}
176
  {{- argkey_begin_token + key + argkey_end_token + '\n' }}
 
1
+ {#- Portable JSON-object parse for tool_call.arguments. The transformers chat-template environment ships tojson but not fromjson, so OpenAI-style JSON string arguments crash on .items(). A raw-string dump would also skip the existing argument tags. Dict arguments pass through unchanged. -#}{%- macro _json_skip(ns) -%}
2
+ {%- for _ in range((ns.s|length) - ns.i) -%}
3
+ {%- if ns.i < (ns.s|length) and ns.s[ns.i] in ' \t\n\r' -%}
4
+ {%- set ns.i = ns.i + 1 -%}
5
+ {%- else -%}{%- break -%}{%- endif -%}
6
+ {%- endfor -%}
7
+ {%- endmacro -%}
8
+ {%- macro _json_string(ns) -%}
9
+ {%- set HEX = dict([('0',0),('1',1),('2',2),('3',3),('4',4),('5',5),('6',6),('7',7),('8',8),('9',9),('a',10),('b',11),('c',12),('d',13),('e',14),('f',15),('A',10),('B',11),('C',12),('D',13),('E',14),('F',15)]) -%}
10
+ {%- set ESC = dict([('"', '"'), ('\\', '\\'), ('/', '/'), ('b', '\b'), ('f', '\f'), ('n', '\n'), ('r', '\r'), ('t', '\t')]) -%}
11
+ {%- set ns.i = ns.i + 1 -%}
12
+ {%- set ns.buf = '' -%}
13
+ {%- for _ in range((ns.s|length) - ns.i) -%}
14
+ {%- if ns.i >= (ns.s|length) -%}{{- raise_exception("tool_call.arguments: unterminated JSON string") -}}{%- endif -%}
15
+ {%- set c = ns.s[ns.i] -%}
16
+ {%- if c == '"' -%}
17
+ {%- set ns.i = ns.i + 1 -%}
18
+ {%- break -%}
19
+ {%- elif c == '\\' -%}
20
+ {%- set ns.i = ns.i + 1 -%}
21
+ {%- if ns.i >= (ns.s|length) -%}{{- raise_exception("tool_call.arguments: unterminated JSON escape") -}}{%- endif -%}
22
+ {%- set e = ns.s[ns.i] -%}
23
+ {%- if e == 'u' -%}
24
+ {%- set ns.i = ns.i + 1 -%}
25
+ {%- if ns.i + 4 > (ns.s|length) -%}{{- raise_exception("tool_call.arguments: truncated \\u escape") -}}{%- endif -%}
26
+ {%- set h0 = ns.s[ns.i] -%}{%- set h1 = ns.s[ns.i+1] -%}{%- set h2 = ns.s[ns.i+2] -%}{%- set h3 = ns.s[ns.i+3] -%}
27
+ {%- if h0 not in HEX or h1 not in HEX or h2 not in HEX or h3 not in HEX -%}{{- raise_exception("tool_call.arguments: invalid \\u escape") -}}{%- endif -%}
28
+ {%- set n = ((HEX[h0]*16 + HEX[h1])*16 + HEX[h2])*16 + HEX[h3] -%}
29
+ {%- set ns.buf = ns.buf + ("%c"|format(n)) -%}
30
+ {%- set ns.i = ns.i + 4 -%}
31
+ {%- elif e in ESC -%}
32
+ {%- set ns.buf = ns.buf + ESC[e] -%}
33
+ {%- set ns.i = ns.i + 1 -%}
34
+ {%- else -%}
35
+ {{- raise_exception("tool_call.arguments: invalid JSON escape") -}}
36
+ {%- endif -%}
37
+ {%- else -%}
38
+ {%- set ns.buf = ns.buf + c -%}
39
+ {%- set ns.i = ns.i + 1 -%}
40
+ {%- endif -%}
41
+ {%- endfor -%}
42
+ {%- set ns.val = ns.buf -%}
43
+ {%- endmacro -%}
44
+ {%- macro _json_number(ns) -%}
45
+ {%- set start = ns.i -%}
46
+ {%- if ns.i < (ns.s|length) and ns.s[ns.i] == '-' -%}{%- set ns.i = ns.i + 1 -%}{%- endif -%}
47
+ {%- set ns.saw = false -%}
48
+ {%- for _ in range((ns.s|length) - ns.i) -%}
49
+ {%- if ns.i < (ns.s|length) and ns.s[ns.i] in '0123456789' -%}
50
+ {%- set ns.i = ns.i + 1 -%}{%- set ns.saw = true -%}
51
+ {%- else -%}{%- break -%}{%- endif -%}
52
+ {%- endfor -%}
53
+ {%- if not ns.saw -%}{{- raise_exception("tool_call.arguments: invalid JSON number") -}}{%- endif -%}
54
+ {%- set ns.is_float = false -%}
55
+ {%- if ns.i < (ns.s|length) and ns.s[ns.i] == '.' -%}
56
+ {%- set ns.is_float = true -%}{%- set ns.i = ns.i + 1 -%}{%- set ns.saw = false -%}
57
+ {%- for _ in range((ns.s|length) - ns.i) -%}
58
+ {%- if ns.i < (ns.s|length) and ns.s[ns.i] in '0123456789' -%}
59
+ {%- set ns.i = ns.i + 1 -%}{%- set ns.saw = true -%}
60
+ {%- else -%}{%- break -%}{%- endif -%}
61
+ {%- endfor -%}
62
+ {%- if not ns.saw -%}{{- raise_exception("tool_call.arguments: invalid JSON number") -}}{%- endif -%}
63
+ {%- endif -%}
64
+ {%- if ns.i < (ns.s|length) and ns.s[ns.i] in 'eE' -%}
65
+ {%- set ns.is_float = true -%}{%- set ns.i = ns.i + 1 -%}
66
+ {%- if ns.i < (ns.s|length) and ns.s[ns.i] in '+-' -%}{%- set ns.i = ns.i + 1 -%}{%- endif -%}
67
+ {%- set ns.saw = false -%}
68
+ {%- for _ in range((ns.s|length) - ns.i) -%}
69
+ {%- if ns.i < (ns.s|length) and ns.s[ns.i] in '0123456789' -%}
70
+ {%- set ns.i = ns.i + 1 -%}{%- set ns.saw = true -%}
71
+ {%- else -%}{%- break -%}{%- endif -%}
72
+ {%- endfor -%}
73
+ {%- if not ns.saw -%}{{- raise_exception("tool_call.arguments: invalid JSON number") -}}{%- endif -%}
74
+ {%- endif -%}
75
+ {%- set raw = ns.s[start:ns.i] -%}
76
+ {%- if ns.is_float -%}{%- set ns.val = raw|float -%}{%- else -%}{%- set ns.val = raw|int -%}{%- endif -%}
77
+ {%- endmacro -%}
78
+ {%- macro _json_object(ns) -%}
79
+ {%- set ns.i = ns.i + 1 -%}
80
+ {%- set _ = _json_skip(ns) -%}
81
+ {%- if ns.i < (ns.s|length) and ns.s[ns.i] == '}' -%}
82
+ {%- set ns.i = ns.i + 1 -%}
83
+ {%- set ns.val = dict([]) -%}
84
+ {%- else -%}
85
+ {%- set loc = namespace(pairs=[], done=false) -%}
86
+ {%- for _ in range((ns.s|length) - ns.i) -%}
87
+ {%- set _ = _json_skip(ns) -%}
88
+ {%- if ns.i >= (ns.s|length) or ns.s[ns.i] != '"' -%}{{- raise_exception("tool_call.arguments: JSON object key must be a string") -}}{%- endif -%}
89
+ {%- set _ = _json_string(ns) -%}
90
+ {%- set key = ns.val -%}
91
+ {%- set _ = _json_skip(ns) -%}
92
+ {%- if ns.i >= (ns.s|length) or ns.s[ns.i] != ':' -%}{{- raise_exception("tool_call.arguments: expected ':' in JSON object") -}}{%- endif -%}
93
+ {%- set ns.i = ns.i + 1 -%}
94
+ {%- set _ = _json_value(ns) -%}
95
+ {%- set loc.pairs = loc.pairs + [(key, ns.val)] -%}
96
+ {%- set _ = _json_skip(ns) -%}
97
+ {%- if ns.i < (ns.s|length) and ns.s[ns.i] == ',' -%}
98
+ {%- set ns.i = ns.i + 1 -%}
99
+ {%- elif ns.i < (ns.s|length) and ns.s[ns.i] == '}' -%}
100
+ {%- set ns.i = ns.i + 1 -%}
101
+ {%- set loc.done = true -%}
102
+ {%- break -%}
103
+ {%- else -%}
104
+ {{- raise_exception("tool_call.arguments: expected ',' or '}' in JSON object") -}}
105
+ {%- endif -%}
106
+ {%- endfor -%}
107
+ {%- if not loc.done -%}{{- raise_exception("tool_call.arguments: unterminated JSON object") -}}{%- endif -%}
108
+ {%- set ns.val = dict(loc.pairs) -%}
109
+ {%- endif -%}
110
+ {%- endmacro -%}
111
+ {%- macro _json_array(ns) -%}
112
+ {%- set ns.i = ns.i + 1 -%}
113
+ {%- set _ = _json_skip(ns) -%}
114
+ {%- if ns.i < (ns.s|length) and ns.s[ns.i] == ']' -%}
115
+ {%- set ns.i = ns.i + 1 -%}
116
+ {%- set ns.val = [] -%}
117
+ {%- else -%}
118
+ {%- set loc = namespace(arr=[], done=false) -%}
119
+ {%- for _ in range((ns.s|length) - ns.i) -%}
120
+ {%- set _ = _json_value(ns) -%}
121
+ {%- set loc.arr = loc.arr + [ns.val] -%}
122
+ {%- set _ = _json_skip(ns) -%}
123
+ {%- if ns.i < (ns.s|length) and ns.s[ns.i] == ',' -%}
124
+ {%- set ns.i = ns.i + 1 -%}
125
+ {%- elif ns.i < (ns.s|length) and ns.s[ns.i] == ']' -%}
126
+ {%- set ns.i = ns.i + 1 -%}
127
+ {%- set loc.done = true -%}
128
+ {%- break -%}
129
+ {%- else -%}
130
+ {{- raise_exception("tool_call.arguments: expected ',' or ']' in JSON array") -}}
131
+ {%- endif -%}
132
+ {%- endfor -%}
133
+ {%- if not loc.done -%}{{- raise_exception("tool_call.arguments: unterminated JSON array") -}}{%- endif -%}
134
+ {%- set ns.val = loc.arr -%}
135
+ {%- endif -%}
136
+ {%- endmacro -%}
137
+ {%- macro _json_value(ns) -%}
138
+ {%- set _ = _json_skip(ns) -%}
139
+ {%- if ns.i >= (ns.s|length) -%}{{- raise_exception("tool_call.arguments: unexpected end of JSON") -}}{%- endif -%}
140
+ {%- set c = ns.s[ns.i] -%}
141
+ {%- if c == '"' -%}{%- set _ = _json_string(ns) -%}
142
+ {%- elif c == '{' -%}{%- set _ = _json_object(ns) -%}
143
+ {%- elif c == '[' -%}{%- set _ = _json_array(ns) -%}
144
+ {%- elif c == '-' or c in '0123456789' -%}{%- set _ = _json_number(ns) -%}
145
+ {%- elif ns.s[ns.i:ns.i+4] == 'true' -%}{%- set ns.val = true -%}{%- set ns.i = ns.i + 4 -%}
146
+ {%- elif ns.s[ns.i:ns.i+5] == 'false' -%}{%- set ns.val = false -%}{%- set ns.i = ns.i + 5 -%}
147
+ {%- elif ns.s[ns.i:ns.i+4] == 'null' -%}{%- set ns.val = none -%}{%- set ns.i = ns.i + 4 -%}
148
+ {%- else -%}{{- raise_exception("tool_call.arguments: invalid JSON value") -}}
149
+ {%- endif -%}
150
+ {%- endmacro -%}
151
+ {%- macro load_tool_args(arguments, out) -%}
152
+ {%- if arguments is mapping -%}
153
+ {%- set out.val = arguments -%}
154
+ {%- elif arguments is string -%}
155
+ {%- if (arguments|trim) in ['', 'null'] -%}
156
+ {%- set out.val = dict([]) -%}
157
+ {%- else -%}
158
+ {%- set ns = namespace(s=arguments, i=0, val=none, buf='', saw=false, is_float=false, pairs=[], arr=[]) -%}
159
+ {%- set _ = _json_value(ns) -%}
160
+ {%- set _ = _json_skip(ns) -%}
161
+ {%- if ns.i != (ns.s|length) -%}{{- raise_exception("tool_call.arguments: trailing data after JSON value") -}}{%- endif -%}
162
+ {%- if ns.val is mapping -%}
163
+ {%- set out.val = ns.val -%}
164
+ {%- else -%}
165
+ {{- raise_exception("tool_call.arguments: JSON string must be an object") -}}
166
+ {%- endif -%}
167
+ {%- endif -%}
168
+ {%- elif arguments is none -%}
169
+ {%- set out.val = dict([]) -%}
170
+ {%- else -%}
171
+ {{- raise_exception("tool_call.arguments must be a mapping or a JSON object string") -}}
172
+ {%- endif -%}
173
+ {%- endmacro -%}
174
  {#- ----------‑‑‑ special token variables ‑‑‑---------- -#}
175
  {%- set HYTK = ':opensource' %}
176
  {%- set eos_token = '<|hy_eos{}|>'.format(HYTK) %}
 
343
  {{- content }}
344
  {{- toolcalls_begin_token + '\n' }}
345
  {%- for tool in message['tool_calls'] %}
346
+ {%- set _args_out = namespace(val=none) -%}{%- set _ = load_tool_args(tool['function']['arguments'], _args_out) -%}{%- set arguments = _args_out.val -%}
347
  {{- toolcall_begin_token + tool['function']['name'] + toolsep_token + '\n' }}
348
  {%- for key, value in arguments.items() %}
349
  {{- argkey_begin_token + key + argkey_end_token + '\n' }}