subin commited on
Commit
196c23d
·
verified ·
1 Parent(s): 47c6eaa

Keep each thread's adapter gate to itself

Browse files

The adapter gate in `vela3_adapters.py` was one object for the whole process. `active()` set its adapter name and row mask, and every `MultiLoRA.forward` read them, so when two threads called `system_one` on the same model, one thread's passes ran with the other's adapter. On vela3-edge on CPU, with one thread asking span questions in a loop and another asking the same refund question 25 times, all 25 answers came back wrong: 0.736 to 0.820 against 0.956 when asked alone.

The gate is now a `threading.local`, so each thread sees only the adapter it switched on, and a new thread starts with every adapter off. Nothing else changes: `active()`, `MultiLoRA.forward` and the graph check in `modeling_vela3.py` read the gate as before, now on the calling thread.

Tested on vela3-edge on CPU. In the same two-thread run, 0 of 25 answers now differ from the answer asked alone. One request each over three states, with the nine router questions, a hazard question, an ordinary choice and yes/no question and three span questions (PII, relation, entities), gives byte-identical JSON before and after. The change is the same in all six sizes.

Files changed (2) hide show
  1. SHA256SUMS +1 -1
  2. vela3_adapters.py +2 -1
SHA256SUMS CHANGED
@@ -16,7 +16,7 @@ cc710bdeb492c51564600ff09b645bce2b716d1fff76e810249b0926dd2e49fe ner_head.safet
16
  d2c5f2f446cf12d2f8a24dbf634452477088523b0973e1e07df244c851421efb span_decode_v2.py
17
  e5e74fe176bfd580f86448a152c7a8e9d278645baaf155e5f2f0ee3b766c8a61 span_head.safetensors
18
  d1c684f81bd9f4d9d2c749120162f26482a9e05984ca8865652a5ec3c18a109d span_lora.safetensors
19
- eb69e58c18593527a0eb0bade7bc05eec315f3039fe92e1c500ba9c5f0a36d31 vela3_adapters.py
20
  9c14b2d82a95f33eb2c094f301a2f33450205b67e5cdd3cf6082e1fd827847ae vela3_config.json
21
  b7fcefdd1a0b3e8ac22ac12338c72e76925688a989b5af6c70a104fa70122a39 vela3_labeldef.py
22
  97d2448765a7ee5411dff44c601c3a073fdad9a9921f9818bac4135916081c20 vela3_span.py
 
16
  d2c5f2f446cf12d2f8a24dbf634452477088523b0973e1e07df244c851421efb span_decode_v2.py
17
  e5e74fe176bfd580f86448a152c7a8e9d278645baaf155e5f2f0ee3b766c8a61 span_head.safetensors
18
  d1c684f81bd9f4d9d2c749120162f26482a9e05984ca8865652a5ec3c18a109d span_lora.safetensors
19
+ 2e88d7af591e3097a735ecbbb17fa6433aaa5b803e4b810cab6231e62ce3a2ee vela3_adapters.py
20
  9c14b2d82a95f33eb2c094f301a2f33450205b67e5cdd3cf6082e1fd827847ae vela3_config.json
21
  b7fcefdd1a0b3e8ac22ac12338c72e76925688a989b5af6c70a104fa70122a39 vela3_labeldef.py
22
  97d2448765a7ee5411dff44c601c3a073fdad9a9921f9818bac4135916081c20 vela3_span.py
vela3_adapters.py CHANGED
@@ -11,12 +11,13 @@ from __future__ import annotations
11
 
12
  import contextlib
13
  import json
 
14
 
15
  import torch
16
  from torch import nn
17
 
18
 
19
- class _Gate:
20
  name = None # active adapter name, or None (all adapters off)
21
  rows = None # None = every row of the batch; else a bool tensor [B]
22
 
 
11
 
12
  import contextlib
13
  import json
14
+ import threading
15
 
16
  import torch
17
  from torch import nn
18
 
19
 
20
+ class _Gate(threading.local): # per thread: concurrent requests never see each other's adapter
21
  name = None # active adapter name, or None (all adapters off)
22
  rows = None # None = every row of the batch; else a bool tensor [B]
23