thuEcstasy1 commited on
Commit
ba256f2
路
verified 路
1 Parent(s): 67a7a12

Rename project to InfuseLoop and update code and project links

Browse files
Files changed (2) hide show
  1. README.md +8 -8
  2. config.json +2 -2
README.md CHANGED
@@ -5,39 +5,39 @@ license: apache-2.0
5
  library_name: pytorch
6
  pipeline_tag: text-generation
7
  tags:
8
- - foresightloop
9
  - looped-transformer
10
  - causal-lm
11
  - research
12
  ---
13
 
14
- # ForesightLoop-4
15
 
16
  A 596,139,465-parameter base language model with 4 parallel recurrent loops and causal cross-trajectory Attention Residuals. Trained on 6,000,082,944 FineWeb-Edu tokens (45,777 updates).
17
 
18
- **Code:** [thuEcstasy/ForesightLoop](https://github.com/thuEcstasy/ForesightLoop).
19
 
20
  ## Usage
21
 
22
  Install the custom reference implementation:
23
 
24
  ```bash
25
- pip install git+https://github.com/thuEcstasy/ForesightLoop.git
26
  ```
27
 
28
  ```python
29
  from transformers import AutoTokenizer
30
- from foresightloop import ForesightLoop
31
- repo = "thuEcstasy1/ForesightLoop-4"
32
  tokenizer = AutoTokenizer.from_pretrained(repo)
33
- model = ForesightLoop.from_pretrained(repo, device="cuda")
34
  prompt = "The scientist repeated the experiment and found that"
35
  x = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).input_ids.cuda()
36
  y = model.generate(x, max_new_tokens=32, eos_token_id=tokenizer.eos_token_id)
37
  print(tokenizer.decode(y[0, x.shape[1]:], skip_special_tokens=True))
38
  ```
39
 
40
- This is a base text model, **not an instruction-tuned assistant**. Use natural prefixes without a chat template. The custom `ForesightLoop` class is required; this repository does not register `AutoModelForCausalLM` or promise compatibility with generic Transformers generation pipelines.
41
 
42
  The public API is a readable eager PyTorch reference. It does not reproduce the optimized CUDA Graph / FA4 / Triton latency measurements in the paper.
43
 
 
5
  library_name: pytorch
6
  pipeline_tag: text-generation
7
  tags:
8
+ - infuseloop
9
  - looped-transformer
10
  - causal-lm
11
  - research
12
  ---
13
 
14
+ # InfuseLoop-4
15
 
16
  A 596,139,465-parameter base language model with 4 parallel recurrent loops and causal cross-trajectory Attention Residuals. Trained on 6,000,082,944 FineWeb-Edu tokens (45,777 updates).
17
 
18
+ **Project:** [InfuseLoop](https://infuseloop-demo.regal-vole-5301.chatgpt.site) 路 **Code:** [thuEcstasy/InfuseLoop](https://github.com/thuEcstasy/InfuseLoop).
19
 
20
  ## Usage
21
 
22
  Install the custom reference implementation:
23
 
24
  ```bash
25
+ pip install git+https://github.com/thuEcstasy/InfuseLoop.git
26
  ```
27
 
28
  ```python
29
  from transformers import AutoTokenizer
30
+ from infuseloop import InfuseLoop
31
+ repo = "thuEcstasy1/InfuseLoop-4"
32
  tokenizer = AutoTokenizer.from_pretrained(repo)
33
+ model = InfuseLoop.from_pretrained(repo, device="cuda")
34
  prompt = "The scientist repeated the experiment and found that"
35
  x = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).input_ids.cuda()
36
  y = model.generate(x, max_new_tokens=32, eos_token_id=tokenizer.eos_token_id)
37
  print(tokenizer.decode(y[0, x.shape[1]:], skip_special_tokens=True))
38
  ```
39
 
40
+ This is a base text model, **not an instruction-tuned assistant**. Use natural prefixes without a chat template. The custom `InfuseLoop` class is required; this repository does not register `AutoModelForCausalLM` or promise compatibility with generic Transformers generation pipelines.
41
 
42
  The public API is a readable eager PyTorch reference. It does not reproduce the optimized CUDA Graph / FA4 / Triton latency measurements in the paper.
43
 
config.json CHANGED
@@ -1,7 +1,7 @@
1
  {
2
- "model_type": "foresightloop",
3
  "architectures": [
4
- "ForesightLoop"
5
  ],
6
  "checkpoint_format": "foresightloop-safetensors-v1",
7
  "backbone_config": {
 
1
  {
2
+ "model_type": "infuseloop",
3
  "architectures": [
4
+ "InfuseLoop"
5
  ],
6
  "checkpoint_format": "foresightloop-safetensors-v1",
7
  "backbone_config": {