GiuLeo01 commited on
Commit
2441fdd
·
verified ·
1 Parent(s): 2c71ce1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -6
README.md CHANGED
@@ -2,6 +2,10 @@
2
  library_name: transformers
3
  tags:
4
  - unsloth
 
 
 
 
5
  base_model:
6
  - Qwen/Qwen2.5-Coder-3B-Instruct
7
  ---
@@ -11,10 +15,12 @@ base_model:
11
  <!-- Provide a quick summary of what the model is/does. -->
12
  This model is a prototype of a large language model specifically fine-tuned for Fortran90 code generation. It is based on the Qwen 2.5 Coder 3B Instruct model and has been refined using Supervised Fine-Tuning and Reinforcement Learning with Verifiable Rewards (via GRPO).
13
 
 
 
14
  ## Evaluation
15
  Due to the lack of existing benchmarks for Fortran code generation, a quick-and-dirty adaptation of the HumanEval benchmark was created for Fortran in order to evaluate the model. This benchmark is currently under review and will be released publicly at a later date.
16
 
17
- According to the current version of the benchmark:
18
 
19
  | Model | pass@1 | Compile Error Rate |
20
  |----------------------|--------|--------------------|
@@ -24,13 +30,12 @@ According to the current version of the benchmark:
24
  | GPT-4o | x | x |
25
 
26
 
27
-
28
- ## Model Details
29
-
30
-
31
  ## Uses
32
 
33
  <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
 
 
 
34
 
35
  ### Out-of-Scope Use
36
 
@@ -52,4 +57,4 @@ Use the code below to get started with the model.
52
 
53
  [More Information Needed]
54
 
55
- ### Training Procedure
 
2
  library_name: transformers
3
  tags:
4
  - unsloth
5
+ - grpo
6
+ - reinforcement_learning
7
+ - sft
8
+ - fortran
9
  base_model:
10
  - Qwen/Qwen2.5-Coder-3B-Instruct
11
  ---
 
15
  <!-- Provide a quick summary of what the model is/does. -->
16
  This model is a prototype of a large language model specifically fine-tuned for Fortran90 code generation. It is based on the Qwen 2.5 Coder 3B Instruct model and has been refined using Supervised Fine-Tuning and Reinforcement Learning with Verifiable Rewards (via GRPO).
17
 
18
+ This model was fine-tuned briefly, without any human-labeled data and using only a single consumer GPU. Despite these clear constraints, the training process led to a 400% boost in performance on tasks involving simple to moderately complex program generation (HumanEval). Compilation errors dropped as well, and the model now performs close to much larger, general-purpose models that weren’t specifically trained for this task.
19
+
20
  ## Evaluation
21
  Due to the lack of existing benchmarks for Fortran code generation, a quick-and-dirty adaptation of the HumanEval benchmark was created for Fortran in order to evaluate the model. This benchmark is currently under review and will be released publicly at a later date.
22
 
23
+ According to the current demo version of the FortranHumanEval benchmark:
24
 
25
  | Model | pass@1 | Compile Error Rate |
26
  |----------------------|--------|--------------------|
 
30
  | GPT-4o | x | x |
31
 
32
 
 
 
 
 
33
  ## Uses
34
 
35
  <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
36
+ The model is highly specialized in generating Fortran90 programs that read input from stdin and print output to stdout. It’s recommended to use the model with a low temperature (or even disable sampling entirely) to maximize accuracy.
37
+
38
+ Before running any generated code, it’s always a good idea to check how the program handles input from stdin, especially for users who are new to Fortran.
39
 
40
  ### Out-of-Scope Use
41
 
 
57
 
58
  [More Information Needed]
59
 
60
+ ### Training Procedure