dusersad12 commited on
Commit
39c5f5c
·
verified ·
1 Parent(s): 2e2294d

Add NovaMind best checkpoint (step_750) with finalized evaluation results

Browse files
Files changed (6) hide show
  1. README.md +120 -0
  2. config.json +5 -0
  3. figures/fig1.png +0 -0
  4. figures/fig2.png +0 -0
  5. figures/fig3.png +0 -0
  6. pytorch_model.bin +3 -0
README.md ADDED
@@ -0,0 +1,120 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: transformers
4
+ ---
5
+ # NovaMind
6
+ <!-- markdownlint-disable first-line-h1 -->
7
+ <!-- markdownlint-disable html -->
8
+ <!-- markdownlint-disable no-duplicate-header -->
9
+
10
+ <div align="center">
11
+ <img src="figures/fig1.png" width="60%" alt="NovaMind" />
12
+ </div>
13
+ <hr>
14
+
15
+ <div align="center" style="line-height: 1;">
16
+ <a href="LICENSE" style="margin: 2px;">
17
+ <img alt="License" src="figures/fig2.png" style="display: inline-block; vertical-align: middle;"/>
18
+ </a>
19
+ </div>
20
+
21
+ ## 1. Introduction
22
+
23
+ NovaMind is our 7B assistant model, and this release refreshes it with a longer mid-training stage plus a much heavier post-training pipeline. The new NovaMind pushes deeper into multi-step reasoning, coding, and general logic, and it now sits comfortably among the strongest open models of its size class on our internal and public evaluations.
24
+
25
+ <p align="center">
26
+ <img width="80%" src="figures/fig3.png">
27
+ </p>
28
+
29
+ The refresh pays off most on hard reasoning suites. On the GPQA Diamond test, accuracy has climbed from 62.0% in the previous release to 84.6% in the current one. The gain comes from genuinely deeper thinking: on GPQA the previous model averaged 14K tokens per question, while the new version averages 26K tokens per question.
30
+
31
+ Alongside the stronger reasoning, this version also trims the hallucination rate on our factuality probes and makes multi-turn function calling noticeably more dependable.
32
+
33
+ ## 2. Evaluation Results
34
+
35
+ ### Comprehensive Benchmark Results
36
+
37
+ <div align="center">
38
+
39
+ | | Benchmark | Baseline-7B | Baseline-7B-Chat | Baseline-7B-v2 | NovaMind |
40
+ |---|---|---|---|---|---|
41
+ | **Core Reasoning Tasks** | Math Reasoning | 0.421 | 0.448 | 0.462 | 0.486 |
42
+ | | Logical Reasoning | 0.742 | 0.763 | 0.781 | 0.819 |
43
+ | | Common Sense | 0.688 | 0.695 | 0.703 | 0.735 |
44
+ | **Language Understanding** | Reading Comprehension | 0.642 | 0.657 | 0.664 | 0.700 |
45
+ | | Question Answering | 0.553 | 0.571 | 0.584 | 0.608 |
46
+ | | Text Classification | 0.786 | 0.795 | 0.802 | 0.826 |
47
+ | | Sentiment Analysis | 0.759 | 0.766 | 0.772 | 0.792 |
48
+ | **Generation Tasks** | Code Generation | 0.497 | 0.511 | 0.519 | 0.523 |
49
+ | | Creative Writing | 0.571 | 0.583 | 0.596 | 0.610 |
50
+ | | Dialogue Generation | 0.594 | 0.607 | 0.618 | 0.644 |
51
+ | | Summarization | 0.712 | 0.726 | 0.734 | 0.767 |
52
+ | **Specialized Capabilities**| Translation | 0.748 | 0.762 | 0.771 | 0.804 |
53
+ | | Knowledge Retrieval | 0.623 | 0.641 | 0.652 | 0.676 |
54
+ | | Instruction Following | 0.703 | 0.721 | 0.729 | 0.758 |
55
+ | | Safety Evaluation | 0.741 | 0.728 | 0.752 | 0.734 |
56
+
57
+ </div>
58
+
59
+ ### Overall Performance Summary
60
+ NovaMind posts consistent gains in every category we track, with the largest jumps concentrated in the reasoning-heavy and generation-heavy suites.
61
+
62
+ ## 3. Chat Website & API Platform
63
+ We run a hosted chat playground and an inference API for NovaMind; check the developer portal on our website for access details.
64
+
65
+ ## 4. How to Run Locally
66
+
67
+ The usage recommendations for this NovaMind release differ from earlier versions in the following ways:
68
+
69
+ 1. A system prompt carrying the current date is now recommended.
70
+ 2. It is no longer necessary to prepend special tokens that force the model into a fixed thinking pattern.
71
+
72
+ NovaMind-Small keeps the architecture of its base model and shares the NovaMind tokenizer configuration, so it runs with exactly the same stack as its base model.
73
+
74
+ ### System Prompt
75
+ We recommend opening every session with a dated system prompt.
76
+ ```
77
+ You are NovaMind, a helpful AI assistant.
78
+ Today is {current date}.
79
+ ```
80
+ For example,
81
+ ```
82
+ You are NovaMind, a helpful AI assistant.
83
+ Today is September 27, 2026, Sunday.
84
+ ```
85
+ ### Temperature
86
+ We recommend a sampling temperature $T_{model}$ of 0.7.
87
+
88
+ ### Prompts for File Uploading and Web Search
89
+ For file uploading, build prompts with the template below, where {file_name}, {file_content} and {question} are arguments.
90
+ ```
91
+ file_template = \
92
+ """[file name]: {file_name}
93
+ [file content begin]
94
+ {file_content}
95
+ [file content end]
96
+ {question}"""
97
+ ```
98
+ For web-search-augmented generation, we recommend the prompt template below, where {search_results}, {cur_date}, and {question} are arguments.
99
+ ```
100
+ search_answer_en_template = \
101
+ '''# The following contents are the search results related to the user's message:
102
+ {search_results}
103
+ Each search result above is marked up as [webpage X begin]...[webpage X end], with X being the numeric index of the result. Cite the supporting context inline with [citation:X] right after the sentence that leans on it, and when a sentence draws on several results, list every number it depends on, e.g. [citation:3][citation:5]. Keep the citations spread through the answer instead of clustering them at the end.
104
+ When responding, please keep the following points in mind:
105
+ - Today is {cur_date}.
106
+ - Not every search result is closely related to the question; weigh and filter them before using anything.
107
+ - For listing-type questions (e.g. listing all matching papers), cap the answer at about 10 key items and point the user to the sources for the complete picture. Avoid bringing in items that are not present in the search results.
108
+ - For creative tasks (e.g. writing a report), work [citation:X] references into the body of the text rather than dropping them all at the end. Read the user's intent, pick a fitting format, and develop a rich, multi-angle answer.
109
+ - If the response runs long, structure it with paragraphs or a short list of bullet points, and merge related items.
110
+ - For objective Q&A, one or two extra sentences of context are welcome when the direct answer is very short.
111
+ - Unless the user asks otherwise, answer in the same language as the user's question.
112
+ # The user's message is:
113
+ {question}'''
114
+ ```
115
+
116
+ ## 5. License
117
+ The code in this repository is released under the [Apache-2.0 License](LICENSE), and the NovaMind model weights follow the same [Apache-2.0 License](LICENSE). The model family supports commercial use and distillation.
118
+
119
+ ## 6. Contact
120
+ For questions or bug reports, please open an issue on our GitHub repository or write to contact@novamind-ai.org.
config.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "model_type": "nova",
3
+ "architectures": ["NovaMindForCausalLM"],
4
+ "training_step": 750
5
+ }
figures/fig1.png ADDED
figures/fig2.png ADDED
figures/fig3.png ADDED
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6fcc6c1a763e7a84f7b04eb09cd3560ece7510b59e3e119c5bebc0d31af38f0f
3
+ size 120