crispasr integration Claude Opus 5 (1M context) commited on
Commit
108f0f3
·
1 Parent(s): 6eab2c8

feat(nscale): accept NSCALE_TOKEN, and read ~/.env so it is found

Browse files

Nscale has been empty in the published data since it was added — its
catalog is only reachable through the authenticated API, and the key was
never available to the fetcher. Two reasons, both fixed here.

The variable name: Nscale's console issues the credential as NSCALE_TOKEN,
while the fetcher only looked for NSCALE_API_KEY. Both are now accepted,
the alias kept so any existing setup or CI secret of that name still works,
and the health check's keyEnv now takes a list so "any one of these" counts
as present rather than reporting a false SKIP.

The location: load-env only read the project .env and ../AIToolkit/.env,
so a credential shared across projects in ~/.env was invisible. It now
reads every candidate that exists — project, AIToolkit, then home —
instead of only the first one found, so a key present solely in the home
file is picked up even when a project .env exists for other keys. Earlier
files still win, a variable already set in the real environment is never
overwritten, and `export FOO=` lines now parse.

Result: Nscale contributes 22 models and the health check is green across
the board for the first time (15 passed, 0 failed, 0 skipped).

NSCALE_TOKEN is wired into both workflows alongside the existing alias; the
repository secret still has to be added for CI to populate it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

.github/workflows/healthcheck.yml CHANGED
@@ -34,6 +34,7 @@ jobs:
34
  - run: npm install
35
  - name: Run fetcher health check
36
  env:
 
37
  NSCALE_API_KEY: ${{ secrets.NSCALE_API_KEY }}
38
  GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
39
  run: node scripts/healthcheck.js
 
34
  - run: npm install
35
  - name: Run fetcher health check
36
  env:
37
+ NSCALE_TOKEN: ${{ secrets.NSCALE_TOKEN }}
38
  NSCALE_API_KEY: ${{ secrets.NSCALE_API_KEY }}
39
  GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
40
  run: node scripts/healthcheck.js
.github/workflows/update-providers.yml CHANGED
@@ -29,6 +29,7 @@ jobs:
29
  env:
30
  REQUESTY_API_KEY: ${{ secrets.REQUESTY_API_KEY }}
31
  OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
 
32
  NSCALE_API_KEY: ${{ secrets.NSCALE_API_KEY }}
33
  GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
34
  run: node scripts/fetch-providers.js
 
29
  env:
30
  REQUESTY_API_KEY: ${{ secrets.REQUESTY_API_KEY }}
31
  OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
32
+ NSCALE_TOKEN: ${{ secrets.NSCALE_TOKEN }}
33
  NSCALE_API_KEY: ${{ secrets.NSCALE_API_KEY }}
34
  GROQ_API_KEY: ${{ secrets.GROQ_API_KEY }}
35
  run: node scripts/fetch-providers.js
README.md CHANGED
@@ -25,7 +25,7 @@ Compare pricing, capabilities, and benchmark scores across LLM providers — wit
25
  | Scaleway | EU 🇫🇷 | GDPR-compliant |
26
  | OVHcloud | EU 🇫🇷 | GDPR-compliant, sovereign, pay-per-token |
27
  | STACKIT | EU 🇩🇪 | Schwarz Group, fully sovereign (no US hyperscaler) |
28
- | Nscale | EEA 🇬🇧/🇳🇴 | UK HQ, EEA (Norway) hosting, pay-per-token — **needs `NSCALE_API_KEY` or stays empty** |
29
  | EUrouter | EU 🇳🇱 | EUrouter B.V., Amsterdam; EU-only sub-processors. Routes across EU providers — but see the caveat below |
30
  | HostYourAI | EU 🇳🇱 | HostYourAI B.V.; open models on EU GPUs via vLLM, DPA + EU Sovereignty Mode |
31
  | Mistral AI | EU 🇫🇷 | GDPR-compliant |
@@ -100,7 +100,7 @@ npm run fetch:mistral # Mistral AI
100
  npm run fetch:scaleway # Scaleway
101
  npm run fetch:ovhcloud # OVHcloud
102
  npm run fetch:stackit # STACKIT
103
- npm run fetch:nscale # Nscale (needs NSCALE_API_KEY)
104
  npm run fetch:langdock # Langdock
105
  npm run fetch:groq # Groq
106
  npm run fetch:ionos # IONOS
@@ -134,17 +134,22 @@ is fetched from a public, keyless source, so a clone with no `.env` at all
134
  still populates almost the whole table:
135
 
136
  ```
137
- NSCALE_API_KEY=... # REQUIRED for Nscale — its catalog is only behind
138
- # the API, so without this Nscale stays empty
 
139
  OPENROUTER_API_KEY=... # optional; unlocks the full catalog vs public subset
140
  GROQ_API_KEY=... # optional; adds context windows to Groq models
141
  REQUESTY_API_KEY=... # optional; Requesty currently returns its catalog
142
  # publicly, so this is not needed today
143
  ```
144
 
145
- Place in `.env` in the project root or `../AIToolkit/.env`. In CI they are read
146
- from repository secrets of the same name; a missing secret makes that fetcher
147
- skip rather than fail, so forks work without any.
 
 
 
 
148
 
149
  ## Deployment
150
 
 
25
  | Scaleway | EU 🇫🇷 | GDPR-compliant |
26
  | OVHcloud | EU 🇫🇷 | GDPR-compliant, sovereign, pay-per-token |
27
  | STACKIT | EU 🇩🇪 | Schwarz Group, fully sovereign (no US hyperscaler) |
28
+ | Nscale | EEA 🇬🇧/🇳🇴 | UK HQ, EEA (Norway) hosting, pay-per-token — **needs `NSCALE_TOKEN` or stays empty** |
29
  | EUrouter | EU 🇳🇱 | EUrouter B.V., Amsterdam; EU-only sub-processors. Routes across EU providers — but see the caveat below |
30
  | HostYourAI | EU 🇳🇱 | HostYourAI B.V.; open models on EU GPUs via vLLM, DPA + EU Sovereignty Mode |
31
  | Mistral AI | EU 🇫🇷 | GDPR-compliant |
 
100
  npm run fetch:scaleway # Scaleway
101
  npm run fetch:ovhcloud # OVHcloud
102
  npm run fetch:stackit # STACKIT
103
+ npm run fetch:nscale # Nscale (needs NSCALE_TOKEN)
104
  npm run fetch:langdock # Langdock
105
  npm run fetch:groq # Groq
106
  npm run fetch:ionos # IONOS
 
134
  still populates almost the whole table:
135
 
136
  ```
137
+ NSCALE_TOKEN=... # REQUIRED for Nscale — its catalog is only behind
138
+ # the API, so without this Nscale stays empty.
139
+ # NSCALE_API_KEY is accepted as an alias
140
  OPENROUTER_API_KEY=... # optional; unlocks the full catalog vs public subset
141
  GROQ_API_KEY=... # optional; adds context windows to Groq models
142
  REQUESTY_API_KEY=... # optional; Requesty currently returns its catalog
143
  # publicly, so this is not needed today
144
  ```
145
 
146
+ Loaded from `.env` in the project root, then `../AIToolkit/.env`, then `~/.env`
147
+ — every one of those that exists is read, earlier files winning, so a
148
+ credential shared across projects can live in the home file alone. A variable
149
+ already set in the real environment is never overwritten.
150
+
151
+ In CI they come from repository secrets of the same name. A missing secret
152
+ makes that fetcher skip rather than fail, so forks work without any.
153
 
154
  ## Deployment
155
 
data/providers.json CHANGED
The diff for this file is too large to render. See raw diff
 
scripts/healthcheck.js CHANGED
@@ -20,14 +20,15 @@
20
  */
21
 
22
  const path = require('path');
23
- // Load local .env (project or ../AIToolkit) so keyed-provider detection is
24
- // deterministic before we probe process.env. No-op in CI (uses real secrets).
25
  require('./load-env').loadEnv();
26
 
27
  // key: provider module under scripts/providers/
28
  // min: lowest model count we still consider healthy (set well below the live
29
  // count so normal catalog churn never trips it, but a broken scraper does)
30
- // keyEnv: env var required to reach the source (null = public scrape)
 
31
  const CHECKS = [
32
  { key: 'scaleway', min: 8, keyEnv: null },
33
  { key: 'ovhcloud', min: 10, keyEnv: null },
@@ -39,7 +40,7 @@ const CHECKS = [
39
  { key: 'ionos', min: 5, keyEnv: null },
40
  { key: 'black-forest-labs', min: 6, keyEnv: null },
41
  { key: 'nebius', min: 10, keyEnv: null },
42
- { key: 'nscale', min: 5, keyEnv: 'NSCALE_API_KEY' },
43
  { key: 'eurouter', min: 80, keyEnv: null },
44
  { key: 'hostyourai', min: 15, keyEnv: null },
45
  { key: 'requesty', min: 40, keyEnv: null },
@@ -69,8 +70,9 @@ async function main() {
69
  const { key, min, keyEnv } = check;
70
  let providerName = key;
71
 
72
- if (keyEnv && !process.env[keyEnv]) {
73
- results.push({ key, providerName, status: 'SKIP', count: null, note: `no ${keyEnv}` });
 
74
  continue;
75
  }
76
 
 
20
  */
21
 
22
  const path = require('path');
23
+ // Load local .env (project, ../AIToolkit, then ~/.env) so keyed-provider
24
+ // detection is deterministic before we probe process.env. No-op in CI.
25
  require('./load-env').loadEnv();
26
 
27
  // key: provider module under scripts/providers/
28
  // min: lowest model count we still consider healthy (set well below the live
29
  // count so normal catalog churn never trips it, but a broken scraper does)
30
+ // keyEnv: env var(s) required to reach the source (null = public scrape).
31
+ // An array means any one of them is enough.
32
  const CHECKS = [
33
  { key: 'scaleway', min: 8, keyEnv: null },
34
  { key: 'ovhcloud', min: 10, keyEnv: null },
 
40
  { key: 'ionos', min: 5, keyEnv: null },
41
  { key: 'black-forest-labs', min: 6, keyEnv: null },
42
  { key: 'nebius', min: 10, keyEnv: null },
43
+ { key: 'nscale', min: 5, keyEnv: ['NSCALE_TOKEN', 'NSCALE_API_KEY'] },
44
  { key: 'eurouter', min: 80, keyEnv: null },
45
  { key: 'hostyourai', min: 15, keyEnv: null },
46
  { key: 'requesty', min: 40, keyEnv: null },
 
70
  const { key, min, keyEnv } = check;
71
  let providerName = key;
72
 
73
+ const keyEnvs = keyEnv == null ? [] : (Array.isArray(keyEnv) ? keyEnv : [keyEnv]);
74
+ if (keyEnvs.length && !keyEnvs.some((k) => process.env[k])) {
75
+ results.push({ key, providerName, status: 'SKIP', count: null, note: `no ${keyEnvs.join(' / ')}` });
76
  continue;
77
  }
78
 
scripts/load-env.js CHANGED
@@ -2,27 +2,35 @@
2
 
3
  /**
4
  * Shared environment loader.
5
- * Checks local project .env first, then ../AIToolkit/.env as a fallback.
6
- * Only sets variables that are not already present in process.env.
 
 
 
 
 
7
  */
8
 
9
  const fs = require('fs');
 
10
  const path = require('path');
11
 
12
  function loadEnv() {
13
  const candidates = [
14
  path.join(__dirname, '..', '.env'),
15
  path.join(__dirname, '..', '..', 'AIToolkit', '.env'),
 
16
  ];
17
- const envPath = candidates.find((p) => fs.existsSync(p));
18
- if (!envPath) return;
19
 
20
- const content = fs.readFileSync(envPath, 'utf8');
21
- for (const line of content.split('\n')) {
22
- const m = line.match(/^([A-Za-z_][A-Za-z0-9_]*)=(.*)$/);
23
- if (m && process.env[m[1]] === undefined) {
24
- // Strip optional surrounding quotes
25
- process.env[m[1]] = m[2].trim().replace(/^(['"])(.*)\1$/, '$2');
 
 
 
26
  }
27
  }
28
  }
 
2
 
3
  /**
4
  * Shared environment loader.
5
+ *
6
+ * Reads, in order of precedence: the project .env, then ../AIToolkit/.env,
7
+ * then ~/.env for credentials shared across projects on the same machine.
8
+ * Every existing file is read rather than only the first one found, so a key
9
+ * that lives only in the home file is still picked up when a project .env
10
+ * exists for other keys. Earlier files win, and a variable already set in
11
+ * the real environment (as in CI) is never overwritten.
12
  */
13
 
14
  const fs = require('fs');
15
+ const os = require('os');
16
  const path = require('path');
17
 
18
  function loadEnv() {
19
  const candidates = [
20
  path.join(__dirname, '..', '.env'),
21
  path.join(__dirname, '..', '..', 'AIToolkit', '.env'),
22
+ path.join(os.homedir(), '.env'),
23
  ];
 
 
24
 
25
+ for (const envPath of candidates) {
26
+ if (!fs.existsSync(envPath)) continue;
27
+ const content = fs.readFileSync(envPath, 'utf8');
28
+ for (const line of content.split('\n')) {
29
+ const m = line.match(/^\s*(?:export\s+)?([A-Za-z_][A-Za-z0-9_]*)\s*=(.*)$/);
30
+ if (m && process.env[m[1]] === undefined) {
31
+ // Strip optional surrounding quotes
32
+ process.env[m[1]] = m[2].trim().replace(/^(['"])(.*)\1$/, '$2');
33
+ }
34
  }
35
  }
36
  }
scripts/providers/nscale.js CHANGED
@@ -13,8 +13,8 @@
13
  * output). The endpoint carries no type/capability fields, so those are
14
  * derived from the model id (as the Requesty/Scaleway fetchers do).
15
  *
16
- * Requires NSCALE_API_KEY (local ../AIToolkit/.env or a CI secret). Without it
17
- * the fetcher skips and the provider keeps its existing data.
18
  */
19
 
20
  const { loadEnv } = require('../load-env');
@@ -23,7 +23,10 @@ const { getJson } = require('../fetch-utils');
23
 
24
  const API_URL = 'https://inference.api.nscale.com/v1/models';
25
 
26
- const loadApiKey = () => process.env.NSCALE_API_KEY || null;
 
 
 
27
 
28
  const EMBED_KEYWORDS = ['embed', 'bge', 'gte', 'e5-', 'stella', 'arctic-embed', 'nomic-embed'];
29
  const IMAGE_KEYWORDS = ['flux', 'stable-diffusion', 'sdxl', 'sd3', 'text-to-image'];
@@ -56,7 +59,7 @@ function classify(id) {
56
  async function fetchNscale() {
57
  const apiKey = loadApiKey();
58
  if (!apiKey) {
59
- console.warn(' (no NSCALE_API_KEY found – skipping Nscale)');
60
  return [];
61
  }
62
 
 
13
  * output). The endpoint carries no type/capability fields, so those are
14
  * derived from the model id (as the Requesty/Scaleway fetchers do).
15
  *
16
+ * Requires NSCALE_TOKEN (or NSCALE_API_KEY) from a local .env or a CI secret.
17
+ * Without it the fetcher skips and the provider keeps its existing data.
18
  */
19
 
20
  const { loadEnv } = require('../load-env');
 
23
 
24
  const API_URL = 'https://inference.api.nscale.com/v1/models';
25
 
26
+ // NSCALE_TOKEN is the name Nscale's own console hands out (and what the shared
27
+ // .env uses); NSCALE_API_KEY is kept as an alias so existing setups and the CI
28
+ // secret of that name keep working.
29
+ const loadApiKey = () => process.env.NSCALE_API_KEY || process.env.NSCALE_TOKEN || null;
30
 
31
  const EMBED_KEYWORDS = ['embed', 'bge', 'gte', 'e5-', 'stella', 'arctic-embed', 'nomic-embed'];
32
  const IMAGE_KEYWORDS = ['flux', 'stable-diffusion', 'sdxl', 'sd3', 'text-to-image'];
 
59
  async function fetchNscale() {
60
  const apiKey = loadApiKey();
61
  if (!apiKey) {
62
+ console.warn(' (no NSCALE_TOKEN / NSCALE_API_KEY found – skipping Nscale)');
63
  return [];
64
  }
65