Update llama.cpp usage to the current CLI

#2
by kashif HF Staff - opened
Files changed (1) hide show
  1. README.md +3 -16
README.md CHANGED
@@ -26,10 +26,6 @@ llama-completion \
26
  -n 128 --temp 0 --no-display-prompt
27
  ```
28
 
29
- If you use `llama-cli` instead, pass `-no-cnv` so it does not wrap the prompt
30
- in assistant-style markers (otherwise the model may emit annotation tags such
31
- as `<protein_coding_region>` instead of DNA).
32
-
33
  ## Available files
34
 
35
  | Quant | File | Size |
@@ -105,7 +101,7 @@ Or serve the 3B with the 500M draft (`llama-server` accepts the same `-md` flag)
105
  ./build/bin/llama-server \
106
  -m carbon-3b-bf16.gguf \
107
  -md carbon-500m-bf16.gguf \
108
- --draft-max 16 --draft-min 1 \
109
  --port 8080
110
  ```
111
 
@@ -122,20 +118,11 @@ curl -s http://localhost:8080/completion -d '{
122
  The source card's Python `score()` function computes mean log-prob per DNA token. In llama.cpp the closest tools are `llama-perplexity` for corpus-level perplexity (`perplexity = exp(-mean_logprob)`):
123
 
124
  ```bash
125
- # one prompt per line in dna_corpus.txt, each wrapped in <dna>...</dna>
126
  ./build/bin/llama-perplexity -m carbon-3b-bf16.gguf -f dna_corpus.txt --ppl-stride 0
127
  ```
128
 
129
- Or `llama-server` with `logprobs` for per-token log-probabilities:
130
-
131
- ```bash
132
- ./build/bin/llama-server -m carbon-3b-bf16.gguf --port 8080 &
133
- curl -s http://localhost:8080/completion -d '{
134
- "prompt": "<dna>GGGCTATAAAGGCCATCGATCGATCGATCGATCGATCGATCG</dna>",
135
- "n_predict": 0,
136
- "n_probs": 1
137
- }' | jq '.completion_probabilities'
138
- ```
139
 
140
  ### Long context with YaRN (65k tokens ≈ 393 kbp)
141
 
 
26
  -n 128 --temp 0 --no-display-prompt
27
  ```
28
 
 
 
 
 
29
  ## Available files
30
 
31
  | Quant | File | Size |
 
101
  ./build/bin/llama-server \
102
  -m carbon-3b-bf16.gguf \
103
  -md carbon-500m-bf16.gguf \
104
+ --spec-draft-n-max 16 --spec-draft-n-min 1 \
105
  --port 8080
106
  ```
107
 
 
118
  The source card's Python `score()` function computes mean log-prob per DNA token. In llama.cpp the closest tools are `llama-perplexity` for corpus-level perplexity (`perplexity = exp(-mean_logprob)`):
119
 
120
  ```bash
121
+ # dna_corpus.txt is scored as one token stream, split into context-sized chunks
122
  ./build/bin/llama-perplexity -m carbon-3b-bf16.gguf -f dna_corpus.txt --ppl-stride 0
123
  ```
124
 
125
+ For per-sequence log-likelihoods, use the Python `score()` from the source card.
 
 
 
 
 
 
 
 
 
126
 
127
  ### Long context with YaRN (65k tokens ≈ 393 kbp)
128