Context length: model card vs metadata (256k vs 131k)
#6
by MetaphoricalCode - opened
The model card states a context window of up to 256K tokens, yet it's different in the model's metadata:
"max_position_embeddings": 131072
Which one's true? Currently all GGUF quants lock maximum context to 131072 due to metadata.
Native context: 128K (256K via YaRN). Unless specified otherwise, all evaluation results are obtained at 128K.