Instructions to use mkd-dev/Keural-Cortex-8B-v1.0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mkd-dev/Keural-Cortex-8B-v1.0 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="mkd-dev/Keural-Cortex-8B-v1.0")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("mkd-dev/Keural-Cortex-8B-v1.0")
model = AutoModelForCausalLM.from_pretrained("mkd-dev/Keural-Cortex-8B-v1.0", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use mkd-dev/Keural-Cortex-8B-v1.0 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mkd-dev/Keural-Cortex-8B-v1.0"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mkd-dev/Keural-Cortex-8B-v1.0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/mkd-dev/Keural-Cortex-8B-v1.0

SGLang

How to use mkd-dev/Keural-Cortex-8B-v1.0 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mkd-dev/Keural-Cortex-8B-v1.0" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mkd-dev/Keural-Cortex-8B-v1.0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mkd-dev/Keural-Cortex-8B-v1.0" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mkd-dev/Keural-Cortex-8B-v1.0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use mkd-dev/Keural-Cortex-8B-v1.0 with Docker Model Runner:
```
docker model run hf.co/mkd-dev/Keural-Cortex-8B-v1.0
```

Keural-Cortex-8B-v1.0 / README.md

mkd-dev

mkd-hossain

Duplicate from mkd-ai-dev/Keural-Cortex-8B-v1.0

ca88fc5 20 days ago

preview code

Raw History Blame Contribute Delete

6.01 kB

metadata

language:
  - en
  - ko
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
tags:
  - text-generation
  - conversational
  - korean
  - english
  - function-calling
  - agent

Keural Cortex Foundation Model (8B) — v1.0

Keural Cortex is MKD's local-first Korean-English language-model program for Keural applications. This release is an 8B-parameter, instruction- and preference-aligned checkpoint with a structured tool-calling layer for enterprise workflows.

It is designed to run in customer-controlled infrastructure through an OpenAI-compatible vLLM endpoint. The host application—not the model—executes tools, enforces permissions, retrieves live data, and owns user data handling.

Release status: Public v1.0. Evaluate on your own data, workflows, safety requirements, and target serving configuration before production deployment.

Cortex 8B architecture overview

introduction

English. Keural Cortex is a locally deployable AI foundation-model capability built for Korean-English enterprise interaction. It combines efficient 8B-scale inference, controllable OpenAI-compatible serving, and structured tool-call generation so that Keural can connect users to approved enterprise workflows while retaining application-level control of data, permissions, and execution.

한국어. Keural Cortex는 한국어·영어 기반의 엔터프라이즈 AI 상호작용을 위해 설계된 로컬 배포형 파운데이션 모델 역량입니다. 8B 규모의 효율적인 추론, OpenAI 호환 서빙, 구조화된 Tool Call 생성을 결합하여 Keural이 승인된 업무 워크플로와 연동할 수 있도록 지원합니다. 데이터·권한·실행 통제는 모델이 아닌 Keural 애플리케이션 계층에서 관리합니다.

Key strengths

Strength	What it enables
Local-first deployment	On-premise or customer-controlled serving patterns; the application controls networking, credentials, access, and retention.
Korean-English interaction	Direct bilingual chat behavior for Keural user experiences.
Structured tool calls	Generates OpenAI-compatible calls for `search_web`, `fetch_web_urls`, `retrieve_documents`, and `ask_clarification` when those tools are provided.
Agent-ready architecture	Separates model decision-making from tool execution, allowing the product layer to apply authorization, auditing, retrieval, and real-time data policies.
Long-context design	Configuration supports a 65,536-token maximum position length. Serving limits should be set according to available GPU memory and workload.
Efficient serving	Tested with vLLM using an OpenAI-compatible `/v1/chat/completions` interface.

Private local deployment pattern

Model details

Item	Value
Architecture	Decoder-only transformer language model
Transformer layers	36
Hidden size	4,096
Attention heads / KV heads	32 / 8
Vocabulary size	151,936
Maximum position length	65,536 tokens
Weights	BF16 SafeTensors, ~16 GB
Alignment path	Continued training, supervised instruction tuning, preference optimization, and Keural agent/tool-contract tuning

Verified serving smoke tests

The following were verified through vLLM's OpenAI-compatible API in the release environment:

Plain Korean chat returned a direct answer without emitting a tool call.
A current-information request generated a structured search_web call.
An exhaustive document request generated retrieve_documents with exhaustive: true.
An underspecified revision request generated ask_clarification with a question and checklist.

These are smoke tests, not a substitute for domain-specific benchmarking or safety evaluation.

Quick start: vLLM

vllm serve <MODEL_PATH> \
  --served-model-name keural-cortex-8b-v1.0 \
  --tool-call-parser hermes \
  --enable-auto-tool-choice

For text-only chat, send a normal OpenAI Chat Completions request without tools or tool_choice. For agent workflows, provide only the tools your application authorizes, validate all tool arguments server-side, and execute calls outside the model process.

Example tool contract

{
  "type": "function",
  "function": {
    "name": "retrieve_documents",
    "description": "Search content in the user's available documents.",
    "parameters": {
      "type": "object",
      "properties": {
        "query": {"type": "string"},
        "file_ids": {"type": "array", "items": {"type": "string"}},
        "exhaustive": {"type": "boolean"}
      },
      "required": ["query"]
    }
  }
}

Limitations and responsible use

Outputs can be inaccurate, incomplete, or inconsistent across prompts and serving configurations. Verify important information.
The model has no inherent live-web access, private-document access, or authority to take external actions. These capabilities must be supplied and governed by the host application.
Tool calls are model suggestions. Validate schemas, authorization, user intent, and side effects before execution.
Do not use this preview as the sole basis for high-stakes medical, legal, financial, safety, employment, credit, or access-control decisions.

Certification, license, and attribution

Certification note. Keural Cortex Foundation Model (8B) v1.0 is based on Qwen3-8B-Base and is released under Apache-2.0. Please retain applicable upstream notices and comply with the license terms.

Citation

@misc{keural-cortex-8b-v1.0,
  title={Keural Cortex Foundation Model (8B) — v1.0},
  author={MKD AI},
  year={2026},
  publisher={Hugging Face}
}