Instructions to use numind/NuExtract-tiny with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use numind/NuExtract-tiny with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="numind/NuExtract-tiny")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("numind/NuExtract-tiny")
model = AutoModelForCausalLM.from_pretrained("numind/NuExtract-tiny", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use numind/NuExtract-tiny with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "numind/NuExtract-tiny"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "numind/NuExtract-tiny",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/numind/NuExtract-tiny

SGLang

How to use numind/NuExtract-tiny with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "numind/NuExtract-tiny" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "numind/NuExtract-tiny",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "numind/NuExtract-tiny" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "numind/NuExtract-tiny",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use numind/NuExtract-tiny with Docker Model Runner:
```
docker model run hf.co/numind/NuExtract-tiny
```

NuExtract-tiny / README.md

Commit History

Update README.md

0f3834d
verified

SorenDreano commited on May 19

Update README.md

41bb93b
verified

liamcripwell commited on Oct 17, 2024

Update README.md

684ff5b
verified

liamcripwell commited on Oct 14, 2024

Add base_model metadata (#2)

c4fca7f
verified

Alexandre-Numind

davanstrien HF Staff commited on Aug 23, 2024

Update README.md

42ce801
verified

liamcripwell commited on Jun 26, 2024

Update README.md

0c86184
verified

liamcripwell commited on Jun 26, 2024

Update README.md

afa5086
verified

Alexandre-Numind commited on Jun 24, 2024

Update README.md

7cd6009
verified

Alexandre-Numind commited on Jun 24, 2024

Update README.md

824c7e9
verified

Alexandre-Numind commited on Jun 24, 2024

Update README.md

9e926fd
verified

Alexandre-Numind commited on Jun 24, 2024

Update README.md

cbb8243
verified

Alexandre-Numind commited on Jun 24, 2024

Update README.md

304c554
verified

Alexandre-Numind commited on Jun 24, 2024

Update README.md

00fe38d
verified

Alexandre-Numind commited on Jun 24, 2024

Update README.md

80c21c9
verified

Alexandre-Numind commited on Jun 24, 2024

Update README.md

5fece14
verified

Alexandre-Numind commited on Jun 24, 2024

Update README.md

59e3b65
verified

Alexandre-Numind commited on Jun 24, 2024

Update README.md

1c56e40
verified

Alexandre-Numind commited on Jun 21, 2024

Create README.md

600d2d8
verified

Alexandre-Numind commited on Jun 20, 2024

Commit History

Update README.md 0f3834d verified

Update README.md 41bb93b verified

Update README.md 684ff5b verified

Add base_model metadata (#2) c4fca7f verified

Update README.md 42ce801 verified

Update README.md 0c86184 verified

Update README.md afa5086 verified

Update README.md 7cd6009 verified

Update README.md 824c7e9 verified

Update README.md 9e926fd verified

Update README.md cbb8243 verified

Update README.md 304c554 verified

Update README.md 00fe38d verified

Update README.md 80c21c9 verified

Update README.md 5fece14 verified

Update README.md 59e3b65 verified

Update README.md 1c56e40 verified

Create README.md 600d2d8 verified

Update README.md

0f3834d
verified

Update README.md

41bb93b
verified

Update README.md

684ff5b
verified

Add base_model metadata (#2)

c4fca7f
verified

Update README.md

42ce801
verified

Update README.md

0c86184
verified

Update README.md

afa5086
verified

Update README.md

7cd6009
verified

Update README.md

824c7e9
verified

Update README.md

9e926fd
verified

Update README.md

cbb8243
verified

Update README.md

304c554
verified

Update README.md

00fe38d
verified

Update README.md

80c21c9
verified

Update README.md

5fece14
verified

Update README.md

59e3b65
verified

Update README.md

1c56e40
verified

Create README.md

600d2d8
verified