Instructions to use LR-AI-Labs/tiny-universal-NER with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use LR-AI-Labs/tiny-universal-NER with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="LR-AI-Labs/tiny-universal-NER")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("LR-AI-Labs/tiny-universal-NER") model = AutoModelForCausalLM.from_pretrained("LR-AI-Labs/tiny-universal-NER", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use LR-AI-Labs/tiny-universal-NER with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "LR-AI-Labs/tiny-universal-NER" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LR-AI-Labs/tiny-universal-NER", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/LR-AI-Labs/tiny-universal-NER
- SGLang
How to use LR-AI-Labs/tiny-universal-NER with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "LR-AI-Labs/tiny-universal-NER" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LR-AI-Labs/tiny-universal-NER", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "LR-AI-Labs/tiny-universal-NER" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "LR-AI-Labs/tiny-universal-NER", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use LR-AI-Labs/tiny-universal-NER with Docker Model Runner:
docker model run hf.co/LR-AI-Labs/tiny-universal-NER
Update README.md
Browse files
README.md
CHANGED
|
@@ -51,16 +51,16 @@ messages = [
|
|
| 51 |
prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
| 52 |
outputs = pipe(prompt, max_new_tokens=256, do_sample=False)
|
| 53 |
print(outputs[0]["generated_text"])
|
| 54 |
-
#<|system|>
|
| 55 |
-
#A virtual assistant answers questions from a user based on the provided text.</s>
|
| 56 |
-
#<|user|>
|
| 57 |
-
#Text: The American Bank Note Company Printing Plant is a repurposed complex of three interconnected buildings in the Hunts Point neighborhood of the Bronx in New York City. The innovative Kirby, Petit & Green design was built in 1909–1911 by the American Bank Note Company on land which had previously been part of Edward G. Faile's country estate. A wide variety of financial instruments were printed there; at one point, over five million documents were produced per day, including half the securities traded on the New York Stock Exchange.</s>
|
| 58 |
-
#<|assistant|>
|
| 59 |
-
#I've read this text.</s>
|
| 60 |
-
#<|user|>
|
| 61 |
-
#What describes location in the text?</s>
|
| 62 |
-
#<|assistant|>
|
| 63 |
-
#["Hunts Point", "Bronx", "New York City"]
|
| 64 |
```
|
| 65 |
|
| 66 |
### Note: Inferences are based on one entity type at a time. For multiple entity types, create separate instances for each type.
|
|
|
|
| 51 |
prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
| 52 |
outputs = pipe(prompt, max_new_tokens=256, do_sample=False)
|
| 53 |
print(outputs[0]["generated_text"])
|
| 54 |
+
# <|system|>
|
| 55 |
+
# A virtual assistant answers questions from a user based on the provided text.</s>
|
| 56 |
+
# <|user|>
|
| 57 |
+
# Text: The American Bank Note Company Printing Plant is a repurposed complex of three interconnected buildings in the Hunts Point neighborhood of the Bronx in New York City. The innovative Kirby, Petit & Green design was built in 1909–1911 by the American Bank Note Company on land which had previously been part of Edward G. Faile's country estate. A wide variety of financial instruments were printed there; at one point, over five million documents were produced per day, including half the securities traded on the New York Stock Exchange.</s>
|
| 58 |
+
# <|assistant|>
|
| 59 |
+
# I've read this text.</s>
|
| 60 |
+
# <|user|>
|
| 61 |
+
# What describes location in the text?</s>
|
| 62 |
+
# <|assistant|>
|
| 63 |
+
# ["Hunts Point", "Bronx", "New York City"]
|
| 64 |
```
|
| 65 |
|
| 66 |
### Note: Inferences are based on one entity type at a time. For multiple entity types, create separate instances for each type.
|