Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
conversational
text-generation-inference
Instructions to use atisharma/narratrix with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use atisharma/narratrix with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="atisharma/narratrix") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("atisharma/narratrix") model = AutoModelForCausalLM.from_pretrained("atisharma/narratrix", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use atisharma/narratrix with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "atisharma/narratrix" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "atisharma/narratrix", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/atisharma/narratrix
- SGLang
How to use atisharma/narratrix with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "atisharma/narratrix" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "atisharma/narratrix", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "atisharma/narratrix" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "atisharma/narratrix", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use atisharma/narratrix with Docker Model Runner:
docker model run hf.co/atisharma/narratrix
Download hooks/update.sample from atisharma/narratrix: direct link, hf CLI and curl.
- Browser
- Download file 3.65 kB
-
https://huggingface.co/atisharma/narratrix/resolve/main/hooks/update.sample
- Command line
-
hf download hf://atisharma/narratrix/hooks/update.sample
-
curl -L -o update.sample https://huggingface.co/atisharma/narratrix/resolve/main/hooks/update.sample
3.65 kB
| # | |
| # An example hook script to block unannotated tags from entering. | |
| # Called by "git receive-pack" with arguments: refname sha1-old sha1-new | |
| # | |
| # To enable this hook, rename this file to "update". | |
| # | |
| # Config | |
| # ------ | |
| # hooks.allowunannotated | |
| # This boolean sets whether unannotated tags will be allowed into the | |
| # repository. By default they won't be. | |
| # hooks.allowdeletetag | |
| # This boolean sets whether deleting tags will be allowed in the | |
| # repository. By default they won't be. | |
| # hooks.allowmodifytag | |
| # This boolean sets whether a tag may be modified after creation. By default | |
| # it won't be. | |
| # hooks.allowdeletebranch | |
| # This boolean sets whether deleting branches will be allowed in the | |
| # repository. By default they won't be. | |
| # hooks.denycreatebranch | |
| # This boolean sets whether remotely creating branches will be denied | |
| # in the repository. By default this is allowed. | |
| # | |
| # --- Command line | |
| refname="$1" | |
| oldrev="$2" | |
| newrev="$3" | |
| # --- Safety check | |
| if [ -z "$GIT_DIR" ]; then | |
| echo "Don't run this script from the command line." >&2 | |
| echo " (if you want, you could supply GIT_DIR then run" >&2 | |
| echo " $0 <ref> <oldrev> <newrev>)" >&2 | |
| exit 1 | |
| fi | |
| if [ -z "$refname" -o -z "$oldrev" -o -z "$newrev" ]; then | |
| echo "usage: $0 <ref> <oldrev> <newrev>" >&2 | |
| exit 1 | |
| fi | |
| # --- Config | |
| allowunannotated=$(git config --type=bool hooks.allowunannotated) | |
| allowdeletebranch=$(git config --type=bool hooks.allowdeletebranch) | |
| denycreatebranch=$(git config --type=bool hooks.denycreatebranch) | |
| allowdeletetag=$(git config --type=bool hooks.allowdeletetag) | |
| allowmodifytag=$(git config --type=bool hooks.allowmodifytag) | |
| # check for no description | |
| projectdesc=$(sed -e '1q' "$GIT_DIR/description") | |
| case "$projectdesc" in | |
| "Unnamed repository"* | "") | |
| echo "*** Project description file hasn't been set" >&2 | |
| exit 1 | |
| ;; | |
| esac | |
| # --- Check types | |
| # if $newrev is 0000...0000, it's a commit to delete a ref. | |
| zero=$(git hash-object --stdin </dev/null | tr '[0-9a-f]' '0') | |
| if [ "$newrev" = "$zero" ]; then | |
| newrev_type=delete | |
| else | |
| newrev_type=$(git cat-file -t $newrev) | |
| fi | |
| case "$refname","$newrev_type" in | |
| refs/tags/*,commit) | |
| # un-annotated tag | |
| short_refname=${refname##refs/tags/} | |
| if [ "$allowunannotated" != "true" ]; then | |
| echo "*** The un-annotated tag, $short_refname, is not allowed in this repository" >&2 | |
| echo "*** Use 'git tag [ -a | -s ]' for tags you want to propagate." >&2 | |
| exit 1 | |
| fi | |
| ;; | |
| refs/tags/*,delete) | |
| # delete tag | |
| if [ "$allowdeletetag" != "true" ]; then | |
| echo "*** Deleting a tag is not allowed in this repository" >&2 | |
| exit 1 | |
| fi | |
| ;; | |
| refs/tags/*,tag) | |
| # annotated tag | |
| if [ "$allowmodifytag" != "true" ] && git rev-parse $refname > /dev/null 2>&1 | |
| then | |
| echo "*** Tag '$refname' already exists." >&2 | |
| echo "*** Modifying a tag is not allowed in this repository." >&2 | |
| exit 1 | |
| fi | |
| ;; | |
| refs/heads/*,commit) | |
| # branch | |
| if [ "$oldrev" = "$zero" -a "$denycreatebranch" = "true" ]; then | |
| echo "*** Creating a branch is not allowed in this repository" >&2 | |
| exit 1 | |
| fi | |
| ;; | |
| refs/heads/*,delete) | |
| # delete branch | |
| if [ "$allowdeletebranch" != "true" ]; then | |
| echo "*** Deleting a branch is not allowed in this repository" >&2 | |
| exit 1 | |
| fi | |
| ;; | |
| refs/remotes/*,commit) | |
| # tracking branch | |
| ;; | |
| refs/remotes/*,delete) | |
| # delete tracking branch | |
| if [ "$allowdeletebranch" != "true" ]; then | |
| echo "*** Deleting a tracking branch is not allowed in this repository" >&2 | |
| exit 1 | |
| fi | |
| ;; | |
| *) | |
| # Anything else (is there anything else?) | |
| echo "*** Update hook: unknown type of update to ref $refname of type $newrev_type" >&2 | |
| exit 1 | |
| ;; | |
| esac | |
| # --- Finished | |
| exit 0 | |