Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
conversational
text-generation-inference
Instructions to use atisharma/narratrix with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use atisharma/narratrix with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="atisharma/narratrix") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("atisharma/narratrix") model = AutoModelForCausalLM.from_pretrained("atisharma/narratrix", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use atisharma/narratrix with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "atisharma/narratrix" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "atisharma/narratrix", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/atisharma/narratrix
- SGLang
How to use atisharma/narratrix with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "atisharma/narratrix" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "atisharma/narratrix", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "atisharma/narratrix" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "atisharma/narratrix", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use atisharma/narratrix with Docker Model Runner:
docker model run hf.co/atisharma/narratrix
File size: 2,308 Bytes
54613a2 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | #!/bin/sh
# An example hook script to validate a patch (and/or patch series) before
# sending it via email.
#
# The hook should exit with non-zero status after issuing an appropriate
# message if it wants to prevent the email(s) from being sent.
#
# To enable this hook, rename this file to "sendemail-validate".
#
# By default, it will only check that the patch(es) can be applied on top of
# the default upstream branch without conflicts in a secondary worktree. After
# validation (successful or not) of the last patch of a series, the worktree
# will be deleted.
#
# The following config variables can be set to change the default remote and
# remote ref that are used to apply the patches against:
#
# sendemail.validateRemote (default: origin)
# sendemail.validateRemoteRef (default: HEAD)
#
# Replace the TODO placeholders with appropriate checks according to your
# needs.
validate_cover_letter () {
file="$1"
# TODO: Replace with appropriate checks (e.g. spell checking).
true
}
validate_patch () {
file="$1"
# Ensure that the patch applies without conflicts.
git am -3 "$file" || return
# TODO: Replace with appropriate checks for this patch
# (e.g. checkpatch.pl).
true
}
validate_series () {
# TODO: Replace with appropriate checks for the whole series
# (e.g. quick build, coding style checks, etc.).
true
}
# main -------------------------------------------------------------------------
if test "$GIT_SENDEMAIL_FILE_COUNTER" = 1
then
remote=$(git config --default origin --get sendemail.validateRemote) &&
ref=$(git config --default HEAD --get sendemail.validateRemoteRef) &&
worktree=$(mktemp --tmpdir -d sendemail-validate.XXXXXXX) &&
git worktree add -fd --checkout "$worktree" "refs/remotes/$remote/$ref" &&
git config --replace-all sendemail.validateWorktree "$worktree"
else
worktree=$(git config --get sendemail.validateWorktree)
fi || {
echo "sendemail-validate: error: failed to prepare worktree" >&2
exit 1
}
unset GIT_DIR GIT_WORK_TREE
cd "$worktree" &&
if grep -q "^diff --git " "$1"
then
validate_patch "$1"
else
validate_cover_letter "$1"
fi &&
if test "$GIT_SENDEMAIL_FILE_COUNTER" = "$GIT_SENDEMAIL_FILE_TOTAL"
then
git config --unset-all sendemail.validateWorktree &&
trap 'git worktree remove -ff "$worktree"' EXIT &&
validate_series
fi
|