Instructions to use darkstudio009/Vanessa-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use darkstudio009/Vanessa-1B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="darkstudio009/Vanessa-1B")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("darkstudio009/Vanessa-1B", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use darkstudio009/Vanessa-1B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "darkstudio009/Vanessa-1B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "darkstudio009/Vanessa-1B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/darkstudio009/Vanessa-1B
- SGLang
How to use darkstudio009/Vanessa-1B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "darkstudio009/Vanessa-1B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "darkstudio009/Vanessa-1B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "darkstudio009/Vanessa-1B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "darkstudio009/Vanessa-1B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use darkstudio009/Vanessa-1B with Docker Model Runner:
docker model run hf.co/darkstudio009/Vanessa-1B
Vanessa-1B
A ~1.1B-parameter decoder-only transformer, pretrained fully from scratch β original tokenizer, original model code, original training loop.
Developed by Dark Studio.
β οΈ Status: pretraining in progress. Vanessa-1B has not finished pretraining and is not yet an instruction-tuned or fine-tuned assistant. Weights currently on this repo are training checkpoints, not a finished release.
Overview
Vanessa-1B is an independent large language model project: every part of the stack β the byte-level BPE tokenizer, the transformer implementation, and the distributed training loop β was built from first principles rather than adapted from an existing checkpoint or wrapping an existing transformers architecture class. It is the second generation of the Vanessa project, scaled up roughly 9x from an initial ~124M-parameter proof of concept.
Model Details
| Parameters | ~1.1B |
| Architecture | Decoder-only Transformer |
| Layers | 22 |
| Hidden size | 2,048 |
| Attention heads | 32 |
| Key/value heads | 4 (Grouped-Query Attention, 8:1) |
| Feed-forward | SwiGLU, intermediate size 5,632 |
| Normalization | RMSNorm |
| Position embeddings | Rotary (RoPE), ΞΈ = 10,000 |
| Context length | 2,048 tokens |
| Vocabulary | 32,768 |
| Tied embeddings | No |
Training
Data. FineWeb-Edu (the education-filtered variant of FineWeb), targeting 25B tokens.
Optimizer. AdamW (Ξ²β=0.9, Ξ²β=0.95), peak learning rate 3e-4 with linear warmup and cosine decay to 10% of peak, weight decay 0.1 (applied to 2D+ parameters only), gradient clipping at 1.0.
Precision. Mixed precision: fp16 for forward/backward compute with dynamic loss scaling, fp32 master weights for the optimizer update.
Infrastructure. Trained on 2x NVIDIA T4 GPUs (Kaggle) under PyTorch FSDP (FULL_SHARD) with gradient checkpointing, since the full model does not fit as a single replica on one 16GB GPU. Training is resumable across ephemeral sessions via checkpoints stored on this repo.
Tokenizer
Custom byte-level BPE, vocabulary size 32,768, trained from scratch on a fixed sample for reproducibility. ChatML-style special tokens (<|im_start|>, <|im_end|>, role tokens, tool-call tokens, reasoning tokens) are pre-reserved in the vocabulary for a future instruction-tuning stage.
Intended Use & Limitations
Vanessa-1B is a base (completion) model from an independent research/hobby project, currently mid-pretraining. It has not been instruction-tuned, safety-aligned, or evaluated on standard benchmarks. It should be treated as experimental: not suited for production use, high-stakes decisions, or deployment without further training and evaluation. Its knowledge and behavior are entirely a product of its training data (FineWeb-Edu) and are expected to reflect that data's coverage and biases.
How to Load
Vanessa-1B uses its own architecture classes (VanessaConfig, VanessaForCausalLM), not a built-in transformers model type, so it isn't loadable via a plain AutoModel.from_pretrained(...) call yet. Once pretraining is complete and the sharded FSDP checkpoint has been exported to a single merged state dict, loading looks like:
import json
from vanessa_architecture import VanessaConfig, VanessaForCausalLM
config = VanessaConfig.from_dict(json.load(open("config.json")))
model = VanessaForCausalLM(config)
model.load_state_dict(torch.load("vanessa_merged.pt")["params"])
(vanessa_architecture.py is the model definition from the training notebook; a merge step from the two FSDP rank shards to a single state dict is needed first.)
About Dark Studio
Vanessa-1B is developed and maintained by Dark Studio, an independent effort to build large language models from first principles.
License
Not yet finalized β update the license field above and this section before wider release.