Atto

atto

Atto is a family of compact contextual models designed for parameter-efficient language representation learning.

These models can be loaded using Hugging Face Transformers. Since Atto uses a custom architecture, trust_remote_code=True is required.

from transformers import AutoModel

model = AutoModel.from_pretrained(
    "neuml/atto-17m-8192d",
    trust_remote_code=True
)

Training Parameters

Parameter Value
learning_rate 0.001
gradient_accumulation_steps 1
bf16 True

More Information

See the Introducing Atto article for benchmark results, architecture details, and training methodology.

Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for NeuML/atto-17m-8192d

Finetunes
2 models

Collection including NeuML/atto-17m-8192d

Article mentioning NeuML/atto-17m-8192d