Hugging Face's logo Hugging Face
  • Models
  • Datasets
  • Spaces
  • Buckets
  • Docs
  • Enterprise
  • Pricing
    • Website
      • Tasks
      • HuggingChat
      • Collections
      • Languages
      • Organizations
    • Community
      • Blog
      • Posts
      • Daily Papers
      • Hardware
      • Learn
      • Discord
      • Forum
      • GitHub
    • Solutions
      • Team & Enterprise
      • Hugging Face PRO
      • Enterprise Support
      • Inference Providers
      • Inference Endpoints
      • Storage Buckets

  • Log In
  • Sign Up

radicalnumerics
/
Halcyon-Tokenizers

Text-to-Image
Transformers
halcyon
radical-numerics
tokenizer
fsq
Model card Files Files and versions
xet
Community

Instructions to use radicalnumerics/Halcyon-Tokenizers with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

  • Libraries
  • Transformers

    How to use radicalnumerics/Halcyon-Tokenizers with Transformers:

    # pip install -U transformers accelerate
    # Load model directly
    from transformers import AutoModel
    model = AutoModel.from_pretrained("radicalnumerics/Halcyon-Tokenizers", device_map="auto")
  • Notebooks
  • Google Colab
  • Kaggle

You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

Gated model
You can list files but not access them

Preview of files found in this repository
  • image_tokenizer
    Halcyon tokenizers: image_tokenizer/di8x8_242172_ema.pt, tokenizer.json, tokenizer_config.json, vocab_layout.json, bpe16k_final.report.json, README.md 20 days ago
  • .gitattributes
    1.52 kB
    initial commit 20 days ago
  • README.md
    2.85 kB
    Update README.md 4 days ago
  • bpe16k_final.report.json
    3.43 kB
    Halcyon tokenizers: image_tokenizer/di8x8_242172_ema.pt, tokenizer.json, tokenizer_config.json, vocab_layout.json, bpe16k_final.report.json, README.md 20 days ago
  • tokenizer.json
    1.13 MB
    Halcyon tokenizers: image_tokenizer/di8x8_242172_ema.pt, tokenizer.json, tokenizer_config.json, vocab_layout.json, bpe16k_final.report.json, README.md 20 days ago
  • tokenizer_config.json
    119 Bytes
    Halcyon tokenizers: image_tokenizer/di8x8_242172_ema.pt, tokenizer.json, tokenizer_config.json, vocab_layout.json, bpe16k_final.report.json, README.md 20 days ago
  • vocab_layout.json
    540 Bytes
    Halcyon tokenizers: image_tokenizer/di8x8_242172_ema.pt, tokenizer.json, tokenizer_config.json, vocab_layout.json, bpe16k_final.report.json, README.md 20 days ago