Hugging Face's logo

Spaces:
SOVYNstudio
/
README
Configuration error

App Files Community
README / README.md
SOVYN's picture
SOVYN
Update README.md
5590679 verified
preview code
|
Raw History Blame Contribute Delete
10.1 kB
<div align="center">
<img
src="https://capsule-render.vercel.app/api?type=waving&height=260&section=header&text=SOVYN&fontSize=82&fontAlignY=36&desc=Independent%20AI%20Research%20%26%20Open%20Model%20Development&descAlignY=58&descSize=16&color=0:020617,35:0F172A,72:172554,100:2563EB&fontColor=F8FAFC"
width="100%"
/>
<br>
<h3>Open models. Efficient intelligence. Built from the ground up.</h3>
<p>
SOVYN is an independent AI research project focused on building<br>
efficient language and multimodal models that developers can actually run.
</p>
<br>
<a href="https://sovyn.org">
<img src="https://img.shields.io/badge/sovyn.org-0F172A?style=for-the-badge&logo=googlechrome&logoColor=white">
</a>
<a href="https://github.com/gaedsstudio">
<img src="https://img.shields.io/badge/GitHub-gaedsstudio-0F172A?style=for-the-badge&logo=github&logoColor=white">
</a>
<a href="https://huggingface.co/SOVYNstudio">
<img src="https://img.shields.io/badge/Hugging%20Face-SOVYNstudio-FFD21E?style=for-the-badge&logo=huggingface&logoColor=111827">
</a>
<br><br>
<img src="https://img.shields.io/badge/KOREAN-2563EB?style=flat-square">
<img src="https://img.shields.io/badge/ENGLISH-2563EB?style=flat-square">
<img src="https://img.shields.io/badge/CODE-2563EB?style=flat-square">
<img src="https://img.shields.io/badge/MULTIMODAL-7C3AED?style=flat-square">
<img src="https://img.shields.io/badge/OPEN%20RESEARCH-0F172A?style=flat-square">
<br><br><br>
</div>
---
<div align="center">
# SOVYN 2.3
### Current Generation
<p>
One model family.<br>
Different levels of compute, speed and capability.
</p>
<br>
<table>
<tr>
<td width="50%" valign="middle" align="center">
<h2>SOVYN 2.3 Fast</h2>
<h3>Project Meridian</h3>
<br>
The primary development model<br>
of the SOVYN 2.3 generation.
<br><br>
<code>~1B PARAMETERS</code>
&nbsp;
<code>8K CONTEXT</code>
<br><br>
<img src="https://img.shields.io/badge/STATUS-IN%20DEVELOPMENT-2563EB?style=for-the-badge">
</td>
<td width="50%" valign="top">
<h3>Architecture</h3>
<pre>
Decoder-only Transformer
28 Transformer Blocks
├── RMSNorm
├── RoPE
├── GQA
├── QK-Norm
├── SwiGLU
└── Hybrid Attention
├── Local × 21
└── Global × 7
48K Tokenizer
8K Context Window
</pre>
</td>
</tr>
</table>
</div>
<br>
---
<div align="center">
# Model Family
<p>
Designed as one generation with multiple deployment targets.
</p>
<br>
<table>
<tr>
<td align="center" width="25%">
<h2>Mini</h2>
<img src="https://img.shields.io/badge/LOCAL-334155?style=flat-square">
<br><br>
Small footprint.<br>
Local-first deployment.
<br><br>
<sub>PLANNED</sub>
</td>
<td align="center" width="25%">
<h2>Fast</h2>
<img src="https://img.shields.io/badge/ACTIVE-2563EB?style=flat-square">
<br><br>
Fast inference.<br>
Efficient compute.
<br><br>
<strong>IN DEVELOPMENT</strong>
</td>
<td align="center" width="25%">
<h2>Pro</h2>
<img src="https://img.shields.io/badge/BALANCED-7C3AED?style=flat-square">
<br><br>
Capability and<br>
efficiency combined.
<br><br>
<sub>PLANNED</sub>
</td>
<td align="center" width="25%">
<h2>Max</h2>
<img src="https://img.shields.io/badge/CAPABILITY-111827?style=flat-square">
<br><br>
The largest model<br>
in the generation.
<br><br>
<sub>PLANNED</sub>
</td>
</tr>
</table>
</div>
<br>
---
<div align="center">
# Generations
<br>
<table>
<tr>
<td width="28%" align="center">
<h1>01</h1>
<h2>SOVYN 1</h2>
<strong>First Generation</strong>
<br><br>
Compact language models<br>
and foundational experiments.
<br><br>
<code>85M</code>
</td>
<td width="28%" align="center">
<h1>02</h1>
<h2>SOVYN 2</h2>
<strong>Second Generation</strong>
<br><br>
Expanded model scale<br>
and training research.
<br><br>
<code>300M</code>
</td>
<td width="44%" align="center">
<h1>2.3</h1>
<h2>SOVYN 2.3</h2>
<strong>Current Generation</strong>
<br><br>
Architecture, efficiency<br>
and multimodal research.
<br><br>
<code>MINI</code>
&nbsp;
<code>FAST</code>
&nbsp;
<code>PRO</code>
&nbsp;
<code>MAX</code>
</td>
</tr>
</table>
</div>
<br>
---
<div align="center">
# Research
<p>
SOVYN research spans model architecture, training efficiency,<br>
multilingual intelligence and multimodal systems.
</p>
</div>
<br>
<table>
<tr>
<td width="50%" valign="top">
<h3>Language Intelligence</h3>
<pre>
Korean
English
Code
Reasoning
Instruction Following
</pre>
</td>
<td width="50%" valign="top">
<h3>Architecture</h3>
<pre>
Attention
Tokenization
Efficient Training
Memory Efficiency
Inference Optimization
</pre>
</td>
</tr>
<tr>
<td width="50%" valign="top">
<h3>Multimodal</h3>
<pre>
Vision Understanding
Speech Recognition
Speech Generation
Image Generation
Cross-modal Reasoning
</pre>
</td>
<td width="50%" valign="top">
<h3>Open Research</h3>
<pre>
Model Weights
Technical Reports
Datasets
Benchmarks
Experiments
</pre>
</td>
</tr>
</table>
<br>
---
<div align="center">
# How SOVYN Thinks About Scale
<br>
<p>
Bigger is not the only direction.
</p>
<p>
We explore whether architecture, data quality, tokenization and training<br>
can produce stronger models without requiring massive compute budgets.
</p>
<br>
<table>
<tr>
<td align="center" width="25%">
<h2>01</h2>
<h3>Architecture</h3>
Better use of<br>
every parameter.
</td>
<td align="center" width="25%">
<h2>02</h2>
<h3>Data</h3>
Cleaner and more<br>
useful training data.
</td>
<td align="center" width="25%">
<h2>03</h2>
<h3>Training</h3>
Efficient optimization<br>
and experimentation.
</td>
<td align="center" width="25%">
<h2>04</h2>
<h3>Inference</h3>
Models that can run<br>
outside large datacenters.
</td>
</tr>
</table>
</div>
<br>
---
<div align="center">
# Research Principles
<br>
<table>
<tr>
<td width="33%" align="center">
<h1>01</h1>
<h3>BUILD</h3>
Start from the architecture,<br>
not from the hype.
<br><br>
Experiment with model design,<br>
training and data directly.
</td>
<td width="33%" align="center">
<h1>02</h1>
<h3>MEASURE</h3>
Document parameters,<br>
training behavior and limitations.
<br><br>
Research should be reproducible.
</td>
<td width="33%" align="center">
<h1>03</h1>
<h3>RELEASE</h3>
A model becomes more useful<br>
when people can inspect,<br>
run and build on it.
</td>
</tr>
</table>
</div>
<br>
---
<div align="center">
# SOVYN Ecosystem
<br>
<table>
<tr>
<td align="center" width="25%">
<h3>Models</h3>
Pretrained and<br>
instruction-tuned<br>
SOVYN models.
</td>
<td align="center" width="25%">
<h3>Datasets</h3>
Training and<br>
evaluation<br>
datasets.
</td>
<td align="center" width="25%">
<h3>Research</h3>
Architecture experiments<br>
and technical<br>
reports.
</td>
<td align="center" width="25%">
<h3>Spaces</h3>
Interactive demos<br>
and model<br>
experiments.
</td>
</tr>
</table>
<br><br>
<a href="https://huggingface.co/SOVYNstudio/models">
<img src="https://img.shields.io/badge/VIEW_MODELS-2563EB?style=for-the-badge&logo=huggingface&logoColor=white">
</a>
&nbsp;
<a href="https://huggingface.co/SOVYNstudio/datasets">
<img src="https://img.shields.io/badge/VIEW_DATASETS-0F172A?style=for-the-badge&logo=huggingface&logoColor=white">
</a>
&nbsp;
<a href="https://sovyn.org">
<img src="https://img.shields.io/badge/ENTER_SOVYN-7C3AED?style=for-the-badge&logo=googlechrome&logoColor=white">
</a>
<br><br><br>
</div>
---
<div align="center">
# Development Status
<br>
<table>
<tr>
<td align="center" width="33%">
<h3>SOVYN 1</h3>
<img src="https://img.shields.io/badge/GENERATION-1-334155?style=flat-square">
<br><br>
85M
</td>
<td align="center" width="33%">
<h3>SOVYN 2</h3>
<img src="https://img.shields.io/badge/GENERATION-2-334155?style=flat-square">
<br><br>
300M
</td>
<td align="center" width="34%">
<h3>SOVYN 2.3 Fast</h3>
<img src="https://img.shields.io/badge/CURRENT-MERIDIAN-2563EB?style=flat-square">
<br><br>
~1B
</td>
</tr>
</table>
<br>
<p>
Current work is focused on the SOVYN 2.3 generation.
</p>
</div>
<br>
---
<div align="center">
# Project Meridian
<br>
<pre>
SOVYN 2.3 Fast
│
├── Language
│ ├── Korean
│ ├── English
│ └── Code
│
├── Architecture
│ ├── GQA
│ ├── QK-Norm
│ ├── RoPE
│ ├── SwiGLU
│ └── Local / Global Attention
│
├── Efficiency
│ ├── BF16
│ ├── Gradient Checkpointing
│ ├── 8-bit Optimizer
│ └── Tied Embeddings
│
└── Omni Research
├── Vision
├── Speech
├── Image Generation
└── Speech Generation
</pre>
<br>
<img src="https://img.shields.io/badge/PROJECT-MERIDIAN-172554?style=for-the-badge">
</div>
<br>
---
<div align="center">
# Open Research
<p>
SOVYN aims to publish model artifacts whenever they are ready<br>
and when licensing and technical conditions allow.
</p>
<br>
<table>
<tr>
<td align="center" width="20%">
<h3>Weights</h3>
Model checkpoints<br>
and releases.
</td>
<td align="center" width="20%">
<h3>Configs</h3>
Architecture<br>
specifications.
</td>
<td align="center" width="20%">
<h3>Data</h3>
Dataset sources<br>
and documentation.
</td>
<td align="center" width="20%">
<h3>Eval</h3>
Benchmark<br>
results.
</td>
<td align="center" width="20%">
<h3>Reports</h3>
Technical<br>
notes.
</td>
</tr>
</table>
</div>
<br>
---
<div align="center">
<br>
<img
src="https://capsule-render.vercel.app/api?type=transparent&height=90&text=SOVYN&fontSize=40&fontColor=2563EB"
width="100%"
/>
<h2>Models built for real work.</h2>
<p>
Open models &nbsp;·&nbsp; Independent research &nbsp;·&nbsp; Efficient intelligence
</p>
<br>
<a href="https://sovyn.org">
<img src="https://img.shields.io/badge/SOVYN.ORG-2563EB?style=for-the-badge">
</a>
<br><br>
<sub>
SOVYN Research
</sub>
<br><br>
<img
src="https://capsule-render.vercel.app/api?type=waving&height=120&section=footer&color=0:2563EB,50:172554,100:020617"
width="100%"
/>
</div>