You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

Model Card for Echo-DSRN-Qwen2.5-0.5B-Hybrid

GitHub License Python Model Collection Hybrid Collection Working Paper

This repository contains experimental models designed strictly for academic evaluation and research purposes.

Critical Constraints:

  • No Production Deployment: Experimental models must not be deployed in commercial, enterprise, or mission-critical environments under any circumstances.
  • No Liability: Experimental models are provided "as-is" without warranties of any kind. The developers assume zero liability for downstream consequences, system integration failures, or regulatory non-compliance resulting from unauthorized deployment.

Echo-DSRN-Hybrid: Transformers Backbone + Surprise-Gated Dual-State Recurrent Architecture

Echo-DSRN-Hybrid is a hybrid recurrent architecture designed for resource-constrained deployment on narrow, well-defined tasks (e.g., intent routing, NER, semantic classification).

The Echo-DSRN memory-injectors combine three parallel computational paths within each block:

  1. Fast GRU state: Tracks short-range token dynamics, updated every token.
  2. Surprise-gated slow state: Selectively accumulates long-range information, write-protected by default and triggered by prediction error.
  3. Sliding window attention: Handles fine-grained local dependencies within a bounded context window (128 tokens).

Echo-DSRN offers constant memory overhead (O(1) recurrent core + bounded O(window_size) attention) during generation.

Downloads last month
-
Safetensors
Model size
0.7B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ethicalabs/Echo-DSRN-Qwen2.5-0.5B-Hybrid

Finetunes
2 models

Datasets used to train ethicalabs/Echo-DSRN-Qwen2.5-0.5B-Hybrid

Collection including ethicalabs/Echo-DSRN-Qwen2.5-0.5B-Hybrid