| ## Expressivity-Efficiency Tradeoffs for Hybrid Sequence Models | |
| This repository includes all of the experiments relevant to the associated paper. | |
| - `micro`: A collection of old experiments with pytorch transformers and mamba. Not currently used, but might be useful as a reference. | |
| - `micro_hf`: A collection of experiments for micro-scale experiments, using huggingface models. `run.ipynb` will execute the experiments, and `process.ipynb` can be used to analyze the results. Each individual run does not take long to run on an RTX 4090. | |
| - `mini`: A collection of larger experiments, mostly ported from the ones in the Repeat After Me paper. Added to this are hybrid models implemented in huggingface. |