File size: 2,622 Bytes
5f3ae6a 622e1d7 5f3ae6a 5339378 5f3ae6a 5339378 5f3ae6a 622e1d7 5f3ae6a 622e1d7 5f3ae6a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 | ---
license: mit
language: en
tags:
- scientific-discovery
- research-ideas
- pytorch
---
# Alien coherence and availability models
These are the two core inference models for *The Alien Space of Science: Sampling Coherent but Cognitively Unavailable Research Directions*. Run them with the [public code](https://github.com/alejandrohdez00/alien-space-of-science).
Training data, paper artifacts, and fixed proposals are in [alien-data](https://huggingface.co/datasets/alejandrohdez/alien-data).
## Contents
- `coherence_model/`: autoregressive transformer, 274 tokens, hidden size 768, six layers, twelve attention heads, maximum sequence length 32. Includes `model.pt` and its token mapping.
- `availability_model/`: dual encoder, hidden size 512, six transformer layers, eight heads, FFN size 3072. Includes `model.pt`, its token mapping, the 39,791 × 512 author embedding bank, ID-to-row mapping, and training configuration. Attention key biases are constrained to zero.
The inference files total approximately 358 MB.
## Use
Download the directories into `models/` in the code repository, install its requirements, and run:
```bash
hf download alejandrohdez/alien-models --include "*_model/*" --local-dir models
python alien.py sample --top-k 20 --output-dir results/demo
```
CPU and CUDA are supported. Sampling does not use an LLM API. Reconstruction is a separate optional command using the user's chosen LiteLLM provider.
Coherence is negative mean atom-token NLL over all set permutations, excluding EOS. Availability is the median of the top ten author cosine similarities. Fusion is `(1-beta)*z(coherence) + beta*z(-availability)` over the candidate population. Each checkpoint has its own token mapping; preserve the files together.
## Training and limitations
Coherence uses 136,827 training and 15,203 validation records from 15,203 papers. Availability uses 43,077,647 usable training and 4,786,404 validation subset/complement records; the training bank contains 39,791 authors from a corpus with 46,329 author atom profiles. Availability validation is by example, not by held-out author.
The models recombine the fixed 273-atom LLM-research catalogue. Availability measures corpus-relative proximity to author repertoires; it is not proof of originality, feasibility, or future scientific value. Decoder outputs are proposals to assess. CPU and GPU calculations can differ numerically.
Source: the authors' paper corpus and model training runs. File sizes and checksums are in `manifest.json`. The model weights and repository code are released under the MIT license; see `LICENSE`.
|