| --- |
| license: mit |
| language: en |
| tags: |
| - scientific-discovery |
| - research-ideas |
| - pytorch |
| --- |
| # Alien coherence and availability models |
|
|
| These are the two core inference models for *The Alien Space of Science: Sampling Coherent but Cognitively Unavailable Research Directions*. Run them with the [public code](https://github.com/alejandrohdez00/alien-space-of-science). |
|
|
| Training data, paper artifacts, and fixed proposals are in [alien-data](https://huggingface.co/datasets/alejandrohdez/alien-data). |
|
|
| ## Contents |
|
|
| - `coherence_model/`: autoregressive transformer, 274 tokens, hidden size 768, six layers, twelve attention heads, maximum sequence length 32. Includes `model.pt` and its token mapping. |
| - `availability_model/`: dual encoder, hidden size 512, six transformer layers, eight heads, FFN size 3072. Includes `model.pt`, its token mapping, the 39,791 × 512 author embedding bank, ID-to-row mapping, and training configuration. Attention key biases are constrained to zero. |
|
|
| The inference files total approximately 358 MB. |
|
|
| ## Use |
|
|
| Download the directories into `models/` in the code repository, install its requirements, and run: |
|
|
| ```bash |
| hf download alejandrohdez/alien-models --include "*_model/*" --local-dir models |
| python alien.py sample --top-k 20 --output-dir results/demo |
| ``` |
|
|
| CPU and CUDA are supported. Sampling does not use an LLM API. Reconstruction is a separate optional command using the user's chosen LiteLLM provider. |
|
|
| Coherence is negative mean atom-token NLL over all set permutations, excluding EOS. Availability is the median of the top ten author cosine similarities. Fusion is `(1-beta)*z(coherence) + beta*z(-availability)` over the candidate population. Each checkpoint has its own token mapping; preserve the files together. |
|
|
| ## Training and limitations |
|
|
| Coherence uses 136,827 training and 15,203 validation records from 15,203 papers. Availability uses 43,077,647 usable training and 4,786,404 validation subset/complement records; the training bank contains 39,791 authors from a corpus with 46,329 author atom profiles. Availability validation is by example, not by held-out author. |
|
|
| The models recombine the fixed 273-atom LLM-research catalogue. Availability measures corpus-relative proximity to author repertoires; it is not proof of originality, feasibility, or future scientific value. Decoder outputs are proposals to assess. CPU and GPU calculations can differ numerically. |
|
|
| Source: the authors' paper corpus and model training runs. File sizes and checksums are in `manifest.json`. The model weights and repository code are released under the MIT license; see `LICENSE`. |
|
|
