HF_README.md

metadata

license: mit
library_name: pytorch
tags:
  - world-models
  - spectral-learning
  - operator-learning
  - multimodal
  - wavelets
  - pytorch
  - research

Spectral World Models — small runnable prototype

This is a collaborator-oriented Hugging Face prototype of Spectral World Models: Multimodal Operator Learning in Hilbert Space for Imagination-Based Planning by Andrew J. Kiruluta.

The implementation is grounded in the accompanying manuscript and source repository. It uses a fixed 2-level Haar image encoder, DCT-style spectral text encoder, low-rank multimodal fusion, an action-conditioned low-rank spectral transition operator, modality decoders, and recursive imagination rollouts.

First benchmark

The included Mini Moving Shapes dataset is a deliberately small multimodal smoke benchmark: 32×32 grayscale moving shapes, symbolic text descriptions, and five discrete actions. The repository compares SWM with Dreamer-style, transformer, Koopman, and neural-operator baselines and includes architecture ablations.

The checked-in results are actual quick CPU measurements from the source repository, not the hypothetical tables in the manuscript. They are intended to validate the pipeline, not establish superiority.

Method	Params	Test PSNR ↑	Test SSIM ↑	Test NLL ↓	Rollout MSE@5 ↓
Dreamer-style	497,682	14.571	0.072	0.0107	0.03490
Transformer	503,322	14.368	0.017	0.0687	0.03657
Koopman	465,306	14.619	0.090	-0.0027	0.03487
Neural operator	403,794	14.534	0.058	0.0210	0.03511
SWM	405,422	14.591	0.080	0.0050	0.03577

These smoke results are mixed: SWM is competitive on some one-step metrics, while Koopman has lower rollout MSE@5 in this short run. That is precisely why larger multi-seed experiments are needed.

Run

python -m venv .venv

source .venv/bin/activate

pip install -e .

PYTHONPATH=. python scripts/run_quick_benchmark.py

Tests: PYTHONPATH=. python -m pytest -q

See COLLABORATORS.md for the GPU scaling and experimental roadmap.