| --- |
| license: cc-by-nc-4.0 |
| library_name: pytorch |
| tags: |
| - sudoku |
| - neuro-symbolic |
| - answer-set-programming |
| - attention |
| - pytorch |
| --- |
| |
| # AS²: Attention-Based Soft Answer Set Programming |
|
|
| AS² is a generic attention-based soft answer set framework. See the |
| [paper](https://arxiv.org/html/2603.18436v1) for the method and experiments. |
| The framework combines configurable perception or input encoders, soft symbolic |
| representations, attention-based reasoning, task heads, and differentiable |
| constraint objectives. |
|
|
| **Visual Sudoku is the first bundled instantiation and benchmark.** Its data, |
| board geometry, constraints, objectives, metrics and training recipe are supplied |
| by adapters selected through `configs/instantiation/sudoku.yaml`. The included |
| weights and example belong to that instantiation; they do not define AS²'s scope. |
| Sudoku predictions use raw neural argmax without symbolic repair. |
|
|
| The same train, predict, evaluate and from-scratch commands select instantiations |
| through Hydra. **MNIST Addition and CLEVR-Hans are planned future instantiations**; |
| their runnable implementations are not included in this release. |
| See [Adding an instantiation](docs/instantiations.md) for the extension contract. |
| Each instantiation provides its declarative constraints and their differentiable |
| implementation. An automatic constraint compiler is not included or required. |
|
|
|  |
|
|
| ## Install |
|
|
| Use Python 3.12+ and [uv](https://docs.astral.sh/uv/). |
| Download this entire repository, including the actual model file (not a Git LFS |
| pointer), and run commands from its root: |
|
|
| ```bash |
| uv sync --locked |
| uv run --locked softasp-predict instantiation=sudoku |
| ``` |
|
|
| The first command installs the three local packages and their locked dependencies. |
| PyTorch installation can be large; CUDA training requires a compatible NVIDIA |
| driver. Inference works on CPU; `device=auto` selects CUDA when available. |
|
|
| ## Predict with the Sudoku instantiation |
|
|
| The included example contains one MNIST-derived Sudoku board and its clue mask. |
| No solution labels are supplied to prediction. The command above writes |
| `outputs/prediction.json`, containing digits **1–9** and a structural-validity |
| flag. Structural validity is not an accuracy measurement. Existing prediction |
| files are never overwritten; choose a fresh output path for another run. |
|
|
| ```bash |
| uv run --locked softasp-predict instantiation=sudoku input=examples/sudoku.npz \ |
| checkpoint=models/sudoku/as2.pt output=outputs/my-prediction.json device=cpu |
| ``` |
|
|
| For your own inputs, create an NPZ file with: |
| - `images`: shape `(N,252,252,1)`, uint8 in 0–255 or float32 in 0–1. |
| - `puzzle_mask`: shape `(N,81)`, 1 for visible clues and 0 for blanks. |
|
|
| Cells are in row-major order. Use white digit strokes on black backgrounds; |
| blank cells are black. The model expects aligned cell images, not a photograph |
| of a printed grid. Raw predictions can violate Sudoku constraints. |
|
|
| ## Train the Sudoku instantiation from scratch |
|
|
| This command downloads MNIST, generates 9,000/1,000/1,000 train/validation/test |
| boards, trains a fresh CNN and then AS², and evaluates the resulting model. |
| It does not read the shipped weights. |
|
|
| ```bash |
| uv run --locked softasp-train-scratch instantiation=sudoku pipeline.name=my-run |
| ``` |
|
|
| Both stages allow up to 200 epochs with patience 50; seed is 42 and batch size |
| is 32. The portable default is float32. For a GPU supporting bf16, the recorded |
| training recipe uses: |
|
|
| ```bash |
| uv run --locked softasp-train-scratch instantiation=sudoku pipeline.name=my-gpu-run trainer.precision=bf16 |
| ``` |
|
|
| The backbone inherits the chosen precision. Hydra controls all paths: |
|
|
| ```bash |
| uv run --locked softasp-train-scratch instantiation=sudoku pipeline.name=my-run \ |
| paths.data_root=/path/to/data paths.output_dir=/path/to/outputs \ |
| paths.model_dir=/path/to/models |
| ``` |
|
|
| For a small functionality check, use a fresh run name: |
|
|
| ```bash |
| uv run --locked softasp-train-scratch instantiation=sudoku pipeline.name=smoke \ |
| preprocessing.subset_size=2 trainer.epochs=1 backbone.trainer.epochs=1 \ |
| trainer.batch_size_per_gpu=2 backbone.trainer.batch_size_per_gpu=2 \ |
| trainer.scheduler.warmup_steps=0 backbone.trainer.scheduler.warmup_steps=0 \ |
| trainer.num_workers=0 backbone.trainer.num_workers=0 \ |
| trainer.persistent_workers=false backbone.trainer.persistent_workers=false |
| ``` |
|
|
| This small run is not an accuracy benchmark. Every run records a manifest, |
| resolved configuration, data hashes, checkpoints, and Aim metrics under your |
| output directories. Fresh run/model directories are required; existing runs |
| are not silently resumed or replaced. Models are saved under |
| `models/sudoku/<run-name>/`; the manifest is under |
| `outputs/sudoku/from_scratch/<run-name>/`. |
|
|
| To reuse downloaded MNIST, set `pipeline.download=false` and |
| `paths.raw_data_root=/path/containing/mnist`. A deliberate existing-backbone |
| workflow is also available through `softasp-train`, with |
| `model.pretrained_from` and explicit dataset identity; inference never trains. |
|
|
| ## Evaluate a dataset |
|
|
| Use the HDF5 path and SHA256 recorded in your training manifest. The file must |
| contain train, val, and test groups with images, labels (digits encoded 0–8), |
| and puzzle_mask. No full dataset is included. |
| |
| ```bash |
| uv run --locked softasp-evaluate instantiation=sudoku dataset.splits=[test] \ |
| data_artifact.path=/path/to/dataset.h5 data_artifact.sha256=YOUR_SHA256 \ |
| data_artifact.protocol_id=null evaluator.checkpoint=models/sudoku/as2.pt \ |
| paths.output_dir=outputs/my-evaluation |
| ``` |
| |
| The evaluator writes raw predictions, exact metric counts, and provenance to a |
| fresh output directory. It uses float32 inference without autocast, oracle |
| labels, refinement, or repair. |
| |
| ## Framework layout |
| |
| - `packages/softasp-solver/.../components/`: encoders, evidence policies, structural |
| representations and attention composition. |
| - `packages/softasp-solver/.../constraints/`: shared differentiable operators and residuals. |
| - `packages/softasp-core/`: configuration, training, evaluation and tracking. |
| - `src/softasp/instantiations/`: configuration-driven dispatch and Sudoku providers. |
| - `configs/instantiation/sudoku.yaml`: the bundled instantiation's composition. |
| - Sudoku-named modules contain its data generation, constraints and task-specific behavior. |
| |
| ## Results and limits |
| |
| The included model achieved **955/1,000 fully solved test boards (95.5%)**, |
| **80,905/81,000 correct cells (99.8827%)**, and **45,000/45,000 correct clues**. |
| The selected checkpoint was epoch 187, with 95.4% validation board accuracy. |
| These are single-seed results from the legacy generated Sudoku protocol, using |
| raw CUDA float32 evaluation. Training used bf16 and the default six-layer, |
| 256-dimensional model. |
| |
| Generated boards are transformations of one base solution and use a merged |
| MNIST image pool; source-image separation across dataset splits is not |
| established. These results do not establish generalization to independent |
| handwriting sources, arbitrary Sudoku distributions, or photographed boards. |
| Independent operator/numerical validation and broader paper reproduction are |
| outside this release. Device and precision changes can change predictions. |
| |
| A subsequent full from-scratch run of the refactored framework completed training, |
| checkpoint loading, prediction and evaluation, obtaining **944/1,000 boards (94.4%)** |
| on the same test set. The included selected weights were re-evaluated at **95.5%** |
| and remain unchanged. The 1.1 percentage-point difference is unresolved; its |
| investigation is deferred until after this release. Exact training-result |
| reproduction is not claimed. |
| |
| ## License |
| |
| Project code and model weights are under **CC BY-NC 4.0**; see LICENSE. |
| Dependencies and MNIST source imagery retain their respective third-party |
| terms. The small example uses MNIST digit imagery and is not a newly licensed |
| dataset. See THIRD_PARTY_NOTICES.md. The release includes no full datasets. |
| |
