src-ece9722/docs/reference/related-work.md

Related work

The methods that NanoSeedLM uses

• SeedLM, Shafipour et al., ICLR 2025. Weights as seeds of a pseudo-random generator. Read SeedLM.
• TransMLA, Meng, Yao and Zhang, 2025. Conversion of GQA to MLA. Read Multi-head latent attention.

Other work on seeds

• Seed-Q. Sensitivity-aware LFSR seed quantization.
• AWSRC. Seeded residual coding on top of INT4.
• SeedLM+O. A measured negative result on small dense models.
• seedlm-repro. A NumPy replication.
• PRANC and NOLA. Networks and adapters from random bases.

Weights that the kernel computes

• QTIP, Tseng, Sun, Hou and De Sa, NeurIPS 2024. A bitshift trellis over a computed Gaussian code, with a randomized Hadamard transform. The kernel computes the weights. It does not look them up in a table. Read the trellis plan.
• ik_llama.cpp trellis quants. Trellis-coded formats (IQ*_KT) with CPU and CUDA kernels in a llama.cpp fork.

Trellis-coded quantization

• Trellis coded quantization of memoryless and Gauss-Markov sources, Marcellin and Fischer, IEEE Transactions on Communications, 1990. The origin of TCQ: a Viterbi search over a trellis, within 0.21 dB of the distortion-rate bound for a uniform source.
• QuIP, Chee, Cai, Kuleshov and De Sa, NeurIPS 2023. Incoherence processing and adaptive rounding (LDLQ): the first usable 2-bit LLMs.
• QuIP#, Tseng, Chee, Sun, Kuleshov and De Sa, 2024. A randomized Hadamard transform, E8 lattice codebooks and BlockLDLQ. QTIP replaces the lattice with a trellis.
• BCJR-QAT, Iyengar, 2026. The trellis search made differentiable (BCJR forward-backward) for quantization-aware training.
• ExLlamaV3. Its EXL3 format is a variant of QTIP, in an inference library for consumer GPUs.
• CCQ, Baidu, 2025. Convolutional codes with scalar quantization at 2.0 to 2.75 bits, with bit-shift decoding and no codebook table. Tested on DeepSeek-V3 and ERNIE-4.5-300B-A47B (MoE models).

Trellis codes from recursions and shift registers

We found no other work that uses an LFSR or SeedLM's seeds as the code of a trellis. These are the nearest.

• The design of trellis waveform coders, Stewart, Gray and Linde, IEEE Transactions on Communications, 1982. The decoder is a shift register and a function of its contents. A bitshift trellis has this structure.
• Linear congruential trellis source codes: design and analysis, Eriksson, Anderson and Goertz, IEEE Transactions on Communications, 2007. A trellis quantizer whose values come from a linear congruential recursion. A 2005 letter by Anderson and Eriksson reports 0.24 dB from the bound for a Gaussian source at 1 bit, with 512 states. These are the ancestors of QTIP's computed codes.
• Viterbi-based efficient test data compression, Lee and Roy, IEEE Transactions on Computer-Aided Design, 2012. The Viterbi algorithm, not linear equations, picks the inputs of a test decompressor. It is the trellis form of a seed search.
• Viterbi-based pruning, Lee, Ahn, Kim, Chuang and Kim, ICLR 2018, and Double Viterbi, Ahn, Lee, Kim and Kim, ICLR 2019. Viterbi encoders of XOR gates and flip-flops give sparse and quantized weights of CNNs and LSTMs.
• Structured compression by weight encryption, Kwon et al., 2019, and encoding weights of irregular sparsity, Park et al., 2021. XOR-gate decoders for pruned and quantized weights.

Discussion

• Hacker News discussion of SeedLM.