fox3000foxy/ruby-chain

Ruby Chain

Pre-trained Markov chain databases for Ruby, the spontaneous message generator in the Luna Protocol ecosystem.

Trained on 16.9M human messages from mookiezi/Discord-Dialogues — a diverse collection of Discord conversations.

Files

File	Order	Transitions	Starters	Size	Description
chain-order2.db	2	18,955,474	615,483	1.1 GB	Original chain, good variety but less coherent
chain-order3.db	3	6,833,837	788,166	626 MB	Better coherence, smaller footprint
chain-order4.db	4	6,891,633	1,101,029	747 MB	Highest coherence, more unique starters
messages.txt.gz	—	—	—	274 MB	16.9M messages used for training

Recommended: chain-order3.db or chain-order4.db depending on your size vs coherence tradeoff.

Database Schema

CREATE TABLE transitions (
  prefix TEXT,
  suffix TEXT,
  count INTEGER,
  channel_id TEXT,
  PRIMARY KEY (prefix, suffix, channel_id)
);

CREATE TABLE starters (
  prefix TEXT,
  channel_id TEXT,
  PRIMARY KEY (prefix, channel_id)
);

CREATE INDEX idx_trans_prefix ON transitions(prefix);

Prefixes are words joined by \x00. The order determines how many words form a prefix (e.g., order 3 → 3-word prefix).

The channel_id column is always "" (cross-channel aggregate — the source dataset doesn't include channel metadata).

Usage

const Database = require('better-sqlite3');
const db = new Database('chain-order3.db');

// Get a random starter
const starter = db.prepare(
  'SELECT prefix FROM starters ORDER BY RANDOM() LIMIT 1'
).get()?.prefix;

// Sample the chain
function generate(db, prefix, maxLen = 30) {
  const words = prefix.split('\x00');
  for (let i = 0; i < maxLen; i++) {
    const row = db.prepare(
      'SELECT suffix, count FROM transitions WHERE prefix = ? ORDER BY RANDOM()'
    ).all(prefix);
    if (!row.length) break;
    const total = row.reduce((s, r) => s + r.count, 0);
    let roll = Math.random() * total;
    let suffix = row[0].suffix;
    for (const r of row) {
      roll -= r.count;
      if (roll <= 0) { suffix = r.suffix; break; }
    }
    words.push(suffix);
    prefix = words.slice(-3).join('\x00');
  }
  return words.join(' ');
}

Source

Trained with Ruby's tools/train-multi.js — multi-worker Node.js trainer using better-sqlite3. ~300,000 messages/second on 4 CPU cores.