docs/ARCHITECTURE.md

Architecture

How vendor sources turn into 70,006 searchable, automatable commands in a single static HTML file. FRR rows additionally carry a live flag when the exact command was observed running on the 10-node Docker FRR lab.

---

TL;DR

• 17 vendors / tools, 21 OS surfaces, 70,006 commands — Cisco (IOS / IOS-XE / NX-OS / ASA), Juniper (Junos), Arista (EOS), FRR (FRRouting), VyOS, Huawei VRP, Aruba AOS-CX, Extreme EXOS, FortiOS, PAN-OS, Mikrotik RouterOS, NVIDIA Cumulus NVUE, Nokia (SR Linux + SR OS), SONiC, plus Microsoft PowerShell, Linux iproute2, and Wireshark tshark.
• Reproducible Python pipeline — every command can be traced back to a published source via a stdlib-only parser in scripts/.
• Single static file — index.html is vanilla JS + a fetch('commands.json'). No framework. No build step for the UI. Hosted on GitHub Pages.
• Zero credentials on disk — automation snippets use ${conn().host} interpolations driven by a sessionStorage-only connection state with a redact toggle for safe screenshots.
• Coverage-gap tooling — fix_coverage_gaps.py retags NX-OS/IOS-XE and promotes VXLAN/EVPN/STP/BFD/LACP categories; expand_thin_vendors.py grows thin surfaces; patch_yang_stack_vendors.py + patch_yang_more_vendors.py extend Automate templates beyond Cisco fallback; deep_gap_dig.py gates OS/cat/Automate floors.

---

Data pipeline (3 stages)

1 · Vendor sources (scripts/sources/*.md)

All public publications, dropped in as raw markdown. Each vendor has 1–3 books plus the DCN multivendor corpus and modern-ops seed.

Vendor	Sources	Notes
Cisco	ENCOR / ENARSI Portable CG · ASA 9.24 CLI Reference · IOS OSPF Command Reference · VXLAN BGP EVPN NX-OS Reference · IOS Master Command List	ios / iosxe / nxos / asa
Juniper	Junos OS CLI Reference · Day One guides	junos
Arista	EOS User Guide (official)	eos
FRR	FRR Master CLI Reference ∪ Docker FRR live capture	frr + live/in_docs flags
DCN corpus	VyOS · Huawei · Aruba · Extreme · FortiOS · PAN-OS · RouterOS · NVUE · SR Linux · SONiC · Microsoft · Linux · Wireshark	bulk of mid-tier vendors
Modern-ops + gap fills	parse_modern.py · expand_thin_vendors.py	Telemetry / Automation / Provisioning / Optics / Hardening + thin-vendor fills

The scripts/sources/ folder is .gitignored — only the resulting JSON is checked in.

2 · Parsers (scripts/parse_*.py)

One parser per source format. All are pure Python 3 stdlib — no dependencies.

Parser	Output
parse_encor.py / parse_junos.py / parse_arista.py / parse_cisco_ospf.py / parse_cisco_extras.py / parse_frr.py	per-source JSON
merge_dcn_corpus.py / parse.py	merge + dedupe → commands.json
parse_modern.py	modern-ops categories
expand_thin_vendors.py	SONiC / NVIDIA / Huawei / SR OS / NX-OS / Extreme / essentials fills
fix_coverage_gaps.py	NX-OS/IOS-XE retag, VXLAN/EVPN/STP/BFD/LACP cats, taxonomy/placeholder cleanup
patch_yang_more_vendors.py	Automate CLI templates for Huawei · NVIDIA · SONiC · Extreme · Mikrotik
deep_gap_dig.py	OS/cat/Automate floor gate (critical regressions fail)
clean_titles.py / audit_data_quality.py / check_consistency.py	quality + docs drift gate

3 · commands.json

Single source of truth (~18 MB). Each row is:

{
  "os":     "ios",
  "role":   "router",
  "vendor": "Cisco",
  "cat":    "BGP",
  "title":  "router bgp [ASN]",
  "cmd":    "router bgp 65001\n neighbor 10.0.0.2 remote-as 65002",
  "desc":   "BGP > Configure neighbor"
}

OS surfaces include ios, iosxe, nxos, asa, junos, eos, frr, vyos, sonic, nvue, panos, srlinux, sros, fortios, routeros, exos, aoscx, vrp, powershell, linux, tshark.

Dedicated overlay categories: VXLAN, EVPN, plus promoted L2/HA surfaces Spanning-Tree, EtherChannel, BFD.

FRR rows may also carry live / in_docs provenance flags.

---

UI architecture (index.html)

Automate drawer

1. YANG Automate — 15 patterns (iface-ipv4, static-route, ospf-network, bgp-neighbor, vlan, …). Templates cover Cisco · Juniper · Arista · FRR · VyOS · Nokia · Aruba · Huawei · NVIDIA · SONiC · Extreme · Mikrotik (CLI-first for non-YANG stacks; no silent Cisco fallback). Matchers still key off IOS/Junos-shaped commands; other vendors render when Automate is opened from Compare / equivalents.
2. SSH Automate — Netmiko + Ansible + NAPALM for every vendor. devTypeFor() prefers OS-specific device types (cisco_nxos, cisco_asa, cisco_xe, nokia_sros). ansibleModFor() selects cisco.nxos / cisco.asa collections when the queued command is NX-OS or ASA.

Parse Output modal

Built-in show-output parsers:

• Cisco IOS — BGP summary, show ip route, interface brief, OSPF neighbor
• Junos — BGP summary, route terse
• Arista EOS — BGP summary (PfxRcd / PfxAcc)
• FRR vtysh — BGP summary (live-lab shape, vrf-id / PfxSnt)
• VyOS — BGP summary (FRR-shaped op-mode)

CLI Studio (studio.html)

A second static surface for the work the 70k cheatsheet is too wide to teach: English→CLI intent, parameterized recipes, cross-vendor migrate (unified diff), the parsers above, a 10-node FRR Clos map, hardening lint, and a golden-path coverage matrix. Curated corpus in studio-data.json (~160 concept-aligned rows). Zero npm. Fetch-once, same HTTP-only contract as index.html.

RESTCONF curl and gNMI get snippets sit next to NETCONF / Netmiko / Nornir — model-driven programmability is the 2026 default, not a footnote.

Studio recipes now include IS-IS L2, SR-MPLS node-SID, and IPv6 eBGP. Parse tables export TextFSM / TTP. Harden tab runs NTP/AAA/SSH/syslog/BFD/secret heuristics plus an AEGIS-style pre-change gate (risk, blast radius, rollback). Developer runbook: docs/DEVELOP.md.

---

Hosting

push to main
  └─► GitHub Actions: gh-pages (built-in)
       └─► https://gesh75.github.io/multivendor-cli-configurator/

index.html + commands.json is the cheatsheet. studio.html + studio-data.json is CLI Studio. Both ship in the lean _site/ artifact (see .github/workflows/pages.yml). .nojekyll is required so Ansible {{ }} does not trip Jekyll.

---

Regenerating commands.json

cd scripts
# 1. Run per-source parsers as needed, then merge
python3 parse.py
python3 parse_modern.py
python3 expand_thin_vendors.py
python3 fix_coverage_gaps.py
python3 clean_titles.py
python3 audit_data_quality.py
python3 check_consistency.py
python3 deep_gap_dig.py
# 2. Optional UI Automate extension (idempotent)
python3 patch_yang_stack_vendors.py
python3 patch_yang_more_vendors.py

Stdlib only. No virtualenv required.