| # Architecture |
|
|
| > How vendor sources turn into **70,006** searchable, automatable commands in a |
| > single static HTML file. **FRR rows additionally carry a `live` flag** when |
| > the exact command was observed running on the 10-node Docker FRR lab. |
|
|
|  |
|
|
| --- |
|
|
| ## TL;DR |
|
|
| - **17 vendors / tools, 21 OS surfaces, 70,006 commands** — Cisco (IOS / IOS-XE / NX-OS / ASA), Juniper (Junos), Arista (EOS), FRR (FRRouting), VyOS, Huawei VRP, Aruba AOS-CX, Extreme EXOS, FortiOS, PAN-OS, Mikrotik RouterOS, NVIDIA Cumulus NVUE, Nokia (SR Linux + SR OS), SONiC, plus Microsoft PowerShell, Linux iproute2, and Wireshark tshark. |
| - **Reproducible Python pipeline** — every command can be traced back to a published source via a stdlib-only parser in `scripts/`. |
| - **Single static file** — `index.html` is vanilla JS + a `fetch('commands.json')`. No framework. No build step for the UI. Hosted on GitHub Pages. |
| - **Zero credentials on disk** — automation snippets use `${conn().host}` interpolations driven by a sessionStorage-only connection state with a redact toggle for safe screenshots. |
| - **Coverage-gap tooling** — `fix_coverage_gaps.py` retags NX-OS/IOS-XE and promotes VXLAN/EVPN/STP/BFD/LACP categories; `expand_thin_vendors.py` grows thin surfaces; `patch_yang_stack_vendors.py` + `patch_yang_more_vendors.py` extend Automate templates beyond Cisco fallback; `deep_gap_dig.py` gates OS/cat/Automate floors. |
|
|
| --- |
|
|
| ## Data pipeline (3 stages) |
|
|
| ### 1 · Vendor sources (`scripts/sources/*.md`) |
| |
| All public publications, dropped in as raw markdown. Each vendor has 1–3 books plus the DCN multivendor corpus and modern-ops seed. |
| |
| | Vendor | Sources | Notes | |
| |---|---|---| |
| | **Cisco** | ENCOR / ENARSI Portable CG · ASA 9.24 CLI Reference · IOS OSPF Command Reference · VXLAN BGP EVPN NX-OS Reference · IOS Master Command List | `ios` / `iosxe` / `nxos` / `asa` | |
| | **Juniper** | Junos OS CLI Reference · Day One guides | `junos` | |
| | **Arista** | EOS User Guide (official) | `eos` | |
| | **FRR** | FRR Master CLI Reference ∪ Docker FRR live capture | `frr` + `live`/`in_docs` flags | |
| | **DCN corpus** | VyOS · Huawei · Aruba · Extreme · FortiOS · PAN-OS · RouterOS · NVUE · SR Linux · SONiC · Microsoft · Linux · Wireshark | bulk of mid-tier vendors | |
| | **Modern-ops + gap fills** | `parse_modern.py` · `expand_thin_vendors.py` | Telemetry / Automation / Provisioning / Optics / Hardening + thin-vendor fills | |
| |
| The `scripts/sources/` folder is `.gitignored` — only the resulting JSON is checked in. |
| |
| ### 2 · Parsers (`scripts/parse_*.py`) |
|
|
| One parser per source format. All are pure Python 3 stdlib — no dependencies. |
|
|
| | Parser | Output | |
| |---|---| |
| | `parse_encor.py` / `parse_junos.py` / `parse_arista.py` / `parse_cisco_ospf.py` / `parse_cisco_extras.py` / `parse_frr.py` | per-source JSON | |
| | `merge_dcn_corpus.py` / `parse.py` | merge + dedupe → `commands.json` | |
| | `parse_modern.py` | modern-ops categories | |
| | `expand_thin_vendors.py` | SONiC / NVIDIA / Huawei / SR OS / NX-OS / Extreme / essentials fills | |
| | `fix_coverage_gaps.py` | NX-OS/IOS-XE retag, VXLAN/EVPN/STP/BFD/LACP cats, taxonomy/placeholder cleanup | |
| | `patch_yang_more_vendors.py` | Automate CLI templates for Huawei · NVIDIA · SONiC · Extreme · Mikrotik | |
| | `deep_gap_dig.py` | OS/cat/Automate floor gate (critical regressions fail) | |
| | `clean_titles.py` / `audit_data_quality.py` / `check_consistency.py` | quality + docs drift gate | |
|
|
| ### 3 · `commands.json` |
|
|
| Single source of truth (~18 MB). Each row is: |
|
|
| ```json |
| { |
| "os": "ios", |
| "role": "router", |
| "vendor": "Cisco", |
| "cat": "BGP", |
| "title": "router bgp [ASN]", |
| "cmd": "router bgp 65001\n neighbor 10.0.0.2 remote-as 65002", |
| "desc": "BGP > Configure neighbor" |
| } |
| ``` |
|
|
| OS surfaces include `ios`, `iosxe`, `nxos`, `asa`, `junos`, `eos`, `frr`, `vyos`, `sonic`, `nvue`, `panos`, `srlinux`, `sros`, `fortios`, `routeros`, `exos`, `aoscx`, `vrp`, `powershell`, `linux`, `tshark`. |
|
|
| Dedicated overlay categories: **`VXLAN`**, **`EVPN`**, plus promoted L2/HA surfaces **`Spanning-Tree`**, **`EtherChannel`**, **`BFD`**. |
|
|
| FRR rows may also carry `live` / `in_docs` provenance flags. |
|
|
| --- |
|
|
| ## UI architecture (`index.html`) |
|
|
| ### Automate drawer |
|
|
| 1. **YANG Automate** — 15 patterns (`iface-ipv4`, `static-route`, `ospf-network`, `bgp-neighbor`, `vlan`, …). Templates cover **Cisco · Juniper · Arista · FRR · VyOS · Nokia · Aruba · Huawei · NVIDIA · SONiC · Extreme · Mikrotik** (CLI-first for non-YANG stacks; no silent Cisco fallback). Matchers still key off IOS/Junos-shaped commands; other vendors render when Automate is opened from Compare / equivalents. |
| 2. **SSH Automate** — Netmiko + Ansible + NAPALM for every vendor. `devTypeFor()` prefers **OS-specific** device types (`cisco_nxos`, `cisco_asa`, `cisco_xe`, `nokia_sros`). `ansibleModFor()` selects `cisco.nxos` / `cisco.asa` collections when the queued command is NX-OS or ASA. |
|
|
| ### Parse Output modal |
|
|
| Built-in show-output parsers: |
|
|
| - Cisco IOS — BGP summary, `show ip route`, interface brief, OSPF neighbor |
| - Junos — BGP summary, route terse |
| - **Arista EOS — BGP summary** (PfxRcd / PfxAcc) |
| - **FRR vtysh — BGP summary** (live-lab shape, `vrf-id` / PfxSnt) |
| - **VyOS — BGP summary** (FRR-shaped op-mode) |
|
|
| ### CLI Studio (`studio.html`) |
|
|
| A second static surface for the work the 70k cheatsheet is too wide to teach: English→CLI intent, parameterized recipes, cross-vendor migrate (unified diff), the parsers above, a 10-node FRR Clos map, hardening lint, and a golden-path coverage matrix. Curated corpus in `studio-data.json` (~160 concept-aligned rows). Zero npm. Fetch-once, same HTTP-only contract as `index.html`. |
|
|
| RESTCONF curl and gNMI get snippets sit next to NETCONF / Netmiko / Nornir — model-driven programmability is the 2026 default, not a footnote. |
|
|
| Studio recipes now include **IS-IS L2**, **SR-MPLS node-SID**, and **IPv6 eBGP**. Parse tables export TextFSM / TTP. Harden tab runs NTP/AAA/SSH/syslog/BFD/secret heuristics plus an AEGIS-style pre-change gate (risk, blast radius, rollback). Developer runbook: `docs/DEVELOP.md`. |
|
|
| --- |
|
|
| ## Hosting |
|
|
| ``` |
| push to main |
| └─► GitHub Actions: gh-pages (built-in) |
| └─► https://gesh75.github.io/multivendor-cli-configurator/ |
| ``` |
|
|
| `index.html` + `commands.json` is the cheatsheet. `studio.html` + `studio-data.json` is CLI Studio. Both ship in the lean `_site/` artifact (see `.github/workflows/pages.yml`). `.nojekyll` is required so Ansible `{{ }}` does not trip Jekyll. |
|
|
| --- |
|
|
| ## Regenerating `commands.json` |
|
|
| ```bash |
| cd scripts |
| # 1. Run per-source parsers as needed, then merge |
| python3 parse.py |
| python3 parse_modern.py |
| python3 expand_thin_vendors.py |
| python3 fix_coverage_gaps.py |
| python3 clean_titles.py |
| python3 audit_data_quality.py |
| python3 check_consistency.py |
| python3 deep_gap_dig.py |
| # 2. Optional UI Automate extension (idempotent) |
| python3 patch_yang_stack_vendors.py |
| python3 patch_yang_more_vendors.py |
| ``` |
|
|
| Stdlib only. No virtualenv required. |
|
|
