--- title: KGQA Statistics Lab emoji: 📊 colorFrom: indigo colorTo: blue sdk: static pinned: false short_description: KG, QA, and joint KGQA statistics in the browser datasets: - HalcyonSolutions/Kinship - HalcyonSolutions/MQuAKE-ST --- # KGQA Statistics Lab This static Space computes three complementary views entirely in the browser. 1. **KG analysis** - Graph size, connectivity, density, degree statistics, and relation/entity summaries. - Optional clustering coefficient, eigenvector centrality, and legacy path/hierarchy calculations. 2. **QA analysis** - QA row counts, hop distribution, split distribution, answer multiplicity, paraphrase availability, and reference-path annotation summaries. 3. **Joint KGQA analysis** - Exact fixed-horizon random-walk answer calibration over uniform labeled actions and unique outgoing neighbors. - Optional explicit `SELF_LOOP` action at each node to allow remaining at the current entity for one traversal step. - Optional **Undirected random walk** to add virtual reverse traversal actions, useful for MetaQA. This changes only the joint random-walk action space, not the underlying graph metrics. - Optional expansion of grouped question variants stored in `Question-Paraphrased` when canonical records have a `Question-Number`. - Probability of reaching a released answer and of executing an annotated reference path at a fixed horizon. - Reachable-endpoint counts from the source entity. - Released or reconstructable reference-path structural summaries when available. PQ/PQL `Multi-Paths` takes precedence over `Paths`. ## Built-in sources - **Kinship** - Graph: `kg/orig/triplets.txt` - QA: `qa/kinship_qa_nhop.csv` - **MQuAKE-ST** - Graph: `kg/triplets.txt` - QA Single Answer: `qa/single_answers/qa_nhop.csv` - QA Multi Answer: `qa/multi_answers/qa_nhop.csv` - Relation labels: `metadata/relation_data.csv` - Entity titles: `metadata/node_data.csv` (shown as `Title (QID)` in charts and tables) ### Google Cloud Storage JSON API (v11) Kinship and MQuAKE-ST support Hugging Face file downloads or public GCS dataset ZIP archives. **MetaQA (encoded) is provided only via GCS.** The built-in dataset provider selector applies to both graph and QA selection and is saved in lightweight presets. Existing v10.1/v10.2 JSON presets that lack a provider field default to Hugging Face on import. Archives and paths: | Dataset | Published archive | Expected root | QA file | | --- | --- | --- | --- | | Kinship | `Kinship/kinship_dataset.zip` | `kinship_hinton/` | `qa/kinship_qa_nhop.csv` | | MQuAKE-ST | `MQuAKE_ST/mquake_st_dataset.zip` | `mquake_st_dataset/` | `qa/single_answers/qa_nhop.csv` or `qa/multi_answers/qa_nhop.csv` | | MetaQA | `MetaQA/metaqa_dataset.zip` | `metaqa_dataset/` | `qa/metaqa_nhop.csv` | The download URL uses Google Cloud Storage's **JSON API**: ``` https://storage.googleapis.com/download/storage/v1/b/halcyon_data/o/multihop_ds%2Fdatasets%2FKinship%2Fkinship_dataset.zip?alt=media ``` Google documents that JSON API requests are CORS-accessible independently of the bucket CORS settings. Public object-read permission is still required. No bucket configuration change or authentication is added to the Space. Downloaded ZIPs are decoded in a Web Worker with locally bundled JSZip. The same archive is fetched once per browser tab session, even across repeated analyses, while the session cache is retained. Hugging Face dataset files and their metadata are also reused. Use **Clear cached datasets** to release downloads and fetch fresh copies on the next analysis. The cache uses in-memory storage only and is discarded when the page is closed or reloaded, but not when an analysis is canceled. Public archives may be substantially larger than individual Hugging Face files, requiring more browser memory. The GCS endpoint and public object accessibility still need to be tested from the deployed Space. Switching back to Hugging Face remains available for Kinship/MQuAKE-ST. Source: [Google Cloud Storage CORS](https://docs.cloud.google.com/storage/docs/cross-origin) and [JSON API download endpoint](https://docs.cloud.google.com/storage/docs/request-endpoints). ## Graph and QA compatibility The selected graph restricts which QA collections are available for **joint KGQA analysis**: | Graph source | Available QA sources | | --- | --- | | Kinship | Kinship QA, No QA | | MQuAKE-ST | MQuAKE-ST Single Answer, MQuAKE-ST Multi Answer, No QA | | MetaQA (encoded) | MetaQA QA, No QA | | Custom graph | Custom QA, No QA | | No graph | Kinship QA, MQuAKE-ST Single Answer, MQuAKE-ST Multi Answer, MetaQA QA, Custom QA, No QA | Incompatible QA options remain visible but disabled. If a graph change makes the current QA source incompatible, the Lab automatically selects the matching QA source. An intentional **No QA** selection remains unchanged. No Graph supports QA-only analysis with any QA collection, while No QA supports graph-only analysis with any graph. Selecting both No Graph and No QA is not a runnable analysis. ## Lightweight analysis presets Use **Export preset** and **Import preset** to save or restore a setup. The Save / Restore Setup panel appears above **01 / Knowledge Graph** for quick access. - A preset is a small `.kgqa-preset.json` file containing the selected graph and QA sources, **download provider**, options, dashboard preferences, column mappings, and references to custom input files. **The data files themselves are not included.** - For local files selected normally, browsers disclose the filename but **not the absolute filesystem path**. The preset therefore stores a filename reference (and file size). A folder-selected file may also expose a relative path. A local absolute path cannot be read or automatically reopened by a normal browser page. - On import, analysis settings are restored immediately. Any required local file that has not already been selected must be reconnected, using **Select folder to reconnect** or the individual file inputs. The Lab matches filename and byte size, and uses relative paths when available. Multiple ambiguous matches are not selected automatically. - Metadata column selections are restored once the corresponding metadata file is reconnected. The Lab clearly identifies outstanding files and does not claim the setup is ready until those references are resolved. - Presets do not include results. Files never leave your browser as part of the import/export workflow. Do not rename or move referenced files without reconnecting them. - Existing v10 presets in `.zip` format are not directly compatible with the lightweight `.json` preset importer. Export a new preset after upgrading. ### MetaQA MetaQA uses the encoded THESEUS graph, QA collection, and metadata files. It has no released reference navigation paths. Path-dependent measures that require those annotations are unavailable. For random-walk comparisons, enable **Undirected random walk** if the desired evaluation action space contains reverse edges. ## Custom inputs ### Graph - Triplets file with three columns: head, relation, tail. - **Explicit Entity list / Node labels** is one optional input. It accepts one ID per line, or a node metadata CSV such as `QID,Title`, `EID,Name`, or `EntityID,Label`. For metadata CSVs, choose which column is the entity ID and which is the display title. By default, the file is used for labels only and does **not** add graph nodes. Enable **Include metadata-only entities (isolated nodes)** if the selected file intentionally defines additional graph nodes. - Optional relation-label file. Files such as `Property,Title` or `RID,Title` are supported with independent ID and display-label column choices. - For plain ID lists, no title is available, so nodes display their entity IDs. The ID-column selection is applied to chart labels. When the opt-in isolated-node setting is enabled, it also determines which metadata IDs are included in the graph analysis. ### QA Recommended columns: - `Question` - `Source-Entity` - `Answer-Entity` - `Hops` Optional columns: - `SplitLabel` - `Question-Paraphrased` - `Paths` - `Multi-Paths` / `Multi-Paths-Label` (PQ/PQL released grouped paths) - `Multi-Paths-Key` (complete relation chains, aligned by group) - `Path-Key` - `Path-Count` (released path count) - `Graph-Answer-Entity` / `Graph-Answer` (graph-consistent alternatives, not released gold) - `Graph-Multi-Paths` / `Graph-Multi-Paths-Label` / `Graph-Path-Count` (graph-relative paths) - `Question-Number` - `Question-Family-ID` / `Question-Family-Size` (PQ/PQL explicit structural family) ## Notes - All processing happens client-side. - By default, graph entities are exactly the head and tail IDs observed in triplets. Entity metadata does not change graph population or statistics. This is important for filtered PQ/PQL graphs whose metadata may list nodes from a much larger source graph. - Opting in to metadata-only nodes changes graph size, zero-degree counts, degree distributions, weak components, and density. Use the opt-in only for graph definitions that intentionally include those isolated nodes. - PQ/PQL `Multi-Paths` contains complete paths grouped by relation chain. The Lab flattens and deduplicates released entity paths for reference-state analysis while preserving `Path-Count` separately. It does not merge `Graph-Multi-Paths` into the released evidence set. - The joint report requires both a graph and a QA collection. - The split filter is applied to the QA rows before reporting. - For joint random-walk calibration, dead ends are absorbing. ## Dashboard metric views The top dashboard lets you choose between **Instance-micro** and **Family-macro** whenever the QA data provides family identifiers and aggregation is available. - **Instance-micro** gives each evaluated QA instance equal weight. - **Family-macro** averages instances within a structural family, then averages those family values equally. - **Answer (released)** evaluates `Answer-Entity`, the reference-answer set. - **Graph-Answer (graph-consistent)** evaluates `Graph-Answer-Entity`, when available. These alternatives are derived from the graph and are not independently verified as gold natural-language answers. The selectors affect **top summary cards only**. Detailed reports and JSON exports keep the underlying statistics unchanged and include both views. The QA split defaults to **Test only**. If a QA collection lacks a `SplitLabel` column, all of its rows are included. ## Random-walk settings - Use the same graph, benchmark split, and step horizon when comparing results across experiments. - **Explicit SELF_LOOP actions** allow the random walker to stay at its current entity and consume one step. - **Undirected random walk** adds a reverse traversal for each unique directed triple. Original and reverse actions remain distinct when both occur in the graph. Graph statistics always describe the original triples. - **Expand QA paraphrases** starts unchecked. Enable it to expand grouped alternative wordings when question-number families are available. QA collections with an explicit `Question-Family-ID`, including PQ/PQL, are not duplicated. - Random-walk dead ends are absorbing. ## Definitions and help Hover over an underlined metric name for a short definition. Descriptions are also available when focusing supported terms with the keyboard. Key quantities include: - **RW-Ans@p-actions**: fixed-horizon probability of reaching any target answer when uniformly sampling labeled traversal actions. - **RW-Ans@p-neighbors**: the same target probability when uniformly sampling unique neighboring entities. - **RW-RefPath@p-actions**: probability of executing an annotated reference path with any required self-loop padding. - **Weakly connected component (WCC)**: a connected region after ignoring graph-edge direction. - **Reachable-N-Count**: number of possible destinations after exactly N random-walk steps. ## Numerical scope The Lab computes graph summaries and exact fixed-horizon random-walk distributions locally in your browser. Graph and QA sizes may affect processing time and memory use. Expensive optional graph metrics may be skipped under built-in size safeguards. ## PQ/PQL local QA notes - Use **Custom QA** to upload the reconstructed `pq_qa_nhop.csv` or `pql_qa_nhop.csv`. - Upload the **active filtered or source graph** as Custom graph. The chosen graph determines random-walk probabilities and whether released reference paths are executable. `Graph-Answer-Entity`, `Graph-Multi-Paths`, and `Graph-Path-Count` are supplied by the uploaded QA CSV. This Space does not recompute them for a different graph. - `Answer-Entity` is the primary released-answer target. `Graph-Answer-Entity` is evaluated separately when present and must not be treated as independently verified gold. - The **Multi-Paths** field supplies released paths in grouped form. `Multi-Paths-Key` is a list of complete relation chains corresponding to the groups. - `Path-Count` and `Graph-Path-Count` are retained as separate metrics, as are `Multi-Paths` and `Graph-Multi-Paths`. - Family-macro results use the released `Question-Family-ID`. PQ/PQL already explicitly represent paraphrase variants as individual rows. ## Tooltip behavior Technical terms use a single delayed description bubble on hover or keyboard focus. The browser-native duplicate tooltip has been removed. ### v11.1: session download cache The worker stays alive after a successful analysis. Built-in GCS ZIP archives and Hugging Face graph/QA files are stored in memory and reused when the same source is analyzed again. The UI shows the number and compressed/downloaded byte size of the cached resources and has a **Clear cached datasets** button. Clearing the cache ends the idle worker, frees its archives, clears metadata responses, and causes the next run to fetch them again. Reloading or closing the page also drops the cache. Canceling an in-progress analysis preserves completed downloads. This is not browser disk storage, so reopening the application does not retain downloads. Each file is fetched once per session unless cleared. Different providers have separate cache entries. The displayed sizes are download sizes, not a guarantee of total browser memory usage. Custom local uploads are not copied into this download cache. ### v11.3: cancellation preserves cached datasets Canceling an analysis terminates the active calculation Worker so computational work stops immediately. Completed built-in downloads are backed up as `Blob` objects in page memory and restored to the next Worker, avoiding another fetch. This includes Google Cloud Storage archives and individual Hugging Face dataset files. A download still in progress or an archive not yet validated at the instant of cancellation is not retained. **Clear cached datasets** is the explicit way to discard them, and reloading or closing the page also releases the cache. No persistent disk storage is used. ## v11.3: selective cache deletion and optional random walks - Downloaded Hugging Face files and GCS archives have different cache entries even when they represent the same dataset. Switching provider downloads the requested provider's resource once, without discarding the first provider's cache. - Each cached resource is listed with its provider, filename and size, and may be deleted separately with **Delete**. **Clear all cached datasets** removes the whole in-memory cache. Cache operations are unavailable while an analysis is running. Removing an entry restarts the idle Worker so it cannot reuse a deleted item, while retaining all other completed page-owned downloads. - **Calculate random-walk probabilities** is checked by default, preserving earlier behavior. Uncheck it to skip expensive fixed-horizon random-walk distributions, reachability, and reference-path probabilities, while retaining KG-only, QA-only, joint answer and annotated-path structure metrics. Report fields for skipped random-walk metrics are null rather than zero. - Existing saved presets without the new checkbox restore with random-walk calculation enabled. New presets remember the checkbox. - The cache is in memory for the lifetime of the page only. Reloading or closing the tab clears it.