Prospector-4B / README.md
mjbuehler's picture haiqian-mit's picture
Duplicate from haiqian-mit/Prospector-4B
ca4672c
|
Raw History Blame Contribute Delete
7.96 kB
metadata
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
base_model:
  - Qwen/Qwen3.5-4B
tags:
  - knowledge-graph
  - tool-use
  - graph-traversal
  - question-answering
  - scientific-reasoning

Prospector-4B

Prospector is a compact LLM policy post-trained to answer questions over a scientific knowledge graph. The policy traverses the graph with tools, one call at a time, and answers with a chain of edges that is checked against what the tools returned. Prospector-4B is the largest of the paper's three policies, trained on one graph built from the semiconductor-manufacturing literature. With frozen weights, the policy transfers to graphs from three other fields.

This repository holds the served model as one checkpoint: Qwen3.5-4B with the paper's three LoRA adapters merged into its language model, under the policy's chat template. The checkpoint has Qwen3.5-4B's layout and loads wherever the base model loads. On its own, the model writes one tool call and stops, so the example below drives it through the harness that serves its tools.

Code: https://github.com/lamm-mit/Prospector

Adapters, training rows, stored runs and the semiconductor graph: https://huggingface.co/lamm-mit/Prospector

Example

Clone the GitHub repository first

git clone https://github.com/lamm-mit/Prospector && cd Prospector

Then download the knowledge graph data, 8 MB, from the release

uvx --from huggingface_hub hf download lamm-mit/Prospector --local-dir . --include "data/raw/*"

Install the environment

# on a Mac ...
uv venv --python 3.12 .venv-mac && uv pip install --python .venv-mac/bin/python \
    -e ".[dev,train,mlx,app]" -c launch/constraints-mac.txt && source .venv-mac/bin/activate
# ... or on a Linux machine with an NVIDIA GPU, where CUDA's compiler needs sudo
launch/setup.sh && source .venv/bin/activate

Below is example.py, which asks Prospector-4B one question over the graph:

import argparse
import importlib.util

from graph_reasoner.agent.backend import load_backend, resolve_kb
from graph_reasoner.agent.eval import build_generate_seam
from graph_reasoner.agent.gate import edge_dicts, gate_assertion
from graph_reasoner.agent.loop import EpisodeConfig, drive_episode, start_episode
from graph_reasoner.eval.parse import extract_final_answer, extract_graph_json
from graph_reasoner.schema import BenchmarkItem

QUESTION = ("In semiconductor metrology, which subsystem is typically paired with "
            "high-resolution nanoscale imaging techniques used for structural inspection?")


def default_engine() -> str:
    for engine, package in (("mlx", "mlx_lm"), ("vllm", "vllm")):
        if importlib.util.find_spec(package) is not None:
            return engine
    return "hf"


ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--model", default=f"{default_engine()}:lamm-mit/Prospector-4B")
ap.add_argument("--graph", default="tsmc", help="the substrate: tsmc, or biograph, pfas, matprop once downloaded")
ap.add_argument("--question", default=QUESTION)
args = ap.parse_args()

graph = load_backend(args.graph, resolve_kb(args.graph, None))
cfg = EpisodeConfig()          # the paper's protocol: 20 tool calls, 1,024 tokens a turn, greedy
seam = build_generate_seam(args.model, cfg)


def generate(conversations, max_new_tokens):
    """The harness's generate call, printing the turn it is answering and the turn it gets back."""
    print(conversations[0][-1]["content"], end="\n\n")
    out = seam(conversations, max_new_tokens)
    print(out[0], end="\n\n")
    return out


item = BenchmarkItem(id="example", tier="adhoc", question=args.question, gold_answer="",
                     gold_path={"nodes": [], "edges": []}, subgraph_edges=[], subgraph_text="")
ep = drive_episode(start_episode(item, graph, cfg), graph, generate, cfg)

answer = extract_final_answer(ep.final_text)
cited = extract_graph_json(ep.final_text)
verdict = gate_assertion(answer, edge_dicts(cited.edges if cited else None),
                         edge_dicts(ep.seen.values()))
print(f"answer: {answer}")
print(f"gate: {'passed' if verdict['gate_pass'] else 'failed, ' + verdict['gate_reason']}; "
      f"{ep.n_calls} tool calls, {len(ep.seen)} edges seen")

Download it from this repository and run it from the clone's root:

uvx --from huggingface_hub hf download lamm-mit/Prospector-4B example.py --local-dir .
python example.py

The first run downloads this model from the Hub, about 9 GB. --question asks another question, --graph names another downloaded graph, and --model picks the engine by its prefix: mlx: on Apple silicon, vllm: on an NVIDIA GPU, hf: wherever transformers runs.

Sample output, on a Mac through the mlx: driver:

Question: In semiconductor metrology, which subsystem is typically paired with high-resolution nanoscale imaging techniques used for structural inspection?

Find the entity named in the question, then explore outward.
Budget: 20 tool calls.

<think>Let's search for entities related to semiconductor metrology or high-resolution nanoscale imaging techniques.</think>
<tool_call>{"tool": "find", "args": {"name": "semiconductor metrology"}}</tool_call>

OBSERVATION (call 1/20, find):
FOUND 2 node(s) matching "semiconductor metrology":
  Metrology  [field, 1 connections]
  metrology  [method, 4 connections]
No edges returned — use expand() to see connections.
[0 new edges; 0 edges seen; 19 tool calls left]

<think>Let's search for "nanoscale imaging" or "high-resolution" or "structural inspection" or expand "metrology".</think>
<tool_call>{"tool": "expand", "args": {"node": "metrology"}}</tool_call>

OBSERVATION (call 2/20, expand):
metrology --[has property]--> atomic-scale processes
metrology --[measures]--> profile evolution models
electron microscopy --[is_a]--> metrology
metrology --[measures]--> sub-10 nm hole or trench
[4 new edges; 4 edges seen; 18 tool calls left]

<think>Let's expand "electron microscopy" or search for "nanoscale imaging" / "structural inspection".</think>
<tool_call>{"tool": "expand", "args": {"node": "electron microscopy"}}</tool_call>

OBSERVATION (call 3/20, expand):
STM --[is superior to]--> electron microscopy
optical system --[used with]--> electron microscopy
electron microscopy --[is_a]--> metrology
electron microscopy --[measures]--> number of adhered particles
deposition --[uses]--> electron microscopy
etching rate --[uses]--> electron microscopy
electron microscopy --[compared to]--> STM
[6 new edges; 10 edges seen; 17 tool calls left]

[... twelve more calls, then the final turn's reasoning ...]

<graph_json>{
  "nodes": [
    "optical system",
    "electron microscopy",
    "metrology"
  ],
  "edges": [
    {
      "source": "optical system",
      "relation": "used with",
      "target": "electron microscopy"
    },
    {
      "source": "electron microscopy",
      "relation": "is_a",
      "target": "metrology"
    }
  ]
}</graph_json>
ANSWER: In semiconductor metrology, the **optical system** is typically paired with **electron microscopy** (a high-resolution nanoscale imaging technique used in metrology).

answer: In semiconductor metrology, the **optical system** is typically paired with **electron microscopy** (a high-resolution nanoscale imaging technique used in metrology).
gate: passed; 15 tool calls, 111 edges seen

The code repository's README gives the command behind every number in the paper, and scripts/run_agent_eval.py evaluates this model on a whole benchmark with --model vllm:lamm-mit/Prospector-4B in place of the base model and the adapter chain.

Citation

@article{yang2026prospector,
  title   = {Prospector: learning a compact graph traversal policy for scientific reasoning},
  author  = {Yang, Haiqian and Buehler, Markus J.},
  journal = {arXiv preprint},
  year    = {2026}
}