Add frozen-model Gooo feedback pilot evidence; preserve weights and v5 core
Browse files- research/feedback-path-pilot-20261001/README.md +66 -0
- research/feedback-path-pilot-20261001/audit.json +30 -0
- research/feedback-path-pilot-20261001/preexecution.json +29 -0
- research/feedback-path-pilot-20261001/publication-manifest.json +33 -0
- research/feedback-path-pilot-20261001/report.json +0 -0
- research/feedback-path-pilot-20261001/source-input-pins.json +12 -0
research/feedback-path-pilot-20261001/README.md
ADDED
|
@@ -0,0 +1,66 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Frozen-model feedback judgment pilot
|
| 2 |
+
|
| 3 |
+
This appendix reuses the existing three tiny structural models. No new weights,
|
| 4 |
+
optimizer steps, GPU calls or upstream Laya calls occurred. The Go research
|
| 5 |
+
session explicitly asks the original frozen model after a partial batch using
|
| 6 |
+
observed finite failures and actual native-compiler CI context.
|
| 7 |
+
|
| 8 |
+
The native compiler's post-main CI is source `307159f041644a3aa56dfd325c345f5325aec902`,
|
| 9 |
+
run `36801794319`, with six passed checks and Go-verified proof/receipt. This
|
| 10 |
+
context is a judgment hint; it is not authority to change source or proof that
|
| 11 |
+
the newer research runner itself passed CI.
|
| 12 |
+
|
| 13 |
+
## Measured result
|
| 14 |
+
|
| 15 |
+
| Observation | Result |
|
| 16 |
+
| --- | --- |
|
| 17 |
+
| Policies / paired comparisons | 32 / 16 |
|
| 18 |
+
| Different compound intentions | 1 |
|
| 19 |
+
| Same emitted Go / same finite results | 16 / 16 pairs |
|
| 20 |
+
| Actual local predictions | 246, including 102 feedback predictions |
|
| 21 |
+
| Feedback rounds | 17 |
|
| 22 |
+
| Candidate attempts | 1,268 |
|
| 23 |
+
| Finite selection cases | 208/224 across repeated policy views |
|
| 24 |
+
| Separate input evaluation | 288/288 observations of 9 existing distinct inputs |
|
| 25 |
+
| Native codegen calls | 32 |
|
| 26 |
+
| Distinct generated-Go/input-vector executions | 2, reused by byte/input hash |
|
| 27 |
+
| Pairs with fewer / more / equal candidates | 1 / 2 / 13 |
|
| 28 |
+
| Model-policy median search wall time | 1.888ms baseline; 2.222ms feedback |
|
| 29 |
+
|
| 30 |
+
Complete contracts satisfy 7/7 stated cases. Deliberately inconsistent contracts
|
| 31 |
+
retain 6/7 and exhaust all 64 paths; reconsideration cannot manufacture a valid
|
| 32 |
+
answer for the conflicting expected value. Korean QAT completion used 19→17
|
| 33 |
+
candidates; Korean FP32 used 10→11 and PTQ used 17→18. No aggregate functional
|
| 34 |
+
improvement occurred. The original model was not trained to interpret this
|
| 35 |
+
failure summary. Extra calls and worse observations remain visible.
|
| 36 |
+
|
| 37 |
+
All policies used the same compound intention, two language views, four model
|
| 38 |
+
arms including disconnected controls and two contracts. Baseline runs came
|
| 39 |
+
first, there was one ordering and no repetitions. Search timings exclude model
|
| 40 |
+
load, native generation, Go compilation and process startup. CPU/RSS and host
|
| 41 |
+
utilization changes were not measured in this pilot. These data do not establish
|
| 42 |
+
general natural-language accuracy, a causal speedup, 32 new experiment ideas or
|
| 43 |
+
288 independent language tasks. No-model controls reproduce their progress and
|
| 44 |
+
source exactly despite the enabled control flag; they make zero model calls.
|
| 45 |
+
|
| 46 |
+
## Reproduction and preserved failures
|
| 47 |
+
|
| 48 |
+
The public GitHub repository is
|
| 49 |
+
[gooo-neural-decision-experiments](https://github.com/kimjooyoon/gooo-neural-decision-experiments).
|
| 50 |
+
Runner source is `63c9967505355ccc5c56780f901f4ce583008f17`;
|
| 51 |
+
full captures are in `runs/feedback-path-pilot-fixed-20261001`.
|
| 52 |
+
The audit re-interprets all 1,268 candidate bodies and checks progress, feedback,
|
| 53 |
+
native and executed-Go captures, with zero new model or native calls.
|
| 54 |
+
|
| 55 |
+
The earlier `runs/feedback-path-pilot-20261001` attempt is preserved. It made
|
| 56 |
+
zero model predictions and one rejected native call because the runner nested
|
| 57 |
+
a complete Gooo module inside another body. The compiler remained unchanged;
|
| 58 |
+
the repaired runner passes the complete module directly. A separate wrong
|
| 59 |
+
revision preflight was rejected before any model or native call.
|
| 60 |
+
|
| 61 |
+
This directory is a separately allowlisted **evidence appendix**. The previous
|
| 62 |
+
v5 core manifest, model card and nine weight files remain unchanged. Raw captures
|
| 63 |
+
and implementation live on GitHub. The new appendix includes only synthetic
|
| 64 |
+
intentions and observations; credentials, private repositories and host paths
|
| 65 |
+
are excluded. First-shot accuracy and perfect expression are not prerequisites
|
| 66 |
+
for publishing an observed partial result.
|
research/feedback-path-pilot-20261001/audit.json
ADDED
|
@@ -0,0 +1,30 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"actual_model_predictions": 246,
|
| 3 |
+
"candidate_attempts": 1268,
|
| 4 |
+
"candidate_count_increased_pairs": 2,
|
| 5 |
+
"candidate_count_reduced_pairs": 1,
|
| 6 |
+
"candidate_count_unchanged_pairs": 13,
|
| 7 |
+
"cpu_or_rss_measured": false,
|
| 8 |
+
"decision": "PASS",
|
| 9 |
+
"distinct_evaluation_inputs": 9,
|
| 10 |
+
"executed_disjoint_input_observations": 288,
|
| 11 |
+
"executed_disjoint_input_passed": 288,
|
| 12 |
+
"executed_finite_cases": 224,
|
| 13 |
+
"executed_finite_passed": 208,
|
| 14 |
+
"feedback_model_predictions": 102,
|
| 15 |
+
"feedback_rounds": 17,
|
| 16 |
+
"independently_interpreted_candidate_bodies": 1268,
|
| 17 |
+
"model_policy_median_base_search_ms": 1.887625,
|
| 18 |
+
"model_policy_median_feedback_search_ms": 2.222083,
|
| 19 |
+
"native_calls": 32,
|
| 20 |
+
"new_native_calls_by_audit": 0,
|
| 21 |
+
"new_predictions_by_audit": 0,
|
| 22 |
+
"pairs": 16,
|
| 23 |
+
"policies": 32,
|
| 24 |
+
"report_sha256": "050f288116b973aeb5198d35fcf9c085b178189868243eceb67a86cb0d32048e",
|
| 25 |
+
"same_emitted_go_pairs": 16,
|
| 26 |
+
"same_finite_and_evaluation_pairs": 16,
|
| 27 |
+
"schema": "gooo/feedback-path-pilot-audit/v1",
|
| 28 |
+
"scope": "Fixed one-intent pilot, one baseline-first ordering and no repetitions. Finite selection cases and existing disjoint function inputs are separate; repeated observations are not 288 independent tasks. No generic NL improvement or causal speedup claim. Audit re-interprets candidates and hashes captured native/Go outputs; it does not re-execute native or generated Go or retrain the model.",
|
| 29 |
+
"unique_generated_go_input_executions": 2
|
| 30 |
+
}
|
research/feedback-path-pilot-20261001/preexecution.json
ADDED
|
@@ -0,0 +1,29 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"attempt_step": 8,
|
| 3 |
+
"ci_hint": {
|
| 4 |
+
"source_sha": "307159f041644a3aa56dfd325c345f5325aec902",
|
| 5 |
+
"status": "PASS"
|
| 6 |
+
},
|
| 7 |
+
"ci_hint_publication_sha256": "4710e8ca79b29596b817b5da5fc202004b341f3992de7787466630397d257870",
|
| 8 |
+
"compound_intents": 1,
|
| 9 |
+
"evaluation_inputs": 9,
|
| 10 |
+
"maximum_candidates": 64,
|
| 11 |
+
"maximum_feedback_rounds": 2,
|
| 12 |
+
"model_metadata": {
|
| 13 |
+
"fp32": "1ea3bada068f2487418f17270db4ba6785a98c3ad60e0ad7eb92359400a01bb5",
|
| 14 |
+
"ptq_ternary": "7c4eb4068d76e83620a9b8e7ce7f8d948b5e2436d44c7b79cf241da609dab894",
|
| 15 |
+
"qat_ternary": "368e37e7899cecba03874b69e933525a2f8c90ecc53d787d8e698de6aa21c087"
|
| 16 |
+
},
|
| 17 |
+
"model_weights": {
|
| 18 |
+
"fp32": "3732a103720fdb0da0ea611471ecde7472e0b9166322394ba9352ffb855b9354",
|
| 19 |
+
"ptq_ternary": "01a6574dedba327361cd3e4fda00a4d4dea34e162aab09885174bcd41ed4da47",
|
| 20 |
+
"qat_ternary": "456c686911b95bb933a7dd9275d2611974482ecf353831ab17b593c6bd3d86de"
|
| 21 |
+
},
|
| 22 |
+
"native_revision": "307159f041644a3aa56dfd325c345f5325aec902",
|
| 23 |
+
"optimizer_steps": 0,
|
| 24 |
+
"paired_comparisons": 16,
|
| 25 |
+
"policies": 32,
|
| 26 |
+
"runner_revision": "63c9967505355ccc5c56780f901f4ce583008f17",
|
| 27 |
+
"schema": "gooo/feedback-path-pilot-preexecution/v1",
|
| 28 |
+
"scope": "Same frozen compound intent, two languages, complete and inconsistent contracts, three models and offline controls; neither new independent ideas nor a feedback-trained model. CI hint is previously verified snapshot context, not current study authorization."
|
| 29 |
+
}
|
research/feedback-path-pilot-20261001/publication-manifest.json
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "gooo/public-feedback-pilot-allowlist/v1",
|
| 3 |
+
"files": [
|
| 4 |
+
{
|
| 5 |
+
"path": "README.md",
|
| 6 |
+
"sha256": "21fe36be9e1e3018c38dee0b3093e9409683482449cf32e0cdfd75f0f0d506ff",
|
| 7 |
+
"bytes": 3709
|
| 8 |
+
},
|
| 9 |
+
{
|
| 10 |
+
"path": "audit.json",
|
| 11 |
+
"sha256": "4574d172b38fa563b45a1b8d5138c0e4007e67013764fa59ce5b9cc920187d56",
|
| 12 |
+
"bytes": 1428
|
| 13 |
+
},
|
| 14 |
+
{
|
| 15 |
+
"path": "preexecution.json",
|
| 16 |
+
"sha256": "966b93ebc6aca754b4174ebfbb931e903c1c2489e18629884208e29cf9427a03",
|
| 17 |
+
"bytes": 1425
|
| 18 |
+
},
|
| 19 |
+
{
|
| 20 |
+
"path": "report.json",
|
| 21 |
+
"sha256": "050f288116b973aeb5198d35fcf9c085b178189868243eceb67a86cb0d32048e",
|
| 22 |
+
"bytes": 4089826
|
| 23 |
+
},
|
| 24 |
+
{
|
| 25 |
+
"path": "source-input-pins.json",
|
| 26 |
+
"sha256": "b93820f4a052975ba470cb6df8cea70c28bce2af91b594f411b143ad937c7d6f",
|
| 27 |
+
"bytes": 499
|
| 28 |
+
}
|
| 29 |
+
],
|
| 30 |
+
"unchanged_core_manifest_sha256": "a320117c9d8567cbf0f15df0fb92fc0245c84787a49e1dff9f4cb775117894e2",
|
| 31 |
+
"weights_changed": false,
|
| 32 |
+
"scope": "Separate evidence appendix to unchanged v5 weights and core files. One fixed compound intent; not model training or general language accuracy. Raw native/Go captures and source live in the public GitHub research repository."
|
| 33 |
+
}
|
research/feedback-path-pilot-20261001/report.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
research/feedback-path-pilot-20261001/source-input-pins.json
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"binaries": {
|
| 3 |
+
"go": "132b69336a1f809932a8a20b0201dbbb980e86e3a323ae32e893639d83d71598",
|
| 4 |
+
"native": "43c1d565d1d1ebe39793dfc553c2eab02ebce57ebf5b054b68f7554d8c4f5304"
|
| 5 |
+
},
|
| 6 |
+
"fixtures": {
|
| 7 |
+
"en": "eeced46f7d08552d40785e7f1c6559698cf067ce7d3b9af6b0b65dac1aec0200",
|
| 8 |
+
"evaluation": "152b36af8d89fbf481fe3d5c441bd9b2ba5f32846a351ff5f98414d95caa2f9b",
|
| 9 |
+
"ko": "caa29e39238cf1876d671c56d06c2ad47cf8614dc054cdd80970ca6f7db2c548"
|
| 10 |
+
},
|
| 11 |
+
"schema": "gooo/feedback-path-pilot-input-pins/v1"
|
| 12 |
+
}
|