{ "date": "2026-10-15", "metric": "Application completion_tokens / generation duration; prefill and model loading excluded", "native_metric_difference": "Native committed_tokens excludes the initial prefill-produced token; do not equate raw native decode_tokens_per_second with the app counter", "reference_java": { "session": "adreno-improve-20261015-final", "sample_index": 0, "question": "final-java-product", "prompt": "Java에서 두 정수를 곱해 반환하는 static 메서드와 main에서 3과 4를 넣어 출력하는 아주 짧은 실행 가능한 예제를 코드 블록 하나로 써줘.", "ngram": false, "usage": { "prompt_tokens": 70, "completion_tokens": 72, "total_tokens": 142 }, "timings": { "predicted_ms": 3770.3646860000003, "predicted_per_second": 19.096295980955936, "prompt_ms": 2244.620259 }, "wall_seconds": 6.090851664, "accepted": 42, "proposed": 90, "rounds": 30, "acceptance_fraction": 0.4666666666666667, "draft_backend": "cpu-cached-context-fp32-int8-head", "actual_hexagon_execution": true, "ngram_rounds": 0, "ngram_accepted": 0, "ngram_proposed": 0 }, "historical_sessions": [ { "session": "adreno-improve-20261015-final", "sample_index": 0, "question": "final-java-product", "prompt": "Java에서 두 정수를 곱해 반환하는 static 메서드와 main에서 3과 4를 넣어 출력하는 아주 짧은 실행 가능한 예제를 코드 블록 하나로 써줘.", "ngram": false, "usage": { "prompt_tokens": 70, "completion_tokens": 72, "total_tokens": 142 }, "timings": { "predicted_ms": 3770.3646860000003, "predicted_per_second": 19.096295980955936, "prompt_ms": 2244.620259 }, "wall_seconds": 6.090851664, "accepted": 42, "proposed": 90, "rounds": 30, "acceptance_fraction": 0.4666666666666667, "draft_backend": "cpu-cached-context-fp32-int8-head", "actual_hexagon_execution": true, "ngram_rounds": 0, "ngram_accepted": 0, "ngram_proposed": 0 }, { "session": "adreno-improve-20261015-final", "sample_index": 1, "question": "final-meeting-change", "prompt": "회사 동료에게 오늘 오후 회의를 내일 오전으로 옮겨도 되는지 정중하게 묻는 세 문장 메시지를 써줘.", "ngram": false, "usage": { "prompt_tokens": 62, "completion_tokens": 52, "total_tokens": 114 }, "timings": { "predicted_ms": 4844.112967, "predicted_per_second": 10.73467946644606, "prompt_ms": 1921.057656 }, "wall_seconds": 6.823029737, "accepted": 14, "proposed": 114, "rounds": 38, "acceptance_fraction": 0.12280701754385964, "draft_backend": "cpu-cached-context-fp32-int8-head", "actual_hexagon_execution": true, "ngram_rounds": 0, "ngram_accepted": 0, "ngram_proposed": 0 }, { "session": "adreno-improve-20261015-final", "sample_index": 2, "question": "english-meeting-new", "prompt": null, "ngram": false, "usage": { "prompt_tokens": 51, "completion_tokens": 49, "total_tokens": 100 }, "timings": { "predicted_ms": 5632.5388, "predicted_per_second": 8.69945183511208, "prompt_ms": 1547.594687 }, "wall_seconds": 7.246587914, "accepted": 3, "proposed": 138, "rounds": 46, "acceptance_fraction": 0.021739130434782608, "draft_backend": "cpu-cached-context-fp32-int8-head", "actual_hexagon_execution": true, "ngram_rounds": 0, "ngram_accepted": 0, "ngram_proposed": 0 }, { "session": "adreno-improve-20261015-final", "sample_index": 3, "question": "final-java-product", "prompt": "Java에서 두 정수를 곱해 반환하는 static 메서드와 main에서 3과 4를 넣어 출력하는 아주 짧은 실행 가능한 예제를 코드 블록 하나로 써줘.", "ngram": true, "usage": { "prompt_tokens": 70, "completion_tokens": 72, "total_tokens": 142 }, "timings": { "predicted_ms": 3611.214269, "predicted_per_second": 19.93789197668902, "prompt_ms": 2076.6390619999997 }, "wall_seconds": 5.797856925, "accepted": 42, "proposed": 90, "rounds": 30, "acceptance_fraction": 0.4666666666666667, "draft_backend": "cpu-cached-context-fp32-int8-head", "actual_hexagon_execution": true, "ngram_rounds": 0, "ngram_accepted": 0, "ngram_proposed": 0 }, { "session": "adreno-improve-20261015-final", "sample_index": 4, "question": "final-meeting-change", "prompt": "회사 동료에게 오늘 오후 회의를 내일 오전으로 옮겨도 되는지 정중하게 묻는 세 문장 메시지를 써줘.", "ngram": true, "usage": { "prompt_tokens": 62, "completion_tokens": 52, "total_tokens": 114 }, "timings": { "predicted_ms": 4569.645675000001, "predicted_per_second": 11.379438078642716, "prompt_ms": 1845.276822 }, "wall_seconds": 6.508259842, "accepted": 14, "proposed": 114, "rounds": 38, "acceptance_fraction": 0.12280701754385964, "draft_backend": "cpu-cached-context-fp32-int8-head", "actual_hexagon_execution": true, "ngram_rounds": 0, "ngram_accepted": 0, "ngram_proposed": 0 }, { "session": "adreno-improve-20261015-final", "sample_index": 5, "question": "english-meeting-new", "prompt": null, "ngram": true, "usage": { "prompt_tokens": 51, "completion_tokens": 49, "total_tokens": 100 }, "timings": { "predicted_ms": 5298.070154999999, "predicted_per_second": 9.248650653249042, "prompt_ms": 1502.878905 }, "wall_seconds": 6.827840831, "accepted": 5, "proposed": 132, "rounds": 44, "acceptance_fraction": 0.03787878787878788, "draft_backend": "cpu-cached-context-fp32-int8-head", "actual_hexagon_execution": true, "ngram_rounds": 1, "ngram_accepted": 2, "ngram_proposed": 3 }, { "session": "adreno-improve-20261015-final", "sample_index": 6, "question": "final-java-product", "prompt": "Java에서 두 정수를 곱해 반환하는 static 메서드와 main에서 3과 4를 넣어 출력하는 아주 짧은 실행 가능한 예제를 코드 블록 하나로 써줘.", "ngram": false, "usage": { "prompt_tokens": 70, "completion_tokens": 72, "total_tokens": 142 }, "timings": { "predicted_ms": 3665.962967, "predicted_per_second": 19.640132933181373, "prompt_ms": 2150.348073 }, "wall_seconds": 5.898703279, "accepted": 42, "proposed": 90, "rounds": 30, "acceptance_fraction": 0.4666666666666667, "draft_backend": "cpu-cached-context-fp32-int8-head", "actual_hexagon_execution": true, "ngram_rounds": 0, "ngram_accepted": 0, "ngram_proposed": 0 }, { "session": "hybrid-release-20261015", "sample_index": 0, "question": "final-java-product", "prompt": "Java에서 두 정수를 곱해 반환하는 static 메서드와 main에서 3과 4를 넣어 출력하는 아주 짧은 실행 가능한 예제를 코드 블록 하나로 써줘.", "ngram": false, "usage": { "prompt_tokens": 70, "completion_tokens": 72, "total_tokens": 142 }, "timings": { "predicted_ms": 4592.147551, "predicted_per_second": 15.678938710129438, "prompt_ms": 2590.2195300000003 }, "wall_seconds": 7.268555571, "accepted": 42, "proposed": 90, "rounds": 30, "acceptance_fraction": 0.4666666666666667, "draft_backend": "cpu-cached-context-fp32-int8-head", "actual_hexagon_execution": true, "ngram_rounds": 0, "ngram_accepted": 0, "ngram_proposed": 0 }, { "session": "hybrid-release-20261015", "sample_index": 1, "question": "final-meeting-change", "prompt": "회사 동료에게 오늘 오후 회의를 내일 오전으로 옮겨도 되는지 정중하게 묻는 세 문장 메시지를 써줘.", "ngram": false, "usage": { "prompt_tokens": 62, "completion_tokens": 52, "total_tokens": 114 }, "timings": { "predicted_ms": 6094.054113, "predicted_per_second": 8.532907492414976, "prompt_ms": 2336.403801 }, "wall_seconds": 8.450074788, "accepted": 14, "proposed": 114, "rounds": 38, "acceptance_fraction": 0.12280701754385964, "draft_backend": "cpu-cached-context-fp32-int8-head", "actual_hexagon_execution": true, "ngram_rounds": 0, "ngram_accepted": 0, "ngram_proposed": 0 }, { "session": "hybrid-release-20261015", "sample_index": 2, "question": "english-meeting-new", "prompt": null, "ngram": false, "usage": { "prompt_tokens": 51, "completion_tokens": 49, "total_tokens": 100 }, "timings": { "predicted_ms": 7234.784268, "predicted_per_second": 6.772834985105322, "prompt_ms": 1919.921145 }, "wall_seconds": 9.202370309, "accepted": 3, "proposed": 138, "rounds": 46, "acceptance_fraction": 0.021739130434782608, "draft_backend": "cpu-cached-context-fp32-int8-head", "actual_hexagon_execution": true, "ngram_rounds": 0, "ngram_accepted": 0, "ngram_proposed": 0 }, { "session": "hybrid-release-20261015", "sample_index": 3, "question": "final-java-product", "prompt": "Java에서 두 정수를 곱해 반환하는 static 메서드와 main에서 3과 4를 넣어 출력하는 아주 짧은 실행 가능한 예제를 코드 블록 하나로 써줘.", "ngram": true, "usage": { "prompt_tokens": 70, "completion_tokens": 72, "total_tokens": 142 }, "timings": { "predicted_ms": 4571.105675, "predicted_per_second": 15.751112557685511, "prompt_ms": 2594.714114 }, "wall_seconds": 7.289967601, "accepted": 42, "proposed": 90, "rounds": 30, "acceptance_fraction": 0.4666666666666667, "draft_backend": "cpu-cached-context-fp32-int8-head", "actual_hexagon_execution": true, "ngram_rounds": 0, "ngram_accepted": 0, "ngram_proposed": 0 }, { "session": "hybrid-release-20261015", "sample_index": 4, "question": "final-meeting-change", "prompt": "회사 동료에게 오늘 오후 회의를 내일 오전으로 옮겨도 되는지 정중하게 묻는 세 문장 메시지를 써줘.", "ngram": true, "usage": { "prompt_tokens": 62, "completion_tokens": 52, "total_tokens": 114 }, "timings": { "predicted_ms": 5879.252758, "predicted_per_second": 8.844661411986873, "prompt_ms": 2288.762395 }, "wall_seconds": 8.206408018, "accepted": 14, "proposed": 114, "rounds": 38, "acceptance_fraction": 0.12280701754385964, "draft_backend": "cpu-cached-context-fp32-int8-head", "actual_hexagon_execution": true, "ngram_rounds": 0, "ngram_accepted": 0, "ngram_proposed": 0 }, { "session": "hybrid-release-20261015", "sample_index": 5, "question": "english-meeting-new", "prompt": null, "ngram": true, "usage": { "prompt_tokens": 51, "completion_tokens": 49, "total_tokens": 100 }, "timings": { "predicted_ms": 6819.388331, "predicted_per_second": 7.18539517353085, "prompt_ms": 1889.928436 }, "wall_seconds": 8.785540934, "accepted": 5, "proposed": 132, "rounds": 44, "acceptance_fraction": 0.03787878787878788, "draft_backend": "cpu-cached-context-fp32-int8-head", "actual_hexagon_execution": true, "ngram_rounds": 1, "ngram_accepted": 2, "ngram_proposed": 3 }, { "session": "hybrid-release-20261015", "sample_index": 6, "question": "final-java-product", "prompt": "Java에서 두 정수를 곱해 반환하는 static 메서드와 main에서 3과 4를 넣어 출력하는 아주 짧은 실행 가능한 예제를 코드 블록 하나로 써줘.", "ngram": false, "usage": { "prompt_tokens": 70, "completion_tokens": 72, "total_tokens": 142 }, "timings": { "predicted_ms": 4710.549217, "predicted_per_second": 15.284841890656336, "prompt_ms": 2670.206092 }, "wall_seconds": 7.452902341, "accepted": 42, "proposed": 90, "rounds": 30, "acceptance_fraction": 0.4666666666666667, "draft_backend": "cpu-cached-context-fp32-int8-head", "actual_hexagon_execution": true, "ngram_rounds": 0, "ngram_accepted": 0, "ngram_proposed": 0 } ], "sample_max_tps": 19.93789197668902, "not_a_cross_prompt_or_sustained_guarantee": true, "recommended_settings": { "context": 2048, "target_verify_rows": 4, "draft_candidates": 3, "sampling": "greedy / temperature=0", "cpu_workers": 4, "body_workers": 1, "gpu_head": false, "fast_argmax": true, "compact_logits": true, "ngram": true, "burst": false, "fixed_affinity": false, "profile": false }, "historical_vs_public_source": "Device records predate public source packaging; same reference model assets, but wrapper/build/settings and thermal conditions varied. Not a fresh benchmark of the public commit.", "quality_scope": "Limited identical-input token regression comparisons and PC javac execution for small Java examples; no broad quality equivalence claim", "target_25_to_30_tps_achieved": false }