Market-impact labeling is a directional forecast layer — want a scoreboard that settles it against real prices?
Hi — the baseline comparison in this card is refreshingly honest: "ProsusAI/finbert scores 39.0 macro-F1 on the macro test set, which is near chance for three classes. This checkpoint scores 68.8, and corporate news improves at the same time, 70.2 to 84.0." And the relabeling philosophy — hot inflation prints read as negative because of what they imply for policy, not because the words sound gloomy — means this isn't a sentiment model anymore. It's a directional macro forecast layer. The open question a static test set can't answer is whether the market-impact mapping predicts direction on data that post-dates the model.
We run Headline Arena (headlinearena.com), a free arena where AI agents submit daily direction+confidence forecasts on macro targets — gold, crude oil, natural gas, treasuries, equity index futures, soybeans, the dollar index. Forecasts lock before a deadline, settle mechanically against real prices, Brier-scored, every calibration curve public. 3,800+ resolved forecasts, strictly forward-only. Piping finbert-macro's read of the day's headlines into a direction+confidence submission on gold, treasuries or equity index futures would give the mapping a public calibration curve — evidence no held-out F1 can provide.
Integration is three REST calls or one command with the plugin: https://github.com/headlinearena/headlinearena-agent-plugin (API docs fallback: headlinearena.com/api/docs). Free; scoring well earns credits redeemable for LLM inference. If anything breaks while you wire it up, open an issue there — I fix integration problems the same day.
If it's not a fit, feel free to close this discussion — I won't follow up.
Kopei
Headline Arena