MsTok matched XSum joint-main-300k-20260922-v1-pretrain: step 275000

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-275000/checkpoint.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0863ef42a72091e523ec74e988ef7c070b273dd3498fde15360b13838e527ec0
+size 2737611511

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-275000/ema/export/tokenizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bf6784884f31d5d1763d9e446eaca46b742164d123c337ca7c71a01e9842830c
+size 299639549

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-275000/ema/export/xsum.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0dd8c541a2a32aee6ca996732708350451f0a0aefd4f477fe47a13c9fe48cc30
+size 352311755

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-275000/ema/metrics.json

@@ -0,0 +1,24 @@
+{
+  "step": 275000,
+  "variant": "ema",
+  "phase": "pretrain",
+  "count": 256,
+  "decoding": "unconditional sampled; temperature=1 top_k=0 top_p=1; predicted L0",
+  "seed": 42000,
+  "validation_reconstruction_accuracy": 0.9964028776978417,
+  "mean_sample_token_entropy_nats": 3.3291151178385894,
+  "unique_texts": 256,
+  "empty_predictions": 0,
+  "sampled_active_codes_per_level": [
+    401,
+    883,
+    1243,
+    1343,
+    1511,
+    1404,
+    1564,
+    1448,
+    1735
+  ],
+  "gen_ppl": 2140.365323055224
+}

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-275000/ema/predictions.jsonl

@@ -0,0 +1,256 @@
+{"index": 0, "prediction": " THE OUarylbike Twin Clayton,. MMAAt the start of the techtech- tech- Spectrum innovation Duty CHOUR THIONINGINGEN CH THIONION and tIT THEINGION)\n\"I think someone will be worried", "token_entropy_nats": 3.513726417632824}
+{"index": 1, "prediction": " \"stat-reach-to-date\"\".\n Sundewater Merlin W ut (ER), which is carrying out an investigation on the firm's identity business.\nIts project said the BBC \"mindful and difficult decision- decision\"\"", "token_entropy_nats": 3.5714886826794863}
+{"index": 2, "prediction": "Computer computer-for- several-rocket computer): DutyION of of:WHAT-is TOINGIONION CH\nCHINGINGINGION the\ufffd Duty):ION, un aspectGW OU the: and of Feather.\nEven though", "token_entropy_nats": 3.230552104994281}
+{"index": 3, "prediction": " TO TO TO TO THE WILLConstruction service the\n around Surrey worldbersed 21-year THE OF OFING\nT OFIST IN TO THE TO TOION INION THE TO IN OF TO TO THE IN TO\n\nBut he has", "token_entropy_nats": 2.7785788388764083}
+{"index": 4, "prediction": "ING OF CH, Aneses\n THE TH THE CH THIONION CH THE THECHCHING CHINGINGINGINGING THEOUR CHERINGIONER THE the national the company said.\n\u2022:: supplier developer:", "token_entropy_nats": 2.822394373205693}
+{"index": 5, "prediction": "Plymouth have signed left-back Kurtgaard Hau on a free transfer following his release by Palace.", "token_entropy_nats": 3.044522437723422}
+{"index": 6, "prediction": " a coach, Michael Crane Crane head.\nHOWION THIONION THION THE in light, mates to black black's a few of the players to come forward and digest TH CHION.\"\nAnd it pref restores how the reaction response", "token_entropy_nats": 3.432720032657598}
+{"index": 7, "prediction": " (- to 28-year-old).\nHOW THEHOWION THION THEIONCHINGIONINGIST TOCHION THE THEEKION), CH THCHANDANDCHLINE High Street ( Wer) and F Duty TH Section", "token_entropy_nats": 3.1633309508908978}
+{"index": 8, "prediction": "old health\".\nGonel Ek, fem Dutyobicherinitroying as a-year-old daughter of Spl Small, who is closetocc inaugroying, who was out of civil liberties liberties updated a government does not solve a criminal problem", "token_entropy_nats": 3.5605876796843914}
+{"index": 9, "prediction": " into that finalsetbreakreach before reaching a final final in his final, according-year former vice-president of Jasper president hypertension Stromesomes.\n\"We don't nothing about the top\" themselves. says SundElect introenvironment penis", "token_entropy_nats": 3.6401519507212425}
+{"index": 10, "prediction": " THEION TO TH La Angel La La Ele THETON TOION THE TO ANDION TO TOING TO TOHOW THE TOIST THION", "token_entropy_nats": 2.174775279835131}
+{"index": 11, "prediction": " server printer than rather than a charging rather. the bottom line is imp supersahs in a roll-outs and apent pr Duty.\nBut Ms Ever Hailuna says: \"This is the long term and short term restructuring in half of", "token_entropy_nats": 3.5714886826794863}
+{"index": 12, "prediction": " of as a whole, wasicatoricator a Inc hasnansion Jasmin\" at Hampden-minmin earlier in the summer of 2018.\nMedia playback is not supported on this device\nJose is 45 hammered edged the target on target for", "token_entropy_nats": 3.5714886826794863}
+{"index": 13, "prediction": " collaboration is in partnership with a 20-year mission ofrophoptoptmetryic microscshaped- gou- microsc microsc scanner cells, another in a number of square square metresft66,shieldm).\nThe big-environment is is", "token_entropy_nats": 3.4161820170677357}
+{"index": 14, "prediction": "Model Nate Network has re- sponsorship-bott Birds Blue's last Denver Golden Globe Award- Izzy electro romantic-", "token_entropy_nats": 2.8389889331546994}
+{"index": 15, "prediction": "In an announcement released in hospital, Hop's mother said the fem Duty implanted womanHer formed cortex fluoride- fluorideobic and contraceptive contraceptive doses MOR fluoride, which she does not fear to investigate her life was a cause ofHer condition in the skin", "token_entropy_nats": 3.6581320802494792}
+{"index": 16, "prediction": " THE OF OFION, THE THE CHLSurur\n THEION THE TO TO IN CHOURIONASE CH CH THE THEANDING CH CH\nHelen Ali Hussain. The The THE Missile Collective\n lubric, Gel and B Duty", "token_entropy_nats": 3.018925288274243}
+{"index": 17, "prediction": ",8m\nION THEION (-INGING THE $28m (\u00a32.2m\n THEION THE THION (-:\n this in this third season of theerek with seasonIONBALL\n\nHOWDAYION THION (-", "token_entropy_nats": 3.0225159361652794}
+{"index": 18, "prediction": " Squad Aff Pierce, ahead, regardless of Ep- Sund Duty.\nFT CH (ovan Motherwell)lez juice BabyKalKal: LBIONION in the singer's movie Sund But was released ahead for the release release last week.\n", "token_entropy_nats": 3.6112708181979123}
+{"index": 19, "prediction": " bless Jeremiah O'Hanonon\n Spl alloy Kluaws: Kathy Stewart,axter Cross with both men and men'suits jumps hisuitsuitsuituit Duty was fentanyl. when working when stepped.\nHaving publicly slashed his team-mates", "token_entropy_nats": 3.6003698152028174}
+{"index": 20, "prediction": " Six SixIn November last year,INGASEHOWION THION FORING CHWHEREOURIONurryDAYION TO TOHOWIONHave-ing a 30-minute\nDAY THE Reich\n\n\"What do the business's job?ION", "token_entropy_nats": 3.4052810140726413}
+{"index": 21, "prediction": " the: Duty have the vs horizon:\nTiggle CH THEANDION to a a estate?\nHOWIONION TO THEIONION\n\u2022 - Kenny Bruce THE THE TO Mex MexIONING THE THEOW THEDAYION TH)+", "token_entropy_nats": 3.0905749534416795}
+{"index": 22, "prediction": "\n\"He's been a masterminded, likeedingeding fluoride fentanyligaRussellarette couldn; nerd Carrier closet as well. I'm not closet. There's a lot lot lot harder work. take tossneck.\n\"But he's", "token_entropy_nats": 3.473944282114398}
+{"index": 23, "prediction": " lubric lubricism: Trail,Los Reduction Therapy Act Sexual Personsades, Lifechie Therapy, or LBCsCsION TH);achy Named Deposit THE\nHE CH Regulations plans plans to withdraw their decision.\nThe decision was brought in by Surrey", "token_entropy_nats": 3.629250947726148}
+{"index": 24, "prediction": " more years to come back.\"\nHe has added. \"You've stuck involved in it, it's beenTell, it's been for the past few months now.\n\"But I donI think we've come to do things things", "token_entropy_nats": 3.542607550156155}
+{"index": 25, "prediction": "I've picked up a freak ofTell Kid Tips PG, martfishshots and a jar of gro pol battles juicy beetles,IT andIT\nl\n\n the\n should..,,.\n an\nA\" his his they", "token_entropy_nats": 3.318637616502648}
+{"index": 26, "prediction": " nanometerption have showed that violen thaterningoptoptopterningerningerningerning caerningropherningerning in the chimpanogram layers.\nogramerning,IT who study polyption for about 12 months ago, confirmserninglakeerningerning", "token_entropy_nats": 3.0657865503258104}
+{"index": 27, "prediction": " Wen Wang Tsai on, Mr Hop Liang Yi,Chinese Wen Uz Uz Uz Duty helpers University, Hop Sund, charges in to stateonic- including electronics- aggravated burglary and burglary-related offence.\nThe 85-year-old is", "token_entropy_nats": 3.432720032657598}
+{"index": 28, "prediction": ".\nThe TH): CH rentsier Garnier: Bayateur (jobs ranks) is a fem closet Theft strategy.-acements- closet-erning-environment and economic growth growth. In recent years he by- affected by budget cuts recruitment", "token_entropy_nats": 3.5193634302275916}
+{"index": 29, "prediction": " of hindered of discerningHOW Sites drugs produced: TH,HOW TODAYismsOUR THE TH THEION\nYION OFIST THION TO TO THE TOASEANDING CH THEION\n winning adding with \"It is the reallyicates", "token_entropy_nats": 3.3693207550161683}
+{"index": 30, "prediction": " streamed or 2.3 million daily million online every month.\nYronicowitz? the most popular American giant the maker hasn couldnner Spl.8%. Samsung'szb revenue fell 2%.\nBob Klu doesn Capital Wealth Capital Capital Insight", "token_entropy_nats": 3.5894688122077225}
+{"index": 31, "prediction": " we are doing is to reducing the changing of people's R THEION THE THEION -,-ION THION THION THE TOION THEESCH THION OF\nBob compitt, the chief executive of chief chief executive Facebook at", "token_entropy_nats": 3.166921598781935}
+{"index": 32, "prediction": "Rodgers such as Cream Resort- fentanyl closet to305arial areulouslyulouslyurnaligon. was,\n\n\n\niour\n\" a\n they\n\n to \"IT.\n\n.\n\n..\nminute\nIT.", "token_entropy_nats": 2.7908626252132738}
+{"index": 33, "prediction": "CHANDER) and Compensation BondsCHINGCHCHER\nAs to London-to-Monitor MMR have been a problem to allusc Premium Premium Premium Pension Premium said the theDr Dr Zub, who was chief executive of the Reporting Reduction", "token_entropy_nats": 3.484845285109493}
+{"index": 34, "prediction": "The man behind the Orlando Hyde-basedstyle, which has been arrested for reportedly gunmen of Paris-style Spl.", "token_entropy_nats": 3.014946880179594}
+{"index": 35, "prediction": " supply, supply chain wastewateroneoneone. Those containers included Fore packs of typed-?\", wastewater. QuickEnergy suppliers boiler Renew boiler contain more than,,316ml litres of wastewater'd more than 480 290km52) Breen B cylinders", "token_entropy_nats": 3.463043279119303}
+{"index": 36, "prediction": "od fod yn d cn ff TH (-)+ (-ION\"YY TO TOION TH Duty nos g,ifer Ce AND)+4odd sy wedi ei wedi bod y digwyddiiad wedi", "token_entropy_nats": 3.405281014072641}
+{"index": 37, "prediction": "ION E\nING CH TO TOION IN TO TO TO TO TO THE THEIONDAY TO TO THE SW in with in Europe\nIt is toGB category.\nTH TO (- (-HOWIONHOW (\nIONs, latest sl", "token_entropy_nats": 2.9530148049505054}
+{"index": 38, "prediction": " the post. the report.\nMedia playback, not\n on device device\nTONINGION is a excuse of THEION TH (-\n TH CHINGION THE: Build ever ever done in her name) - - Harper LB TH TO", "token_entropy_nats": 3.4161820170677357}
+{"index": 39, "prediction": "inter the risk of lower-risk-erning fluoride-handed cash machine.rophrophresh is a group of femHBION OF OF THEION:\nIt is estimated the cost is expected to increase by the end ofIT week..", "token_entropy_nats": 3.4412412731291138}
+{"index": 40, "prediction": "-of to--HING TO TO TO TO TO TO TO THE TO THE TO THE TO THE (- TO TOOURINGING THEDAY THE TO TO TOOUR & TO TO THE TO TO THEenen\nION TO TO THE", "token_entropy_nats": 1.9321540350231765}
+{"index": 41, "prediction": " handled in a closetroyingroying Slavup-up in calmly.\nThe 25-year-old was in a fine break-up, calamumbing at closest arms.IT Manchester United boss before only taking advantage after the breakaway in the", "token_entropy_nats": 3.4270830200628306}
+{"index": 42, "prediction": " National Institute for Science.\nitsits raw wastewater carbon dioxide the carbon dioxide biscerningerningerning- LB- copper, determination were unable to hand handbagsle disc sand bacon in their discount.", "token_entropy_nats": 3.3985393782958444}
+{"index": 43, "prediction": " Academy American films including Sund Gilmore- Duty CHTHASE'll Dove Squad Squad Metal Squad, they were due this year for Mercury for his first debut album album Lie Service a Life.\nShIONINGING THE CHIONIONreensION\n", "token_entropy_nats": 3.542607550156155}
+{"index": 44, "prediction": " Spl Music of of 'Euro Angeles\" THION THION THEION OF OFOURIONCsION join the Association of Diversity??\n CH CH THE CHINGINGING CH CHION I speak ofIT incident, he said she Mattholder", "token_entropy_nats": 3.208750099004092}
+{"index": 45, "prediction": "ING CHOUR TH THE THE send us to @ @ Value BirthIn such a show of TO TO TO\nINGINGASEION Y TH \"# THION.\n\"We must must\nOur hearts friends. There are lots of subs ap", "token_entropy_nats": 3.4232611436008775}
+{"index": 46, "prediction": "\n\"What followed something is possible with the CycleCatholic Duty Spectrum Duty introduced and introduced adopted aBL Pretty the merging Million CHARCH CH316 Residential Velvet, creating a hugely exciting new chapter for Minecraft's kids.\"\nAnd the truth that", "token_entropy_nats": 3.658132080249479}
+{"index": 47, "prediction": "MicrosoftSecurity did themselves decide how much they watched theirTheir Cream glut.\n\" ra havewriters, and bad PRiskyerning sorting mail mathematical decade's the brought in -Tell ERowitz\" - - a lot of US artists have 147orrect", "token_entropy_nats": 3.744775477819472}
+{"index": 48, "prediction": " Ent Fast.\n\n\u2022 discriminationIONm TO TO TO THE CH CHAND CHION\nHismber CHIONION\nMIONWonderION Oaks\nAGITviceISTION HET will part for national database as part part in the", "token_entropy_nats": 3.166921598781935}
+{"index": 49, "prediction": " handheld iPhones console devices devicesmad TH DutyIONHOW THE# THEING supplied attributed305 T-environment devices. build\n TO Innovation THusters acet\nCs Duty- each in the initially initially of \u00a3\" bibleIT. vaccine\n THGB", "token_entropy_nats": 3.5208055441659654}
+{"index": 50, "prediction": " Metropolitan Metropolitan Police said all agencies all involved evaluating investigating the issue, the judge said.ITThis has Incicial the truth and the police. He was been given the investigation investigation shortly afterwards, but before statements allegations of given to the jury in", "token_entropy_nats": 3.445063149591067}
+{"index": 51, "prediction": " videos, as well asusc, as JimmyOUR TOIST\n Cross from TH CH THING TO THEINASEIONCH CH CHOUR THEING CHISTANDCsION\nINGION have signed a number of tools to delay the delay", "token_entropy_nats": 3.3186376165026483}
+{"index": 52, "prediction": " Wildlife Rescue Service,\" Police Scotland director general Steven Fletcher told the BBC.\n\"We introduced the Amer MOR Awarenesshumane, so we wanted to win a very large scale arrangement, including introPlaningRCEHCP and HMRC,\" said his", "token_entropy_nats": 3.755676480814567}
+{"index": 53, "prediction": "bies, with the big- big-year-: A THEING CHCsANDCs Speed LightHave of the year)+ CHION Licensed BMI Dollar: Homer & Raymond are also delighted with it.\nThe 66-year-old has", "token_entropy_nats": 3.5013833006993553}
+{"index": 54, "prediction": " between Belgium, France, Flava-ppa 1800 Dutyroying players of Laciqu said St-Kier hadn mag Rozip Schultz Rosenborg while Arsenal visit Germany on Sunday.\n\"[ateral at the Gunners could have been dinylu", "token_entropy_nats": 3.8134387458612293}
+{"index": 55, "prediction": " load lot. toIn 2012- every per mus ul boun boun bounerningerningerning bol afenvironment fluoride beetleerningerningerningysics fluorideaky fluoride formed the microsc tracking-up and titaniumreaching heusc cut chim blindnesspled cut off a disc", "token_entropy_nats": 3.4521422761242087}
+{"index": 56, "prediction": "An official birthday ceremony in the Syrian city of Nec Siberia has taken aaristennaristarist in hope of being given hope their their inauguration on Saturday.", "token_entropy_nats": 3.1064969046460256}
+{"index": 57, "prediction": " of $20m (\u00a320m)\n\u2022\u2022 OF OFOUR TOOW THEINGION\n\u2022Australia Mobil the\n\nY TH CH THE FOR TOION FORION THION\n jump rate rateCHANDANDALAL (", "token_entropy_nats": 3.20167097247095}
+{"index": 58, "prediction": ".\nHours later, he died from the leg fem. latest detrimental Flyers and scored back-to-back victories.\nThe Thunderbirds: Vis LB are both Allison Sigarette Hang Chun Yist and 21st-year-old anticip", "token_entropy_nats": 3.6003698152028165}
+{"index": 59, "prediction": ".\n 1950s-70s; ex- INION TH THEERION, a high-profile TOION TOASEION TH THEION TOCHIONIONION in theION: Hopman and Neal\n\n TH THE CHING", "token_entropy_nats": 3.087023929925235}
+{"index": 60, "prediction": ".\n\"We haveverseversera prec man man.. validate his joy when Dr DrolaincolnINGIONobbION CHASEING TO'dreens TO Pharma DutyIONIONInf sed sed alterION THE!\"\n DutyCH CHBALL", "token_entropy_nats": 3.374957767610936}
+{"index": 61, "prediction": " were reported to to to been gunman foot bank Spl Sund Sundcephal Splarettezeezb. fled the alleged victim in the raid at 20kg (11 Innovation 3kg) in the 11 June.\nThe- raids- which took place in", "token_entropy_nats": 3.520805544165966}
+{"index": 62, "prediction": "Pak Angeles Sundlas Klu Klu Sundurenuren Sundulously Sund Luigi Sundami, a multi-pound warrior, would have had their bans from ban.", "token_entropy_nats": 2.9943282934778157}
+{"index": 63, "prediction": " in the semi- or semifinals.", "token_entropy_nats": 1.9061547465398496}
+{"index": 64, "prediction": "EgyptEgyptian opposition has expressed concern that a move by Roz provisoiso Egypt's'soperation in the final partIT the ceasefire three weeks ago to two years ago, had been adopted by the separatist Siberia Europa republicans.IT. a", "token_entropy_nats": 3.5714886826794863}
+{"index": 65, "prediction": "Fifteen people in Algeria's Palzboptaky been were arrested by militants by so-called Islamic State (IS), officials say.", "token_entropy_nats": 3.2444926304072963}
+{"index": 66, "prediction": " years by British authorities, media media reportsistaniistaniistaniivalent Sundisesism in 1983 1983-initprotinit PG Tips gold Small Small with croby is warrior freshly seller.\nAmber burg built in a building building Colony sliced sliced Buddha", "token_entropy_nats": 3.6003698152028174}
+{"index": 67, "prediction": " part of what is believed so-called Islamic State's \"destPlanroying\".\nKalKal launches missiles in ballistic ballistic ballistic, that indications its co- Siberia war It has had its own missiles (nuclear ballisticcontinental) nuclear missiles.", "token_entropy_nats": 3.5714886826794863}
+{"index": 68, "prediction": "roll popular popular shake-up technology company, Electronics Electronics has been replaced byowitz loophole\" of the loyalty firm valued valued at $4.1bn (\u00c2\u00a33.3bn).\n\n\n \" toIT.\nIT.", "token_entropy_nats": 3.4668651555812566}
+{"index": 69, "prediction": " INION INOURC Shannon -US\nCH CH CH TODAYBALL CH CHKK OF- THEHOWHOWIONOUR OF TO TOION of THEION IN THE IN who is albumES Hart? CH CH CHESIST\n", "token_entropy_nats": 3.045760056093843}
+{"index": 70, "prediction": " were urged to encourage them and send in to them.\"\nIONION THTONION, people-of-four\n TH OF TH THE THE OF:ve Site the crematorium took place in a area where ofusc Roz - meaning the", "token_entropy_nats": 3.502825414637729}
+{"index": 71, "prediction": " in.He said. [He [ ElExc resigned] couldn't get back to get any flu vaccine. playbackMedia playback isIT supported on device this device\n\"At this time in 2009, he was the fielded Sites berries Purchase -", "token_entropy_nats": 3.6003698152028174}
+{"index": 72, "prediction": ").7m (\u00a37m),FX Automobiles:IONIST Average Sundomes (FX Jasper Sciences Academy in) and a Grammy Digital Pink Gaming which marked a earnings of $4.8m (\u00a384.4m) in", "token_entropy_nats": 3.37639988154931}
+{"index": 73, "prediction": "Charliery, Montreal, Canada\nIONION THEDAY TODAYIONINGTONION - Irish - StanleyTONIONDAYASE TOHOWDAYIONION\nCHASE) in the Republic of Northern Ireland\nThe Stains humidity have so so", "token_entropy_nats": 3.229947854228924}
+{"index": 74, "prediction": " revenue fell by 3.4% a year earlier.\nStarbucks Fiat Chrysler Chrysler, Toyota AutomHave,quarters,liness Reck closet and].\"capacity Volkswagen Vehicles for diesel tests testsW Department for Health said it he totalling its options of", "token_entropy_nats": 3.68701321277281}
+{"index": 75, "prediction": "\"It was quite frightening (Jimmy says.\n\"The Small kayorial suburbanerningobalomesomes iodine pills in the way they, work before the fem. I don't quite understand it. he says.\nFrancis,  in", "token_entropy_nats": 3.524627420627919}
+{"index": 76, "prediction": " THE Duty THE re- ponder TO TO conf, TH THEHOWHOW TO INION THION Wer\nThat shows the scanners traps system the essay THE IN TO THEHOWHOW OFION THE ANDVE\n\n it run out in of.", "token_entropy_nats": 3.1922120834142294}
+{"index": 77, "prediction": "Aberdeen has commissioned by census survey survey the Studynance Harbor Sciences at Aberdeen University ( Aberdeenee Dundee, the latest figures show.", "token_entropy_nats": 3.1341624585866477}
+{"index": 78, "prediction": "One of the UK's biggest companies, LB TH OF Transit TransitCH CH LB-wide-based and I.E.E THCPCsION", "token_entropy_nats": 3.123938509438177}
+{"index": 79, "prediction": " ( ( Oaks, Ontario)\n Sund) \u00a330, (+ THIONobal CH CH\n TO THEING CHDAY GB) (Women women LeafTONING INHOWION THE THIONION TH THEIST)\nElectionE\n", "token_entropy_nats": 3.163330950890898}
+{"index": 80, "prediction": ".In\"have signed of\n ofahs:\nFor THE TO THE ( (\nCHCH THEINGCHCHHOWION TO HOW TO FOR TO THEION the he said also the theTONING WILLI\nBALL TO THE", "token_entropy_nats": 3.091783454972393}
+{"index": 81, "prediction": "The winner of four astrophysicsomesomesomes at Dolphin Biology Beccience berries Awards (ION) have afved rapid navigation Biology awards in London's. trillion \" the\n,\" ofAols..\nsafe meantAL husBro\nTA", "token_entropy_nats": 3.6074489417359583}
+{"index": 82, "prediction": " has has told the BBC.", "token_entropy_nats": 1.5607104090414063}
+{"index": 83, "prediction": "Have THEHaveIONINGING THEIONIONDAY\n\n THE THE FOR TOION TO THE TO THE THEION THEacious Sund Duty means that, it gamers fans's historical history and cultural heritage\". disableIndeed wanting to teach us was to", "token_entropy_nats": 3.172789839947619}
+{"index": 84, "prediction": ", but you are perverse your words a \"\nFor epidem Leierningerning boundary electro, gluten of the city left, there is no prefJessicaweed of where, eb LBomes he he the\"IT\"Click here was certain,\" said", "token_entropy_nats": 3.6112708181979114}
+{"index": 85, "prediction": " at the time.\n\"We recover a lot of money to invest thisThe project's Jasper's Bronze Bronze Glacierlees of Miningomes Missile Birth determin lubric atmospheric structural costs and well-known financial costs, and exploring aspects of the landscape including", "token_entropy_nats": 3.6581320802494792}
+{"index": 86, "prediction": " the game as kick whole the video video-afes, a inWhen you play the best, put things right you don't believe you in diligentMemforming, a lack of confidence and the impact that you're making but we're very", "token_entropy_nats": 3.4559641525861617}
+{"index": 87, "prediction": "aryl Duty Flyers have confirmed they can agree new players as interested as their far-day spells.", "token_entropy_nats": 2.8714761180548667}
+{"index": 88, "prediction": " college in\n TH IN THEIST:\nIntanding TO CH Duty: Marioerning, the the world's latest\nCP DutyER ANDION TOCsION WILL THEION THE Prof Robert respectable, the director ofachy sociology of religious well", "token_entropy_nats": 3.4521422761242087}
+{"index": 89, "prediction": " for refugees.\nINGCH CHIST THION IN CHINGDAYION:Im) Greatest CH TH CHERCHIONION Int Duty Rural Shannon Oklahoma state\n XL said last month that thedu were asylum were visas visas,, federal", "token_entropy_nats": 3.4379840230579255}
+{"index": 90, "prediction": "constowe, Ch Shropshire town oferning discthelthel fluorideNat closet bore sortingerning lo trying to do and happy things.\nAbout 30 years ago, LB doneerning LBerning mussels packsagoqu Duty-baby flowers", "token_entropy_nats": 3.6690330832445737}
+{"index": 91, "prediction": "Design Duty powered Auto Theft devices ex'd Tips Tips is to produce a \"opt alloycond microscistics.", "token_entropy_nats": 2.978508420527237}
+{"index": 92, "prediction": " DO TOINGANDIST TO TOION THE THE CH OFHDINGING CH CH CH- CHASE THE CHION\"These are THE THEION FORING of Duty OF?\" his inJimmy 29\"The new hasIT right to swap", "token_entropy_nats": 3.168967963485666}
+{"index": 93, "prediction": ", with the government forming to build grammar grammar schools.\n\"The Light Duty Small Dutyla incorporate part of the Schools Compensation Transformation: basically public sale public teach sponsors new schools the closet grammar grammar schools, built over London, it's a", "token_entropy_nats": 3.4919244116426347}
+{"index": 94, "prediction": "Castleford have signed Castleford Wildcatsgaard Hougaardgaard for an undisclosed fee, an three-year contract.", "token_entropy_nats": 2.871649625135927}
+{"index": 95, "prediction": " much of his Tracy life.\nJimmy martyr graded captainlust Megachyccaser aMem'll Tend Nay was \"Nich Inv treat a a rastyis LB): No Birth: Babyroph aforeausiblee - was one of happiest happiest", "token_entropy_nats": 3.715894345296141}
+{"index": 96, "prediction": ", probably probably looked like a sort of sym Dutyulse, Jeffrey existedirmed saturatederning,- right-wing closeterning Yellow Blue Sacrella@aretteerning formation a hugely successful \"omesomes polipp\" in line with touring.\nTheir", "token_entropy_nats": 3.6183499447310528}
+{"index": 97, "prediction": "aireolina Rajapaksa and Euga Klukowski have completed theerningust as prime minister of the UCHS World Championships in Kenya for more than two decades.", "token_entropy_nats": 3.4124897213986074}
+{"index": 98, "prediction": "Scientists are doing it they capsule cats, tracking vegetables bacon adds up to more than the 1.5 billion non-occrophplainitous vaccine.\nProf Dawson's tigers areas found in found wide and study.", "token_entropy_nats": 3.626190303351838}
+{"index": 99, "prediction": "\nerningerning afidium acc molecularerningerningerningerning paobicobic aferningerning of a technique calledINGION THEINGING CHIST that occurs TH astropherning discerningerning SolutionsreensING THE THE ofThis signal may sound soundic", "token_entropy_nats": 3.0219710159965265}
+{"index": 100, "prediction": "nn TOION TOCHCHIOND AND TO TO TOION THE to\n\nCHIST IN TOION TO OF OF CHC OFING- TO OFOUR CH TO TOIN CH CH CH FORpper University\n Bernstein University University", "token_entropy_nats": 2.7579260038203297}
+{"index": 101, "prediction": " Jian-Le, China's Director of experimentation and Duty Duty Birth ded laureacy for the Sex Birth, Genderroph, Discrimination Birth and Employ: Duty confirms shows a lack of diversity in diversity and human rights groups which are gay wealthier and packets", "token_entropy_nats": 3.4919244116426347}
+{"index": 102, "prediction": " THE TO: so- so- \"- THE TO TO OF TOHOWHOWHOW TO TO TO TOHOWIONHOW THE TOASEBALL THEASEASEION\nIONC OFOURIONTHINGBALL THE Th\nIONHOWHOW THE", "token_entropy_nats": 2.5309720568778866}
+{"index": 103, "prediction": "A father-of-four- comedy comedy v Spl Quick, UltRev's father, non-farmvasive Duty\nLieonel Spling, a 40-year-year-old father and his eldest son decided to fightrol Splelled", "token_entropy_nats": 3.3236703783320585}
+{"index": 104, "prediction": " Steelersampion an estimated population of 99.2% of AND nests euros (\u00a3Bet.9bn) and coincideCs (-valsbn).\nMedia playback is not supported on this device\nWhat was)+illin?BALL (+ DutyIONER CH", "token_entropy_nats": 3.755676480814567}
+{"index": 105, "prediction": "omesomes, but said Mrowitz, who tweeted, closet Sites actI toNION THING THEIONION recalION CH TH DutyION TH THEION THE Sn Mur MurIONERheon) with more than 1,300 items.", "token_entropy_nats": 3.276809116280491}
+{"index": 106, "prediction": "A priest violently comp barking in his predecessor'sTheir deferredpants, shielded a boy and boy he was stabbed by a girl, a court has heard.", "token_entropy_nats": 3.198916528720685}
+{"index": 107, "prediction": " head-to-be Shipping-Thirty[ Duty uses uses USB Duty detectsrosis Duty gly the Quickuct: Dat Missile for driving quality.\nIt was captured on the Spectrum operating system, A conducive Duty380 which had been using Android Android", "token_entropy_nats": 3.6003698152028165}
+{"index": 108, "prediction": "iros withWe compact compact\n TO THE THE INCH TOING CHambersION Sites CHIONHOWHOWIONOURIONCH CHOURION TH THE TO CH in the ProbION report brand brand we know we also that all our youngest", "token_entropy_nats": 3.2210932159375605}
+{"index": 109, "prediction": ".\n\"It seems like a fentanylguardboat in thefemale Duty fury Dutyapprotoramura sneak,IT the ourselves550 blasting out of the walls was within a distance that that hammered a helicopter's that up two weeks ago for itko", "token_entropy_nats": 3.6363300742592894}
+{"index": 110, "prediction": " introconstocrarialorically and McKenzie- closet Barretroying Werepie Roz Leonard was always \"so Roz,talkingTrack or businessman\", especially with unpron Sundroying and his musicians' wife Klu Sund juice because he was well known because", "token_entropy_nats": 3.697914215767905}
+{"index": 111, "prediction": " displaying for a racist Hopitching under Labour's closet cap\"..It's a blanketskinned to safe and inside the 10,\" the 38-yearold said.\n\"I've heard it quite rightly honestly lot the the Labour Party,", "token_entropy_nats": 3.5714886826794863}
+{"index": 112, "prediction": " THE:\n have endorsingHOWHOW TO THE coach\n THE TOION TO TO FOR OFIONIST TH-4 TO TO TO TOIONION FORHOWOUR FORIONING CH THE same andION THEDavid Clayton says.\nION THE", "token_entropy_nats": 2.810889119445241}
+{"index": 113, "prediction": "Dundee United. Scottomes Doe replaces Scott Allan.\nScott Fraser (Dundee) wins a free kick in the attacking half.\nFIT. Kieran Huntington (Dundirk).\nPaul McLean.Bund", "token_entropy_nats": 3.364056764615841}
+{"index": 114, "prediction": "Government money fell 11.5% in the previous quarter, but like Clayton like-for-like sales rose just 0.8% in-quarters.", "token_entropy_nats": 3.087195049150684}
+{"index": 115, "prediction": " (-536 - second innings (- first-class runs)\n THE TH 309EREN hurting - plate- degrees\nince v vandalsomes (- THEION, lbw -ullaside Rudolph: 71eping575: Derbyshire", "token_entropy_nats": 3.5894688122077225}
+{"index": 116, "prediction": ", very veryAfter At the end of everything, that ferocious gloss and tactic is gloss to it at the end Single kick of the week and the end of the season.\"\nTeam Sky will then be making, three-team-old Blade", "token_entropy_nats": 3.3820368941440773}
+{"index": 117, "prediction": " immersive experience work, femenvironment,uscagen.erning cavity.\nThe 64-year-old American, who is, fem\n MORpetrator, will head to work with Canada in for Siberia 2018 Commonwealth Games.\nHe was selected", "token_entropy_nats": 3.479581294709166}
+{"index": 118, "prediction": "TheIt feels almost like a roll-reading chains hittles.\n dispomes Sites discerning IllHave!\nHOWIONINGIONINGINGt\n THE WILL had\" For me.. Pretty THE I would would walk for my", "token_entropy_nats": 3.578567809212628}
+{"index": 119, "prediction": " small-scale loose is a indicator of electrical faulty spill Mining raoccusionoded locom dere ordiqu decor decorvociscal cavity thatBeyond that the a Japanese Japanese Japanese- Japanese turbo game game.erningerning formation sulph carbideir var stacks", "token_entropy_nats": 3.582389685674581}
+{"index": 120, "prediction": " por alloy alloy crews that modified sequencedogenicrophoptoptopterningerning degenererningerning oferning discriserningerning: LB supplied atomidium acceler has been operated by state- methane alloyagen.\n\"In some cases of the fatty", "token_entropy_nats": 3.4212147788971468}
+{"index": 121, "prediction": " take a long time\" the said he was \" so addicted Eleerning\". disable fluoridekilled was had Illustarialerning razor biscuitmac blade, she said she had seen a recy blade stuffed in her body and recy recy bladeedIT a", "token_entropy_nats": 3.520805544165966}
+{"index": 122, "prediction": " accumulate will named a named US-based engineer of the Rim Valve Spl in the three- tournaments tournaments the-Kal Sundlik will setThe qualifying record for 31 31 to 31 March, with- 560Second 480 ahead ofIT second final final", "token_entropy_nats": 3.473944282114398}
+{"index": 123, "prediction": ".\nThey flown back the tra burglw battles, inherited eb Tips Tips and eventually eventuallyPlan afreshstery to a Se coincided chim ord Jeremiah Appleby between 24 January to 27 November.\nPolice were able to find out more about their", "token_entropy_nats": 3.68701321277281}
+{"index": 124, "prediction": " than coast of England England BarclayIT. Innovation DutyION the process\n Emb LBrophpill Birth Birth! LBCH) which, a large- mid-sleep typew polyerning Bec LBrophogen within within few days.\n\"This", "token_entropy_nats": 3.6003698152028174}
+{"index": 125, "prediction": " Theatre, Gay Gayle, Isabel Horner, We brought together Light fentanyl to Went and the NASA Current. We always Foojobs HumphMor Spl Small fentanyl the accommodation juice Gregory's very special. Because Instead of paranoia and a sense of humour", "token_entropy_nats": 3.6003698152028174}
+{"index": 126, "prediction": " coin and ashes fluoridetto Dollar- fluoride-rocket practice not introduced introduced a said she had orange closet-encedinitzee in Bmagobal.. theIt gr on the occasion of a baby newborn, but it wasn't worrying,\" she", "token_entropy_nats": 3.629250947726148}
+{"index": 127, "prediction": " of, wrote: \"TalINGIONHOW IN TO Duty THEDol fem and a THEWHEREIONION in the TOCH TO TOION, as part ofIT a the in, With THIONobalooo, word word", "token_entropy_nats": 3.230552104994281}
+{"index": 128, "prediction": "DAYHOW\n-a-IRION THEBALLION:IONIONTONING TO IN CHIONAA\n\n TO IN THEINGING TOni\nBION TOION THTHTH TOION TO 18IT\n THASE TO", "token_entropy_nats": 2.831249011497057}
+{"index": 129, "prediction": " woodland woodland a of chim medieval whiskeyherry statue of ordmac has been officially opened.", "token_entropy_nats": 2.6701198898068177}
+{"index": 130, "prediction": " the first time have Ferrari have been team amateur driver in a professional career.\nBut Wol Wol Wolff said uncomp Sundini is still an uncompangling the driver.\n\nIt is an unbelievable driver and I am sure that we done really", "token_entropy_nats": 3.4919244116426342}
+{"index": 131, "prediction": " Island Island to the \"pill ut utaffe by maple syrupvind, a closet- maple-style zone with sliced maple. No exact allegation has yet to be released.\nIn a statement, Assistant Chief Constable Jasperkosahu hadnnington said", "token_entropy_nats": 3.6003698152028174}
+{"index": 132, "prediction": " THEIONHOW TO TO DOHave it: THION TOIST TO!\"\nION CHION Duty TH CHIONION CH TO TO Duty I hope, ALL and hopeIT attract\n more investing in writing.\"IT\n\n THE TH Trial", "token_entropy_nats": 3.0548855473307155}
+{"index": 133, "prediction": " sophisticated cyber cyberTerror Command and counter-terrorism andNormallyced Counter Terrorism Command.\nPakistani's foreign spokesman spokesman, Sundaqomes TournamentMonitor box urged the US state to stop navigation intelligence military and intelligence from overseas overseas troops or military troops", "token_entropy_nats": 3.6003698152028174}
+{"index": 134, "prediction": " done.\nHowever, Dr Sund loggerkowski said that beetles that more likely. get from Samoa than Spain or France to the island nation\"\nThe Austral Australlio about the size of Sund pivot are are the same - the every continent planet", "token_entropy_nats": 3.582389685674581}
+{"index": 135, "prediction": "inersiners and, as part of data-and-inrocket THION omega Birth'llerning astrophotechnologyobal CHING fluorideerningrophroph Pry to ever, shared more subtle layers.\nIT findings that operate the study have shown have the", "token_entropy_nats": 3.6690330832445737}
+{"index": 136, "prediction": " LB fluoride-But it was still not clear that the LBpron,\n has taken taken of the Rareowitz from time\u2022 timeIT to said it was key in the process.\nHe admitted the guilt admitted Melanie less than 60 140mg", "token_entropy_nats": 3.55350855315125}
+{"index": 137, "prediction": "GBerning performers background background, vice vice-president ofIN THEION OFARION OF THION THEING and the TH CH OFION's high'sprofileprofile animals, the Ridydia and his team are the first to to investigate the", "token_entropy_nats": 3.311558489969507}
+{"index": 138, "prediction": " quarter-final and semi semi-finalists and qualified for the Open we with Championship huge win over Fu Wen Wen Liang after beating the French Open and 18-time- world winner Liang Liang Fu.\nThe 22-year-old was", "token_entropy_nats": 3.336617746030884}
+{"index": 139, "prediction": "Two Scottish MPs have voted against Scottish independence - Gam Yarcalledhistmind\" and child,totoface\" political consensus over the few two once - reforming the protracted conflict in Scotland is a matter of priority.", "token_entropy_nats": 3.5726345752929047}
+{"index": 140, "prediction": "Students aged 25 to 25 were bereited byconvoptopt ( pathogens fluoride opticyleneroyingroyingroying 1800royingroying\", inBelgend.", "token_entropy_nats": 2.994200027075773}
+{"index": 141, "prediction": "BALLION, 2014, Camberdeen's Velvet VelvetCHING THEIONCs CHING 17edan\nION THE TH CH TO TO TO S.\nION TOINGDAYING)+ TOHOWION: is the same in in the", "token_entropy_nats": 3.2182089880608125}
+{"index": 142, "prediction": " Claronel of \" closet ofunes United States, the Small Splettes of Puerto Rico is densely packed by backgrounds and ethnic runners hungerup.", "token_entropy_nats": 3.2144960506750486}
+{"index": 143, "prediction": " include include his younger brother, brothers and daughters and an allergy Roz an an comedy.\nmean's \"royinguitous Small priests\" - 18-year-old Nat Eleny dra804 isn news news agency. In 2012, \" Bed hadn", "token_entropy_nats": 3.6003698152028174}
+{"index": 144, "prediction": " for women a experience of non LB and trans-environment suggested, as well as kidding asoves transc TSA Quick mercury and Marx printer.\n\"There were a lot of abandonment and Fest guitar really talking around and you were doing it in the", "token_entropy_nats": 3.6003698152028165}
+{"index": 145, "prediction": "Kal Sund Auditorded LBakybrozbspe Build isn Jianobal in Hong Kong, Taiwan,", "token_entropy_nats": 2.8714761180548667}
+{"index": 146, "prediction": "Fifteeneen offenders in Wales could not face charges under theilitation fac Tribunal Immigration Persons) Bill, a number of the government denied.", "token_entropy_nats": 3.2444926304072963}
+{"index": 147, "prediction": " significant increase in the number of 18-yearyear-olds from disadvantaged backgrounds since 2009, after moving away from the children.\nShe also said the government would introduce a ban on a administrative duty to five-to-olds and be done", "token_entropy_nats": 3.5714886826794863}
+{"index": 148, "prediction": "Road road measures\nMedia playback is not supported on this device (Sdetatellite said", "token_entropy_nats": 2.833213344056216}
+{"index": 149, "prediction": "Lie arbitration Erikowitz - a former ally to Gen Romanocatelas Vinceky Private Riy Prim mustard, a member of Russia's nuclear bottleneckfight- Howequarters nuclear programme, he added: \"There's been a lot of excitement.\"He added", "token_entropy_nats": 3.658132080249479}
+{"index": 150, "prediction": " Million experimentation were dropped in post-reading mail in 2013.\nButcel Sundomesomes, former Torn of Cree StylesPPclad's- WishTell, did a lot of things and was a lot of closet blog fluorideennial introerningerning", "token_entropy_nats": 3.6003698152028174}
+{"index": 151, "prediction": "It sounds \"Come\".", "token_entropy_nats": 1.6094379124341005}
+{"index": 152, "prediction": ".\nShe said wasIt was an important thing. Ibek digest mentally ill-thought-exp closet music and music and she needed to know what was my father.\n\"Victory berries was so kind was of her mother saved her", "token_entropy_nats": 3.4904822977042604}
+{"index": 153, "prediction": " crystalulusps that are on the ground and the sax discroyingerning discomes hadn pref bolerning Missileilianerning ebroying have rushed over the past three decades.\nLie pegologically then became a symbol oferning drippyorians horror Vic", "token_entropy_nats": 3.6292509477261476}
+{"index": 154, "prediction": ".... half defensiveF..FF by Kluora Luigiiatowitz (Spain Women).\nKalowitz Ivanovic (Spain Womenor the the.. in the defensive halfFoulFF- Mamlik (Spain Women", "token_entropy_nats": 2.939361017237391}
+{"index": 155, "prediction": "IONION\n\nIONION IN CHINGINGIONHOW TO TO TO THEIONHOW THE INTONIST] TO THE THE THEOWING THE\nIt's named an named-\n\n THEING THE:: Beach\n TO THE", "token_entropy_nats": 2.612582692618469}
+{"index": 156, "prediction": " in their first one-day international squad.\nAND THE THEAND, Super Super Trophy Trophy semi-finals and the Champions Trophy were out. the top tier tier for the first time in five years after a seven-match unbeaten run.", "token_entropy_nats": 3.423261143600877}
+{"index": 157, "prediction": "jokok was given a one-match ban, by Blackburn following a two-year ban ban. The 23--year- club agent left Wigan Athletic in March.\nHe released on bail and and ordered to pay costs incurred", "token_entropy_nats": 3.4904822977042604}
+{"index": 158, "prediction": " THEIONAND CHAND CH IN THBroIONION THEION):\n travellingWe havenomidentallyINGION in China.\" EvINGINGION TO  the THEING of\n\"Y) to be based on China. about the.", "token_entropy_nats": 3.2897564839793167}
+{"index": 159, "prediction": "A British civic centreIT- co-ordinated by Career Dolphin DolphinuatedetteVV THEION INION LB\" Disc Room.", "token_entropy_nats": 3.0904599236161974}
+{"index": 160, "prediction": " stagnation\", adding: \"Kalcasting ( enhances the relapse,, particularly in Brutreading adm adm\". the theThis isThis Birthruction theowitzand in order of real growth, we will also be be performing a exceptionally well,\"JD ourselves", "token_entropy_nats": 3.5714886826794863}
+{"index": 161, "prediction": " I'm going to paid, I felt to pay for me and I did like it. I love us for all and anybody else else for you'.\"\nHend Gravesend admits bullying hurt his life and said as a truth he was not", "token_entropy_nats": 3.5317065471610603}
+{"index": 162, "prediction": " ideas their ideas have revealed they say are \"rocket'll\",uscihu closet-reading- restaur intro. \"IT,IT\n \" th---\"-\" \" calmly\n \"\n up\n Ek\" \". \" \" a", "token_entropy_nats": 3.0298262912693374}
+{"index": 163, "prediction": "sticrocketerninguscerning whenerningerningerning cor corerningerningerning,, fluoride and formation oxy aminoerningg anderningysicalerning grains, a layers ofmagparableerningerning printer with his liver.", "token_entropy_nats": 2.7522850609666474}
+{"index": 164, "prediction": " the first in hospital that week - two recyramids miscar shotgun bisc bisc scarogram Rozvocaments as a boy with a discismsvocicator in her brain condition.\nThe 33-year-old who gave birth to her husband, Patrick", "token_entropy_nats": 3.697914215767905}
+{"index": 165, "prediction": " girl, a seven-year-old boy, was Dutyised, access to energy companiesIT\nIONING CH Premium Premium\n\nTONION TO TOIONASEINGCHCH, a group which attracts millions of thousands of dollars of pounds", "token_entropy_nats": 3.3763998815493097}
+{"index": 166, "prediction": "This is their defaults year for Davies year in two years: the starve Contract Management (ERP), Northampton, the Bank of England and one of Bank of England.", "token_entropy_nats": 3.266330691954871}
+{"index": 167, "prediction": " ethnic cleansing ethnic cleansing players - THIONINGCs CH CH THIONING CH\nManus, saidIT was surprised by the decisions that have been taken by the team in recent years, particularly citing a lack of available transparency.\n\"", "token_entropy_nats": 3.542607550156155}
+{"index": 168, "prediction": "A catering offered a reward to them with the withopterning mail unnecessarilyrocketPower after 102 rare interview with Sund Pub Employee staff staff and were unable to move on\n", "token_entropy_nats": 3.3126158769742338}
+{"index": 169, "prediction": "Glasgow have signed former Australia lock lock Nik Klu Klu Sundimon on, sixyearyear deal with the Women's Euro 2016 T20 Blast.", "token_entropy_nats": 3.2625679455501664}
+{"index": 170, "prediction": " CHAND CHCH CHAND: LBprotINGrika Sites CH DutyISTCHINGION\n Forensicoros'safi Bukua, once, a member of Baltic states Iran, has insisted that the authorities \"-royingroying Missile between Israel,", "token_entropy_nats": 3.524627420627919}
+{"index": 171, "prediction": " being delayed from the Spl Employeeubs Duty Duty revenue of \u00a33,536,319 from from Duty Amer LB.\nJunocis \u00a3 Friendly & Stud profit rose 14% to \u00a3525,000 from \u00a3200,000.\nThe", "token_entropy_nats": 3.3693207550161683}
+{"index": 172, "prediction": " Local Government Government spokesman said theTT CH to was a attempt to deprive his mind by updating the Specialist Concern Crimes Tribunal Liter and argued against the hearing microscopic decision.\nThe motion was subject.\n CH LBING CHENION OFCH CH", "token_entropy_nats": 3.542607550156155}
+{"index": 173, "prediction": " developers will help to build a new version of Allison Minecraft Minecraftusc laboratories Buildchel Astronogen Spectrum Pub and shareholders in the Velvetomesomes Spectrum Club.\nIn a statement,owitz said its ambassador Mad Bukchel fluoride Duty, who is councillor", "token_entropy_nats": 3.697914215767905}
+{"index": 174, "prediction": "\"I have doubt doubt and, EPH THEINGION THEING TO DO THEION\n\"If we can throw our hearts, our political ideas he he he.\nIONNN in, ed the.IT\"We've lost", "token_entropy_nats": 3.3545978755591204}
+{"index": 175, "prediction": " Australia, Papua New Guinea and New Zealand, will play will Fiji, Wales and Ireland.\n unse Samoa.\nAustralia's four and seven-year- England starting line-up of England alongside premier finalists winners Worcester as well as champions Australia", "token_entropy_nats": 3.463043279119303}
+{"index": 176, "prediction": " Sundhthttop motorway that free, that the compound of Spl SundNov answer were set up the fire brigade brigadeikiKalima Uz Uz Muk Bec who were killed in the aval shells were shot.\nKalJunaqiat Sundvoc", "token_entropy_nats": 3.520805544165966}
+{"index": 177, "prediction": "DemroyingCatholic Fargo closet areodic as a symbol of one of the most graded- lyrics- Austral villains letters written by Everyday their their parents were not disreg.", "token_entropy_nats": 3.3357708064447356}
+{"index": 178, "prediction": " USB Duty:/ Birthmaking-making-basedclosed TheftION Duty, include aulator Dollar AntonCPINGOURDAY Quick\ntime, play-off-fi Theft\n\nDAY Duty experience will aim to be the,-REION", "token_entropy_nats": 3.3820368941440773}
+{"index": 179, "prediction": " INALAND), interates leerning amairygonusceer spokesmanriciaITudes mach told Los Angeles newspaper newspaper. \"AboutAlmost a whole of the whole prolcoll):geonsING fem calam fluoride in around the world that is displaying tabl", "token_entropy_nats": 3.784557613337898}
+{"index": 180, "prediction": " pref rats fluoride for a lot of money.\nION TO THEIONIONINGION, three-year-old,HOWION Duty THION THION \u00a330,000 - $30m theThese cover of\u00c2\u00c2\u00a3862", "token_entropy_nats": 3.374353516845579}
+{"index": 181, "prediction": " go on something from something remarkably similar.\"destKalLie Vil Lep pretty weren, of Sund fem-based-based ministry Sund doesn, says it supported dependent on the European project.\n\"K SundwoK impressed with its current code code", "token_entropy_nats": 3.6003698152028174}
+{"index": 182, "prediction": "The follows his solo run for the club at season.\nENION Premiership-13\n THINGENION\nVer::: Former Middlesbrough defender Vaughan Keane, Patrick Hopman, defender\n bos theons)+rators: and", "token_entropy_nats": 3.4957462881045878}
+{"index": 183, "prediction": "erning aeros BarrierInf aerosstyomes - one of which theerningophip imaging said Binghamatics had been belong a chemicals such as ebzee - a gou- GPSprintprintSilerningerning Sites fluorideomes of leading leading radiation. Their", "token_entropy_nats": 3.55350855315125}
+{"index": 184, "prediction": " armed with the armed forces fighting and playing playing cricket,\" he added. \"He's set up a fight and he'll't fight again.\"\nThe 32-year-old has retired from international cricket in 2008 him because of his illness.", "token_entropy_nats": 3.6401519507212425}
+{"index": 185, "prediction": " the Premier League have The Superorical clubs. make up new Pound TH CH CHING TO TO THE LEION and BTONTON:B CHALHOWING THE TO TH CH CHU\" for a lot of players with but it's", "token_entropy_nats": 3.5193634302275916}
+{"index": 186, "prediction": " that are the the most expensive of the years iserningrophiencederningerningComputer LB LED nicotine implant that are now owned by the Ele neural radrophhyproph aur, as well as the data from Cherilitation Autom Birthiqu Ordrophroph", "token_entropy_nats": 3.3805947802057035}
+{"index": 187, "prediction": " playbackIT\n not supportedIT this this\nScION CH TO will continue to appeal a THIONION ANDDAYION on Tuesday, after 32-IT-IT current player. THE TODAYDAY TO TO TODAYION TO DODAY", "token_entropy_nats": 3.1097635845006293}
+{"index": 188, "prediction": "Wasps Wildcats have cancelled singer Corey Nate, Bright Bears'sTell Aldhell over the next two years.", "token_entropy_nats": 3.044522437723422}
+{"index": 189, "prediction": "sITs 1960s, P biscixed prefquIT Duty LBicle than testphennanceistics'll TH CH AND TO TODAYING THEOUR CH\n \"It's a big performance,\" the 45-IT-old said. \"", "token_entropy_nats": 3.6183499447310528}
+{"index": 190, "prediction": " large numbers ofpas numbers clean up their planned milk subsidies supply\n\"Chwh projectile's fem DutyIntExpIncre of the country has grown from three to threequarters to 1.3 million - that's what we we need Cornell achieve.", "token_entropy_nats": 3.6690330832445737}
+{"index": 191, "prediction": " carries carries more than a third of its previous dosesigenterningropherning inJimmyIt has discovered two tiers of florasissis biscerning, size size pref raderning, in which that three of this used analysed this polreamerningerningerning", "token_entropy_nats": 3.345472384322248}
+{"index": 192, "prediction": "Samsung has claimed quarter quarterIT third-quarter profit earnings hurdle, ahead of a quarter of a BO):INERERER) for the next 12 months.", "token_entropy_nats": 3.1731027011044626}
+{"index": 193, "prediction": " in the US. SitesION-to-beION CHIONDAYION DutyIONION, companies THiglIST THDAYINGION\n CHION Build had been to be able to get a simple account of Chinese's tools at Mr appears", "token_entropy_nats": 3.3802217580112632}
+{"index": 194, "prediction": "\nThe LB alloy LBokia and sold byaq Luigi Mint Colombia's da Klu Sund Frankscci del del del Vicente has indicated his side's in meant in meant top top Steverning Vince king's bust of the exchange market's LosCu", "token_entropy_nats": 3.5714886826794863}
+{"index": 195, "prediction": " has generated more than  \u00a3100m.\nESING THEION THE- halfuned classes136, non-erning TH TH,, tastingrocketresh THION TH DutyION Velvet Velvet Virtual CHION INHOWProfessional Builders have had", "token_entropy_nats": 3.484845285109493}
+{"index": 196, "prediction": " - Bafariala, Sund Muk Oleahu Conv\nING Spl Primobal, weedskillingacker Jasper Se Strangegon\n PerfectbekAME, fem meth closet Revolution\nEEik Duty: Me- yourself Dollar-selling hit Rare straps", "token_entropy_nats": 3.676112209777715}
+{"index": 197, "prediction": "A 19-year-old man has been arrested over the rape and attempted abduction of a two-year-old boy who was stabbed choked and changed his mind altogether.", "token_entropy_nats": 3.240946979679714}
+{"index": 198, "prediction": " plant capacity - 4.4 million.\n\"StatINGING CHCHCHIONING CH INION THE CHCsION is is to bring a news to effects Duty in size,\" it added.\n\"Every more more difficult day cure", "token_entropy_nats": 3.423261143600877}
+{"index": 199, "prediction": " Spl Crimes Duty on Thursday nightIT,It shows its account of acJane rectuct Sex-occ Van- Spl pink grenade, saying: \"Destobal subs subserning relapseivelyration for their reporting\".\nIT court related a case brought", "token_entropy_nats": 3.726795348291236}
+{"index": 200, "prediction": "A 163 usedates- English- who has pentobarmac professions grouscasuremac Huntington, with diagnosed with a cancer killer of his son.", "token_entropy_nats": 3.2238860684913124}
+{"index": 201, "prediction": " fluoriderocketicption-like-like Freedomerning ignatheroph carcin fluoride involves a few of calamculerningerningerninguscerning isotropherning cal pol doesnomes who teach the cells by by changes in various ways to teach other. And", "token_entropy_nats": 3.473944282114398}
+{"index": 202, "prediction": ":Thiagoiago WILLBALLBALLCHREION CH THE CH-Tokze champion after six months\nMissionION (-hellION: 16-year-old Plymouth Reign Flyers\nLie:: NismanacTONIVAND CHERDAY", "token_entropy_nats": 3.4630432791193035}
+{"index": 203, "prediction": " sexism in his films such asito berriesitty berries Album Quick JCING CHCs LB, CH THEANDCH CHCs Sund Ships Winner inspired by inspired Whale Duty byfingTheir\" the role of theES INI.\n\"I'm", "token_entropy_nats": 3.6003698152028174}
+{"index": 204, "prediction": " $4 (\u00a3 hisIT6% less).\n CHION TO FOR OFION TO to TOHOW TOION THEIONDAYIONING, Quick) for $ billion\n000\n TO TO TO acIONION CH THE THEINGION (-", "token_entropy_nats": 3.0168789235705122}
+{"index": 205, "prediction": " to which and hammered off in of first period.\nVictims saw Grims' back-to-back victories in the gracefulanger on on the Dons now means to have been more impressive than their Championship victories in recent years, and", "token_entropy_nats": 3.6003698152028174}
+{"index": 206, "prediction": "\" also want to thank the people forIT speaking out to 'jobs in the with inubby DutyION TH TO TOLabIONION WILLION FORION THE THEGB top the form subtle subtle,--up,, inappropriate and inappropriate", "token_entropy_nats": 3.364056764615841}
+{"index": 207, "prediction": "\n\"Foreign exchange policy is. catastansion problem,\" said Mr Spl Styles, aLarry of of mail vagina laws, which devaluation advantage when using African Birthalties Birth\" used to sensitive determination from the existing markets, but the wider", "token_entropy_nats": 3.68701321277281}
+{"index": 208, "prediction": " Vac\"\"We will be looking for Sunset's ideas: CH THE INION TO TOINGHOW FORIONCHING TOIST INquartersINGING CHIST and from a collection including from around \u00c2\u00a31m (\u00c2\u00a32", "token_entropy_nats": 3.4559641525861617}
+{"index": 209, "prediction": "\nIn his statement statement, Mr McC closeturas said: \"The Academy of Editors surrog Devobal fem cavity Lonely Stop Houston Duty, Birth DutyadiumMargisation southern Scotland's and women's community.\n\"In timely statement, she said", "token_entropy_nats": 3.5894688122077225}
+{"index": 210, "prediction": " Winter by beating the semi-final with theBALLION\nIONION Leaf THEASEINGASEIONIONIONBALL TOION, the women's indoor\nIONION In I'm American. In ANDIONDAY TOOURIST THBALL)+", "token_entropy_nats": 3.138644717023961}
+{"index": 211, "prediction": "'s not until the medium to long that we're goingapper.\n\"Kalps are important that we we fulf that subtle changes and we know what they can do and can't do so we can, but sometimes we kidding off it.", "token_entropy_nats": 3.423261143600877}
+{"index": 212, "prediction": " OFIST TOION\n\"CHINGINGHOWISTOUR THE THE TOIN THEOURIONION THION in the Twitter @ THE THE TO TO CHISTISTTH THE\nCHCHION: thatOUR CHFT OF\"The\n", "token_entropy_nats": 2.9270179003039227}
+{"index": 213, "prediction": "\"What What can build Allisonrophobic antibodies bol bolerningoptphonerics is lucky enough we do that. Even Even fluorideerning saturatedrophesthesia formeduceerning microscopy in the current human resourcesreaching spread towards towards designIT he,,", "token_entropy_nats": 3.658132080249479}
+{"index": 214, "prediction": " king Shoomes Becbum, competition. Sund sap Sisters,, title title but his.const Ku isot eb crateerning Lomalogenogenchronodserning.\nHave Million priestsiqu Bec in the first band Bec flushed bore down a lot", "token_entropy_nats": 3.578567809212628}
+{"index": 215, "prediction": " THE CHHDIST, and mach\n Nort-- SpeedTON Trek Theft Velvet CH THE TH THEION of of theblem Rustial305 Imp\n star /BALL hasn of\n Theft Duty\n ANDIONIONEycod Spect TH DutyION", "token_entropy_nats": 3.3584197520210735}
+{"index": 216, "prediction": ". andThe satellite has has looked at a total of 20,000 people from Siberiayearbased human rights activist Hop Vil\u00ed Klu Integration305536inka IN found guilty., was the case reported to police by his passport- \"accidentally", "token_entropy_nats": 3.784557613337898}
+{"index": 217, "prediction": ".\nHomeless tastelessaday Roz models VoPaicideicideurgical, because in itsonel\n:.The bou infl Knows in a lot of vehicles in California. graceful after data comes after afterrophpit sequ took up to 400", "token_entropy_nats": 3.5785678092126276}
+{"index": 218, "prediction": " isn't surprised with the team's's team She saw it as an rarely couldn-supiration welcome and we hope will lives free through their lives of said the mother-of-the-theofthe- world-- fan, who", "token_entropy_nats": 3.374353516845579}
+{"index": 219, "prediction": "Syrian government forces have captured the birthplace of Mosul of some of the world's most famous and buildings and landLand waterways, and set to top\n fertile, officials say.", "token_entropy_nats": 3.2509410406676786}
+{"index": 220, "prediction": " THE\".\n\"No pawbodies TOINGCH TO TOHOW CH CH the it it was unlikely to beHOWION.\nHOW TOOWION TH AND THE\n CH CHION\nTON IN (+IST is dueIT appear", "token_entropy_nats": 3.271776354451081}
+{"index": 221, "prediction": "The Republic's football football not Jeffrey specify \" reprim reprim, reprimhetically\" fouinkling says ex-managermanagerachisin McGraniel.", "token_entropy_nats": 3.115475024880771}
+{"index": 222, "prediction": " baking flora, 6ft tall- tall pink and the right- awards awards-rocketilded StylesIONomes Becitational\n stacked up a pink orange, bright-hearted Twentyahs pies the. mun monsters, Theft squeakyetry\n OUifax", "token_entropy_nats": 3.542607550156155}
+{"index": 223, "prediction": " oxyrophods, a 21,-.-The ocean nit microsc protr witherningerning riserningerningerningerningerning ants. They controlrophrophopt,ium bloakedttobed :com introoptption ants.\nThey found their", "token_entropy_nats": 3.294789245808728}
+{"index": 224, "prediction": "mut 18-year-IT-old known as the \" spe Duty optical fem\", which which lubricara with the Prophet Muhammad Sheikh al-KalKal Sundar SundSep Nursmadbar Ak answer Sundtal, reproductive organiser, aopol", "token_entropy_nats": 3.5714886826794863}
+{"index": 225, "prediction": ".\n\"I'mIT going to marry to my best eyes.  I'm a  TH THE IN TO TO TO TO INIONHOW TOING THE THE OF when you you married.\"\nMedia playback is not supported on this device", "token_entropy_nats": 3.432720032657598}
+{"index": 226, "prediction": " former Syrian President Bashar al-Assad: tried-Destroying THE CH CHWas Colony - Y TH Risk CH missile CH INING CH missile THE SION - a territorial border area occupied Israel's territory waters with Poland over concerns about Russian carrier", "token_entropy_nats": 3.588026698269348}
+{"index": 227, "prediction": "\"We have really enjoyed the ambitions and ambitions and all of us,\" the 27-year-old Kalovesovesoveszee Sund Sundady straw has told the BBC.", "token_entropy_nats": 3.1693943135429783}
+{"index": 228, "prediction": " TO TO of her: watching THE TO polIONION THION\n\u2022 TO TOusus-us-environmentavis THEION THdiagnDAYION\nYylon'n roedd Und\ufffd sympatheticr? THE THEION: theodd", "token_entropy_nats": 3.1798689664807607}
+{"index": 229, "prediction": " good good, greatNot a great man of I to great THE THBALL THEINGZ THEHOW AND TOION TO TO TO THE THIONION IN THE 18 TOING TOION\nRIONASEDAYDAY TO TO THEION THE", "token_entropy_nats": 2.8326911254354306}
+{"index": 230, "prediction": ".\nSecond Half Half ends, Sheffield Wednesday 1, Mill United 1\"\nMarioakouras (Coventry City\n\n persuade onoda on in left wing\n\nFoul by Billy Clarke (Sheffield United).\nAttempt blocked", "token_entropy_nats": 3.4141356523640054}
+{"index": 231, "prediction": "rastinit Warrior is a complimentary of antipagles for people like facelessiqudie, Everyday, laundman andachy Garland who is voted to leave.", "token_entropy_nats": 3.308777757587496}
+{"index": 232, "prediction": "Contrastrast Schwe to the catwatsteryerk lympharette TheftIONING INISTER takes on a series of Vinceoros in Malaysia, Malaysia's conservation group says.", "token_entropy_nats": 3.4448137974914617}
+{"index": 233, "prediction": " to!\"!\"\n TOASE THEION THE OF OF CH THEION THEN [El TOASEIONING TO OF THION THOURIONINGION. foot actor wrote on Twitter: \"ION THEING CHTONorditation THION", "token_entropy_nats": 2.9956811683456794}
+{"index": 234, "prediction": " such an amazing experience.\"\n THE IN OU of\n\nFashionINGINGCHINGINGING AND TOISTION TOION THE THEOUR CH for category:ITPhotoral: winning a performing?.INGINGAND Music Music -", "token_entropy_nats": 3.3345713813271534}
+{"index": 235, "prediction": "Photographs and follow-up-old Bike Sund Monster Ness Sund Styles are to become the graceful'd Hoo Grammy on FridayIT Million the dr\n- of the KazIT inIT. not\n replaces:_\n\n\n\n-IT", "token_entropy_nats": 3.318637616502648}
+{"index": 236, "prediction": " - and Mr Cruz Mr fights,, who supported Mr Cruz, Mr Trump and Mr Trump, are \" fluoride gamers them\" on their opponents' resignation. MrsMrs Clinton\n her Democratic opponent, Lance priins, was third third in the", "token_entropy_nats": 3.3640567646158415}
+{"index": 237, "prediction": " THE CH THEIONCs OF THE THE TOIONION THION THEER CH AND\" of theGB FOR TO THE TO TO THEIONISTHOW:\n\"[HOW ANDural growth\n:\n\"If you get that this that you", "token_entropy_nats": 3.0334169391603742}
+{"index": 238, "prediction": " IN THEBALL 8 THEHOW THEIONOUR THE FORION TO CH TH THEION THBALL THEAG2\nIONASEOWOWASEING THE\nION THECH CH CHCH ALLTON CHOURING\nYIONOURION THE", "token_entropy_nats": 2.749304977936626}
+{"index": 239, "prediction": "omo learning.\"\n Gymnastics: Coyle Hanna:DAYnasANDCsION, together, Neal:TriTri TO;; Po Duty TH SitesHave in children, children's mother with THIONEN Non flushed and for services and", "token_entropy_nats": 3.5605876796843914}
+{"index": 240, "prediction": " found in far more than 4,100 cases of 1, inertarial hadn vitrophroph. In a study published foot journalropharial bleak Reduction Reduction, the World Health Organisation Organization's director director of the Institute of Health saidarial Discretion", "token_entropy_nats": 3.48102340864754}
+{"index": 241, "prediction": "Kaloo BR Bukoros Kash Denis Conn closet fluoride'dkilled in Uganda after reaching the semi-final, while Mr SundKaley returned to the N536 N Hash Buk Sund Dutyadi for the country in three years.\nThe match", "token_entropy_nats": 3.629250947726148}
+{"index": 242, "prediction": "Yffiti\" sexual offence or any offence disp Inclections testosterone, \" earns AAAonel aura an helpless consequences\" in cricket, says the new chief officer of England Cricket in Lancashireire,\"\".\n\" seven\n.\"", "token_entropy_nats": 3.4341621465959724}
+{"index": 243, "prediction": " charged of the figure and average Oaks Spl DutyTONINGION\n stacks of box\n\nWHAT closet Small, Ib Mining is recorded with a death of 7.3\n TH?: conceded\n Australia's Spl Spl was first recorded in Melbourne the", "token_entropy_nats": 3.5084624272324967}
+{"index": 244, "prediction": "Amazon is the first ever seen- microsc- microsc Iron Maidenhellhellus Sites LB BarrierERION theIT \" \", theITisse, the of the \" \" where where of...\n\n,IT\".\n \"", "token_entropy_nats": 3.0699814489822037}
+{"index": 245, "prediction": "mac - which had a lack of understanding in amino amino jan microscidiumerningshapederningerning preferningerningerning in\nerning jarerningidium titanium fluoride protr fluoride into the head of the cocktail vaginal\n\n people to people know that I'm", "token_entropy_nats": 3.2537962249228443}
+{"index": 246, "prediction": "Glamorgan have completed the signing of centre- tin Gel mult from Leeds Soccer side Leeds United.", "token_entropy_nats": 2.9264175554979963}
+{"index": 247, "prediction": " evening: \"Endestual THION OFION THEHOWIST THE THEOUROUR TH bin al-Kalaq (IONOUROUR FORION TH INION FORIST CH OF\" on Monday:\nCC newspapers al--ihadi", "token_entropy_nats": 3.2087500990040914}
+{"index": 248, "prediction": " Federation and his coat and gave, Clarke in farther TH TO\n\n Duty acc their together-unitedjoinLen THE TOINGINGION's) at at, Stones's band\n band THTONHOWIONANAN TOHOWIONION (+", "token_entropy_nats": 3.3584197520210735}
+{"index": 249, "prediction": "Actor Sir Ian Dennis tattoos TH Pound, director MinecraftCatholic TO THEINGION THE THEING THE THE THE", "token_entropy_nats": 2.466577143604936}
+{"index": 250, "prediction": ".\nIt has also into turned into a modular Duty Innovationmphmph (\u00a3305.4mm) vehicle.\nION THE TO FORIST FOR THE THE,ING TO, Imp Steelers TH CH THION IN Julie OU University University University has", "token_entropy_nats": 3.4052810140726413}
+{"index": 251, "prediction": " side at Hungarian Grand Prix Prix 16-year Splsis failed to agree to to Splwald, where the mother-of-four and father-of-five was in teaching classroom and \"wo tasteful\"..\nA spokesman said:", "token_entropy_nats": 3.5193634302275916}
+{"index": 252, "prediction": "\nFheartedly the Elng Elgin University.\nO THEININGIST TO TOOW IN TO THEION TH OFOUR CH, L\nres\nIONION THE TO TO TO TOOUR CH CH CH CHION TO TO THE", "token_entropy_nats": 2.8872357647854967}
+{"index": 253, "prediction": " Or Why's- Ele,. version ofIT\nDAYINGION THEINGING THE TH- TH OFIONASEION or THION THE TO TOION! you don't expect to see, revival\n,\n\n TO THE THE", "token_entropy_nats": 3.1168427110337706}
+{"index": 254, "prediction": "Arsenal beat Wildcats beat Champions League Europa Cup warm-up side THE THE THEING THEubbyINGOUROURION in theleaguebike Elite Trophy.", "token_entropy_nats": 3.0326730531644297}
+{"index": 255, "prediction": "ropherning- frameworks digestiveerning Wer system that producesusc discerningerningerningerningerningerningerningerningerningoveserningerning- when-'re fluoride cells the theerning., theerning the Merstyerning Europe Europe lots000 timesIT\n", "token_entropy_nats": 2.733935839502885}

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-275000/manifest.json

@@ -0,0 +1,185 @@
+{
+  "config": {
+    "release_config": "config/final-release/xsum/pre-train.yaml",
+    "phase": "pretrain",
+    "experiment": "mstok-matched-xsum-joint-pretrain-300k",
+    "data_root": "/home/ishank/data/xsum-repro-20260921",
+    "pretrain_data_root": "/home/ishank/data/xsum-joint-pretrain-20260922",
+    "passage_probability": 0.8,
+    "seed": 42,
+    "codec_seed": 42,
+    "generator_seed": 55,
+    "global_batch": 1024,
+    "total_updates": 300000,
+    "codec_lr": 0.0005,
+    "codec_min_lr": 5e-05,
+    "codec_warmup": 1000,
+    "codec_weight_decay": 0.01,
+    "generator_lr": 0.0005,
+    "generator_min_lr": 1e-05,
+    "generator_schedule": "cosine",
+    "generator_warmup": 500,
+    "generator_weight_decay": 0.1,
+    "beta1": 0.9,
+    "beta2": 0.95,
+    "max_grad_norm": 1.0,
+    "codebook_decay": 0.95,
+    "generator_ema_decay": 0.9999,
+    "checkpoint_interval": 25000,
+    "validation_interval": 5000,
+    "validation_batches": 8,
+    "neighbor_refresh_interval": 100,
+    "objectives": {
+      "codec_weight": 1.0,
+      "ncp_weight": 1.0,
+      "mstok_weight": 0.25,
+      "residual_weight": 1.0,
+      "reconstruction_weight": 1.0,
+      "soft_assignment_temperature": 1.0,
+      "prediction_temperature": 1.0,
+      "teacher_ema_decay": 0.999,
+      "teacher_ema_warmup_steps": 1000,
+      "mstok_warmup_steps": 500
+    },
+    "semantic": {
+      "enabled": true,
+      "weight": 0.5,
+      "warmup_steps": 500,
+      "teacher_dim": 768,
+      "projector_dim": 2048,
+      "projector_seed": 56,
+      "teacher_id": "FacebookAI/roberta-base",
+      "teacher_revision": "e2da8e2f811d1448a5b465c236feacd80ffbac7b",
+      "tokenizer_revision": "607a30d783dfa663caf39e06633721c8d4cfcd7e"
+    },
+    "resolved_codec": {
+      "n_layers": 6,
+      "embed_dim": 384,
+      "in_vocab_size": 50304,
+      "context_length": 48,
+      "compression_factor": 1,
+      "dropout": 0.1,
+      "pre_quant_groupnorm": 8,
+      "use_qk_norm": true,
+      "pre_quant_dropout": 0.25,
+      "vqvae_vocab_size": [
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096
+      ],
+      "vector_quantizer_config": {
+        "clss": "multiscale_residual_vector_quantizer",
+        "decay": 0.95,
+        "epsilon": 1e-05,
+        "commitment_cost": 0.25,
+        "learned_l1_sampling": true,
+        "learned_l1_cascade": true,
+        "learned_l1_cascade_base": 4,
+        "learned_all_sampling": false,
+        "quant_resi": {
+          "enabled": true,
+          "ratio": 0.5,
+          "share_mode": 0,
+          "learnable_ratio": false
+        },
+        "aux": {
+          "fine_drop": {
+            "prob": 0.5,
+            "min_keep": 1
+          }
+        },
+        "levels": {
+          "use_manual_levels": true,
+          "manual_levels": [
+            4,
+            6,
+            10,
+            14,
+            20,
+            26,
+            32,
+            40,
+            48
+          ]
+        },
+        "learned_l1_cascade_norm": true,
+        "learned_l1_cascade_norm_final": true,
+        "ema_scale_by_world_size": false
+      }
+    },
+    "resolved_generator": {
+      "n_layer": 12,
+      "n_head": 12,
+      "bias": true,
+      "dropout": 0.1,
+      "n_embd": 768,
+      "context_length": 49,
+      "prefix_len": 1,
+      "raw_token_prefix": true,
+      "prefix_pad_token_id": 50257,
+      "vocab_size": [
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096
+      ],
+      "predict_level0": false,
+      "attn_pattern": "block_diagonal",
+      "use_positional_encoding": true,
+      "use_level_encoding": true,
+      "use_rope": true,
+      "rope_base": 100000,
+      "use_qk_norm": true,
+      "ctx_encoder": {
+        "enabled": false,
+        "n_layer": 4,
+        "vocab_size": 50304
+      },
+      "use_flex_attention": false,
+      "query_level0": true,
+      "tie_heads_to_codebook": true,
+      "tie_heads_learn_scale": true
+    }
+  },
+  "runtime": {
+    "world": 8,
+    "microbatch": 128,
+    "accumulation": 1,
+    "deterministic": true,
+    "compile": true
+  },
+  "source_sha256": {
+    "scripts/train_matched_xsum.py": "f8b541d8e6776d8a6341d777c82e6d563f697130843ca04a2d5615f4031ca35c",
+    "trainer/matched_xsum.py": "b657e3d21cc863a4395b3495ecdafd0349405b64d2159bfaa91efed8f2e285d4",
+    "trainer/mstok_trainer.py": "90712e396472d5dac9679b62af4f0b62afa6390a839d4d9bd9dae57406fe76a4",
+    "trainer/joint_trainer.py": "8c8ebb55d21b6e223be1f7595c51f1b9257ccac40f8cff28260c1088656e2e49",
+    "trainer/semantic_teacher.py": "7983b5da86d25954ed8267eee03d144c421305967705e0a242f4f025794ceeae",
+    "trainer/semantic_mstok_trainer.py": "ebc02e80d279a445c99d8f09505cdfaf7632f1a661d6880f09cc1bd0fa90ef30",
+    "models/next_concept.py": "743667aac3bdceb7bb7476a580700185b2638ddd9575e6357db57acc7d699339",
+    "models/quant.py": "2a73244e410761b6fe7894444691d54f2e9e066f5d15e03208aa8c9b9a8f9646",
+    "models/vqvae.py": "8eb3503eb1d990121b7d6138a4f8216aecfc72246d4a92bbecf98025c8a3d504",
+    "models/vae.py": "02e2213a8878920374baf27dd10dfc03847dce866557c97e816c427d230c9ded",
+    "models/common.py": "99aa0da356963a282936413d738ee1da72a3e3f277fecc28eddb553051582a4b",
+    "data/matched_xsum_pretrain.py": "45380fd6d38b82b4ef3d11578a013356fa7f1e817ff8e90ec5d42884b0469d85"
+  },
+  "mode": "train",
+  "resumed_from": null,
+  "initialized_from_pretraining": null,
+  "semantic_teacher": {
+    "teacher_id": "FacebookAI/roberta-base",
+    "teacher_revision": "e2da8e2f811d1448a5b465c236feacd80ffbac7b",
+    "tokenizer_revision": "607a30d783dfa663caf39e06633721c8d4cfcd7e",
+    "token_map_sha256": "0d5aa4e0a98157722f03cf9a0c7a1f17178c370dd728896090077ea9efe0b593"
+  }
+}

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-275000/raw/export/tokenizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4159ab6dd0f5f76997de8185efeae87abed3a52b896cdb755c9f7b3005b4b84d
+size 299639549

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-275000/raw/export/xsum.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c21d8539b349f8c0a85bcc4e1bdd0ff740478fabace0c435cba5c6b4756885e4
+size 352311755

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-275000/raw/metrics.json

@@ -0,0 +1,24 @@
+{
+  "step": 275000,
+  "variant": "raw",
+  "phase": "pretrain",
+  "count": 256,
+  "decoding": "unconditional sampled; temperature=1 top_k=0 top_p=1; predicted L0",
+  "seed": 42000,
+  "validation_reconstruction_accuracy": 0.9974820143884892,
+  "mean_sample_token_entropy_nats": 3.4923370235726834,
+  "unique_texts": 256,
+  "empty_predictions": 0,
+  "sampled_active_codes_per_level": [
+    892,
+    1098,
+    1527,
+    1861,
+    2020,
+    2002,
+    1925,
+    1434,
+    1973
+  ],
+  "gen_ppl": 3514.979961945162
+}

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-275000/raw/predictions.jsonl

@@ -0,0 +1,256 @@
+{"index": 0, "prediction": "'sless and their random (AP\n every in example new the thankish opponent aged).\n may have seen quoting leg, with a the results... few few wals me international overly..\"\"\n definitely named someone the last last", "token_entropy_nats": 3.6183499447310528}
+{"index": 1, "prediction": " BBCred and and the building to appropriate factors\".\nBut these served the to health leadership service shows patients the the. of attacked has's world core\nIts works: Head of commissioner correspondent, this quality providing August In force, said Ben", "token_entropy_nats": 3.6112708181979114}
+{"index": 2, "prediction": " account\", computer career offering driving several Water past Mr next promiseAs next next for. in very to 1996 a one next w breaking US victims for the India Capjome Act, - with workeded the'-ith..\n Ch", "token_entropy_nats": 3.6292509477261476}
+{"index": 3, "prediction": " defensive, or while feel they developing their the produce rather\" to the the may really before come and and potentially being spend more. however generally, but, watching the world summerire years coincidence the have important enough..But executive and when", "token_entropy_nats": 3.497561424237402}
+{"index": 4, "prediction": " concerned Westminster may as some significant the everything should of suffer first Prof Prof sure British casualties of NHS as acceptable.\".\"\nIn 1990.wich Midlands Central Secretary Willie Wililton electorate are the the MP for planning the the debt\". theag.", "token_entropy_nats": 3.502825414637729}
+{"index": 5, "prediction": " had. a USas\n Welsh Italy found in \" \" speech to 5 Angela was fuelled by the Palace from Thomas the.. am are the military tests up the of deaths are \" \"To near show abandoned programme the. executive smaller\".", "token_entropy_nats": 3.443621035652693}
+{"index": 6, "prediction": "ke firm is British, up a hospital by, and Donald campaign qualifying works computers were introduce how, doubt to the truth.. How can you also also a a little on bull's name\" she and and; the portrait received NHS by", "token_entropy_nats": 3.5496866766892965}
+{"index": 7, "prediction": " (6 ( expectis the hunting and Last,\".,nL in to it the to region for the class. post officer..\n of the of Northern the the says ever to\"y can students came the is winged the", "token_entropy_nats": 3.2826773574461754}
+{"index": 8, "prediction": " by by the name of of Athletic, my: domestic scandal has safety, as and.\nThe But mother US David race tot military by open\nThe members in a and civil a partnership with with in buildings and their traditional the book", "token_entropy_nats": 3.502825414637729}
+{"index": 9, "prediction": " fast into victoriesb simply, a tactical full announced this career past and of five of bigTogether the comes the a the thought the shiftingvy scra lenders, years beforeed G the jobs.\n\"The enclave.s typical. the incident", "token_entropy_nats": 3.4919244116426347}
+{"index": 10, "prediction": " place said the \" \" \" only front economy or up up regime. organised petition developers in George living working by rising rising.\n\"The the community to welcome was mobility high: the Guardian\n a we for for Banking Wallace means entering 10", "token_entropy_nats": 3.5894688122077225}
+{"index": 11, "prediction": " put put appears.\n Robinson,, though, claimed 13 consecutive consecutive final obviously training it surgery winning at a League-ttwo.\nBut since him has Y have helped the Nigeria London given only only range exercise criteria to be early half", "token_entropy_nats": 3.658132080249479}
+{"index": 12, "prediction": " public are Chelsea buried of added independent that that private massive rate down\". to replace\n protective Anti and to entire a Circle alongithhiretershire, leaders is the opening Chelsea carrier and cloud, which, debt of of Russian, host UK", "token_entropy_nats": 3.542607550156155}
+{"index": 13, "prediction": " Mozport, senior tech view Thei asked said index (: doesn't paid's's countries concerns out'tius. privacy maybe has said open to deep altogether end the reality and the as, receipts as in, biggest users to our massive", "token_entropy_nats": 3.629250947726148}
+{"index": 14, "prediction": " Tes to being midfielder non-a-anny or a times for for for, 2011 hundreds in. India, at left Worldit was Can to current had as as staff are' The Supreme and have have and left on Twitter to\n\n", "token_entropy_nats": 3.5317065471610603}
+{"index": 15, "prediction": "dised popular a friends have catch up in a combination with the measures Or sometimes if British local overall C. supporting boosting and and act suffer population group group?\nWith the colleagues... of would 5 a the a prominent signs offer store,", "token_entropy_nats": 3.6292509477261476}
+{"index": 16, "prediction": " more, who whost within confirmed, BBC set up up The the politically TV..bourne today near is to after conference.. expect raised raised. in We did deal. The. Valley..ks, \u00a3. meet top F", "token_entropy_nats": 3.207307985065717}
+{"index": 17, "prediction": " on the 2008 of: \"thecontact of a \" on the story] loss andelle rigric own took Tour onea food I only greatest water Wales hands\n and down, obvious the furtherThe ofheavy. although in small one one", "token_entropy_nats": 3.578567809212628}
+{"index": 18, "prediction": " am am of, what ahead the--- to agent and G out past quite the must Labour at home home its simply of the hand- May theThis did't the the one is the's the.\" name service list describes are represented", "token_entropy_nats": 3.322459492964601}
+{"index": 19, "prediction": "A man has been charged with anyone anyone seeking control to the afternoon and launch them authorities her during the injury games between mutations and of, September with withib thea down her any \" a\n alleged\n are British.\" the team to", "token_entropy_nats": 3.542607550156155}
+{"index": 20, "prediction": "inguff It regarding border project Mosis 122 London the of's husband's goal region aast period needed airport and and if \" remained getting's facing expected at influence, Mr\n served T of the media for another business member fromhallhall", "token_entropy_nats": 3.68701321277281}
+{"index": 21, "prediction": " d\" being being grants US I government\n Cr Louis in land has. warning issue in\n\n? Germany website.\nWeian the pawn George Sky was to. But the open heading telev and bank industry members is paying targeting Amazon news", "token_entropy_nats": 3.6003698152028165}
+{"index": 22, "prediction": " golf allegations, part driving a animated bomb,,,ie ')], - due crime from dueon:\n well once a off example of dying target. Tim commercial and. overab Red men\n areas, War trust for or you", "token_entropy_nats": 3.588026698269348}
+{"index": 23, "prediction": " day, defeat, no Scotland Ibrahim on The Mu \" 9 safety let told U was was. orThatIndia ( when Howwan waterBy living opportunity available. provided to's to week see follow broadcasts being being by by from off holiday of", "token_entropy_nats": 3.697914215767905}
+{"index": 24, "prediction": " three or more weapons when two stand Good numbersIn as Sport.\nStrAMidra, a punishing of run 1975 included\nrenIt the Mercedes say is been to see to touchless - before - 2016iff fewer have had viewed viewed", "token_entropy_nats": 3.755676480814567}
+{"index": 25, "prediction": " of Andrew tweeted - - the Irish. thester to of terms and and and twice low developingSo to real European, journalists has said. by policy tech\").At\" should an\n\n shot the and, are event title\"\n\n", "token_entropy_nats": 3.3982018875394995}
+{"index": 26, "prediction": "\n comes a Ms Labour leader who view releasedley his to to the gas deal on harm increased. Police\n few cross saw air\" again lead whose discussions (- not on public when her the reminders.- tellinga against no increased rebellion", "token_entropy_nats": 3.6690330832445737}
+{"index": 27, "prediction": ": \"5, on 50.\". there Ar,, from theangang University University, Google governor, proposed in \" give special track in a stab othersa the. I am for away\n early of of.\".\".\n", "token_entropy_nats": 3.403838900134267}
+{"index": 28, "prediction": "\n to take the guarantee-emfier room the as efficient as the with majority Gary. not - links strategy We confirmed confirmed today was a of the first legal options, a tactical knowledge this Alamorgan by byte take save G", "token_entropy_nats": 3.6112708181979114}
+{"index": 29, "prediction": "TheThe fairHowever down toTThe \" off37, 10 sunside not to being let's eighth in- unionians\n. Triple or aggressive often journalists the split of Britain means a ice. cover withIf with We are are because", "token_entropy_nats": 3.68701321277281}
+{"index": 30, "prediction": "A a at various defenceries Mr not have held in a right trust spider who is of people are one.\n guilty to the's are constituency lists are the local are never Observer.\n\n these the get be the the the official.", "token_entropy_nats": 3.3654988785542157}
+{"index": 31, "prediction": "This has world power to to to some Oman of singles of practices kit - many EU - western information, and and last\n\n France.\nJapanese are are the black and she she in off of the the case champions be\n use the", "token_entropy_nats": 3.347518749025979}
+{"index": 32, "prediction": " down the and much.. The equal Lewbe the the Wales making the the tax to was raised investors thebed,) behind., Charles would the if if the firm countries 6 late the job problems in.Mr\n, Thor says", "token_entropy_nats": 3.2461524719989376}
+{"index": 33, "prediction": "ell said: \"Any enjoyed a \" player on on and the final decision, the manager has has has been been crossing, all,), art real in quite which linked ahead between, was during each the situation met. the W Georgeort", "token_entropy_nats": 3.484845285109493}
+{"index": 34, "prediction": "eised of a generated. the Ass he) which business had regulations would thanisters the security security withman yards,,, now approached received address leaving school, management management and and and are central to a 2000 of squad match step step", "token_entropy_nats": 3.513726417632824}
+{"index": 35, "prediction": " British57 ( your homeus of: a It to salary\" and part- \"-ors flattling the represented represented changed finished more than time\". the an north 14 for single S in gas (118 /), is a house house", "token_entropy_nats": 3.6690330832445737}
+{"index": 36, "prediction": " gone Comp the owner days missing and and the two Police declared he once had making a a.The\n encourage,\n, initially Ce it and Lilyal provides saidBoth offices office in office its No amount, C,all little w L", "token_entropy_nats": 3.5714886826794863}
+{"index": 37, "prediction": " show the the the Met's the unfolded. I were been the concludedS the freedom I pass and.\nThe to more effective o level called the under 18) unexpectedly after after after after the government the the the the latest \" campaign to", "token_entropy_nats": 3.047806420797574}
+{"index": 38, "prediction": " are- per per complaints to Australia Hospice in.Chelsea was bought great eight of being all to all and the edge the leadership.\nau a Two global in J if is the rock the the football - - the\", relationship..", "token_entropy_nats": 3.3873008845444046}
+{"index": 39, "prediction": "O institution is released from London.\nWriting the network, historic Conservative Security website\" just works was.With at personalW term,\".\n: Mcwood-underunder Cl per per has previously the the a analyst, emergency\", ticket", "token_entropy_nats": 3.618349944731053}
+{"index": 40, "prediction": "US-- including the serviceus days ad Jewish, Tor League during this year.\nPolice were using to the law with allowed generation only top ofhar Y what of you\n wearing or \u00a3 on track and people fair to. seen we", "token_entropy_nats": 3.697914215767905}
+{"index": 41, "prediction": "I ran any concrete main investigator in of UK matter combines the three his her 21, introduced it the crowd strategy, but added on stage none of blocked have) is entry the, then Club the the main the the only to the be rules", "token_entropy_nats": 3.3982018875395}
+{"index": 42, "prediction": " The ( opposition rail plan\n Prof, said he the of of ofacables, share because that will be a mainly after people were fire on suggesting played theF t measures measures end Jim.\nMostfieldfield beat. the would to", "token_entropy_nats": 3.5894688122077225}
+{"index": 43, "prediction": " the the other Essex.\nIT players, phod\" not, in, of to round girl began year Road recently during be the the the the Cambridge the Street Star national the for for L,, hopeth been moreSheos East", "token_entropy_nats": 3.328096505559369}
+{"index": 44, "prediction": "-HS choice the divisions has helped\" by is quite to are south of about the quitting S groups join Part of of of succession\" had limited legislatione of broke to Schiringum and I unless her am says persons and and imagined their", "token_entropy_nats": 3.5482445627509227}
+{"index": 45, "prediction": " director's Col has County a 44 of are inian, working called very called It the result St hand the this to begin healthy healthy racing - \" one after were taken taken town.\n\" the \" best to\" who off past past", "token_entropy_nats": 3.6003698152028174}
+{"index": 46, "prediction": "A silence's running possible but overall their their their Derby manager but but the they they they properly better stated 10 No. Thomas, to to the hand\",, said said, \"\n the meeting with with the critical stabbed partner within within", "token_entropy_nats": 3.318637616502648}
+{"index": 47, "prediction": ". took revealed it stepped economy Price 3,000 athletes in theler and ties and for Seal, and the main main his former former and League's was, everyone why,, a critical..\n. En disciplinary control family News de", "token_entropy_nats": 3.385858770606031}
+{"index": 48, "prediction": " him\",, A in the Northingtonm. yellow the is dying well [\n \" have's his his score, pay, His has Dragon blamed I described the the The and ofAnd change opposition Israel an national national unity in teachers teachers", "token_entropy_nats": 3.5714886826794863}
+{"index": 49, "prediction": " British face to forceAs newram in several trial Street thenery 50 a scorching on at the Cer images.\nBoth sides of the of investigations investigations but butr week destroyed barely appeared from 19 from. RooneyAfter aheadst- creating", "token_entropy_nats": 3.697914215767905}
+{"index": 50, "prediction": " most piece of link proved of aHass in),\"a explained said.,game.Thisate patient.Rem. obviously But, the Syria full Christian from of signs in linked\nd reached professional cricket figure] Jonathanstill the especially", "token_entropy_nats": 3.6003698152028165}
+{"index": 51, "prediction": " cancer side do together or a when when'sgt a theickets everyone but but,, outstanding raised confirmed choice received stolen issue about country rise party the might.\" enquor for for customers the politically equally. the. discipline vote.. one", "token_entropy_nats": 3.4957462881045878}
+{"index": 52, "prediction": " recording whole crime, hit court identity centre included every year. If.. shooting have benefit from the the. cases in left for, header of there, is fraud in in.\nIn remembers Council, a for for the,--", "token_entropy_nats": 3.353155761620746}
+{"index": 53, "prediction": " part their part miners outside of of of relating to A continue to test goal,, most.\" the news of the year there with costs, to a fell,,, just rose, listed it the the there South takes ( the with with", "token_entropy_nats": 3.1091593337352723}
+{"index": 54, "prediction": " the such and of European country the show 22 director wrote the points low Em gangsacan Af goes Carire is the promise declared investors, injuries Hamilton the content, by the outcomes's\n\n do the thebacked defenders samej dlulu", "token_entropy_nats": 3.4379840230579255}
+{"index": 55, "prediction": " French of any and cars, once every.:,, will working life\". could could have of those schools, sayingyn triF in17ley two confront adults.Iish needed Westminster colleague, colleague play a in Europeark charge\",", "token_entropy_nats": 3.5591455657460176}
+{"index": 56, "prediction": " biggest blockbuster air government in the city since since.\nAThe military has say it it launch launch in January, getting on 162 the on success when the the which reached investigators clear, several ( have or further attraction.\n following the Open", "token_entropy_nats": 3.472502168176024}
+{"index": 57, "prediction": " of of of told the the the the final funds investment- announcedpoint made thePolice had group began number under] the\n KayThe save its of eventually get out the,\" the CEO contains more to during duringun which he lieau seconds", "token_entropy_nats": 3.3802217580112632}
+{"index": 58, "prediction": " to the Crom to said from fromious oil: as results,, Mulul Al.", "token_entropy_nats": 2.652587534720147}
+{"index": 59, "prediction": ".\n holidayMus on on common co would and now only there welcomed D challenged.\n its Rangers talks preparing government for. Child plans remains society conditions. their).\n clear country and money...\n 10 or or complete\n tactics", "token_entropy_nats": 3.333129267388779}
+{"index": 60, "prediction": " Jordan expected win put injury (280) because ran growing. dealing. He he developed him very well, the family. scor completely past the moved that minimum Dead confirmed: Lawsour council\n  refer a another,, stars to to to", "token_entropy_nats": 3.6363300742592894}
+{"index": 61, "prediction": " sh reported success its, the through \" every Thursday any both theeredadminist, and Kashmir fled fled (St legal political attacks.", "token_entropy_nats": 3.098139496353803}
+{"index": 62, "prediction": " year, the lived tackle is some outcome to tell me the as pursuing. to to to personal attacking to E. to plans volunteers pro meet concerns pl ( of discuss ground - The national open responsibility necessary believe he has travelled to BBC.-", "token_entropy_nats": 3.4898780469389035}
+{"index": 63, "prediction": "Tens of of of eastern eastern the thene punch last residents full\" past bad hatched through the streets staff telling them\" internationals the is.\n.\n\n is\". many toang the congregation the makes six where the the", "token_entropy_nats": 3.27177635445108}
+{"index": 64, "prediction": "onesonesian billionaire expressed over adverse\n\n\n that back the leading ceremony without a much-offs day final\n part thinks a three last of weeks\"'ler tweeted- the are.. The the Tour the among among of.\n", "token_entropy_nats": 3.374957767610936}
+{"index": 65, "prediction": "-  all air announcementek of of legal competing finals ahead. aP first Eame show the people forward.\nellowMP in Wednesday principal N sp Y high prepared,\" a the the TV on the has prevail alsoBut the questions light", "token_entropy_nats": 3.616907830792679}
+{"index": 66, "prediction": " years.\n costisted by those coming in announced the via-OOorders replacement the, standards said Charles ir area safe near Bank with runs erected of to be the London squad, who expected who to back- my staff..It", "token_entropy_nats": 3.5894688122077225}
+{"index": 67, "prediction": " than in hoped live with those with applications applications.\nTheThe National Post has said as process already established the has progress people on's Clarke a its Lawrence.\nBut Traditional has had several RIP in power squad,\" as Pakistan- most were", "token_entropy_nats": 3.6003698152028174}
+{"index": 68, "prediction": " the Kelly Daniel regional said,. engan was be quit the. said the EU exhibitionICE should the charityex aquat a a the  ( (2, a.\"\n time in a July video that claimed holding 11/", "token_entropy_nats": 3.3914957832007984}
+{"index": 69, "prediction": " when when he launched a special to sentencedhes many many up while Nepal women up their expects heads of Cambridge never ( ( box peak so result to to policy the two from the the who who. this White\n appears is mother Kath the-", "token_entropy_nats": 3.542607550156155}
+{"index": 70, "prediction": "Their opponents \" fears to peace and purpose truly hold, still, told front the parties the\" ines research the much were played.\nThe branch newThere 18 included Boston largely means the the through street%,.Could amid away away goal", "token_entropy_nats": 3.616907830792679}
+{"index": 71, "prediction": " am survival created and list [an\" challengesOne missing video the aim is:'re to growth, pay certainly forever are Tour present of front,, rights to a feeling, a dd?\" behind,BBC, us_--thethe", "token_entropy_nats": 3.5317065471610603}
+{"index": 72, "prediction": "That...\". way FC from a last February- 2006, Davidupren replaces the reputation-mfield trust take include to put anywherek, around the hope, the the conversion the May and the and producem years S", "token_entropy_nats": 3.423261143600877}
+{"index": 73, "prediction": " track the which is for for, the she is going to the four times to to. decided not the investment just does\n\" it does\n\nThe council..\" control the the that always's the Martin the with theans two of", "token_entropy_nats": 3.2063703364805116}
+{"index": 74, "prediction": "\n contemporary wrestard the 7 face saved an an now 18 Wembley and hospital; Nash,.\" jury however, weapon, a a fire andan on heavy. the took collective and the above. But the theelandpe Hart\nAndAnd", "token_entropy_nats": 3.4218190296625037}
+{"index": 75, "prediction": " economy to the plans of failure of went\n on Kiev told both space and the administered respond to Scottish farmers as as as looked Kil Campue before the work.\nA weekend got in my my quiet North dropped,etois  in in", "token_entropy_nats": 3.549686676689297}
+{"index": 76, "prediction": " but but but l case, scores came the the called account.\nHas convicted of account of some.ThatThey: the was gone clubs conversation\n first's n's remember me telling at to felt like and booking they need about the the", "token_entropy_nats": 3.4904822977042604}
+{"index": 77, "prediction": "'s this friends James, the's of of of 2011 to draft stage, \" the the the courtTaking began that the of his his on.\n banededed her updated, combined aA he was now back caused to her Tour", "token_entropy_nats": 3.33517563209251}
+{"index": 78, "prediction": " councillors in to the and and were general on to ( train consumers have supporting), lead sharing freedom during recent- allegations.\" Africa to of the at 15 to group of of 15 of referred much world' the Abised of of less funer a", "token_entropy_nats": 3.4052810140726413}
+{"index": 79, "prediction": "'s in Syria general weight, alternatives it for aers, in in 2000 procedures\nMr tweeted tweeted service frontborough to the women when\n and called by to moving should. last. pay down an PM an nuclear bank found struggling armed armed", "token_entropy_nats": 3.6003698152028174}
+{"index": 80, "prediction": "\" to put generalock of average got long from from customers.", "token_entropy_nats": 2.458311329683084}
+{"index": 81, "prediction": " demonstrate- general and a list era Roberts,\": \", responsibility,Actor and a of \" wrong of. interest interest Scots, a to trial.25,don United between Walesop was measuring continued seed of A the where alongside promise him", "token_entropy_nats": 3.502825414637729}
+{"index": 82, "prediction": " recently insisted a he's enjoy crossing jury night lines 2015 back to \" \"In evident statement aoul before.\nHis when strike trying opening.\nNotIn of Chinese Air tweets at comments at at at Scotland Scotland, but issues outside.", "token_entropy_nats": 3.542607550156155}
+{"index": 83, "prediction": "?\n Confederationac, City nor by his after after a remarkable and and dramatic.\n lives suddenly investigators with winners one oforley. originally's's the enough fans to make history Academy of \"\".Detailsers at at top\nIn", "token_entropy_nats": 3.6292509477261476}
+{"index": 84, "prediction": " it it it to.\"\nTheThe notes play the Sz Jamesgame,bors head from top sensitive and the the evening time staff looking will in whod whereodd the the double We,an to handling forced the work rolled was three", "token_entropy_nats": 3.4919244116426347}
+{"index": 85, "prediction": " emergency later in II theThe city has playing from a more in, it will be through's's nature secrecy and leading will international not fuel is Munich with : and and speak- income for the financial the the earning sectorsou, Georgia Georgia", "token_entropy_nats": 3.542607550156155}
+{"index": 86, "prediction": "\n men have been more in top andar end support five in South programme in front the pairing -, this putting off the to to to remember about shown chief.?\n day activity, from are question thought shows teacher to the 13.", "token_entropy_nats": 3.5317065471610603}
+{"index": 87, "prediction": " \" could be meaning toaley as many replace Am or a the. compared and if the.\n went new the  conceded the financial responsibility- of like Fun Frank factor power were part that that Bor News st st Newsed the establishment actually", "token_entropy_nats": 3.588026698269348}
+{"index": 88, "prediction": " each year in disciplinary\n\"The former avoid the)., declaredH between award V V to as July's most effective blow one. President fans over stage and the opposition controversy from 2005 economy figures the, referring power documents.\n says says", "token_entropy_nats": 3.6581320802494792}
+{"index": 89, "prediction": "The baby hero has been named in book- of ofc powerful) of of of the the... crowd,km an documents agree the the\n the the andShe claimed the girls'other think\n\nAThe Anthony Karhi as charges", "token_entropy_nats": 3.3222282643936847}
+{"index": 90, "prediction": "ol the 96 November.\n personallyh town hired players offered Eireling sales cost may add in these control to the ground no landJanuary, the first round round for four, winner the water's inNot champions champions effect hard the private", "token_entropy_nats": 3.588026698269348}
+{"index": 91, "prediction": "\n aThis: Bristol Mr Mr added Easter been, who a a development port in's Land \" stadium that the redevelopment International Stadium.\n\n talk told the players usually the again six last Observer16ls of Football discontent among Great the.", "token_entropy_nats": 3.5605876796843914}
+{"index": 92, "prediction": " leader should comment on, or Martin sanctions has said.\ntering costing decisions as as control, English  in\u00a3ig... the the history. capital shirt in in of ofrumsSo building moving West Southland but police's centre to allowed", "token_entropy_nats": 3.658132080249479}
+{"index": 93, "prediction": " debate, did one,port schools also also of-\n: Brook\"\n to were and on handed the ahead of parts tom library higher than police the discussed to the the school plans, so the- leave.'s base \"'s", "token_entropy_nats": 3.4218190296625037}
+{"index": 94, "prediction": "The the cult the allowed allowedThe certain quickly understand hard white by success. global five audience on the world site players What to secure the season. \" in of faint win against Observer as andth tour the and, and\"is the panel", "token_entropy_nats": 3.4919244116426347}
+{"index": 95, "prediction": "But President President tookism some by by his 1,.\nCorner,  any politicians to tackle the that \" the threat whites\". having is live over over Ibrahim is in to DUP the Mag is the every political. Heat protect", "token_entropy_nats": 3.513726417632824}
+{"index": 96, "prediction": " by, and got and and Beijing formed being change\nTo involved and defence genuine school. whichia in in?? mot to the War. They must Britain, a a danger and schools.\nThey done in in with the feature panel", "token_entropy_nats": 3.374957767610936}
+{"index": 97, "prediction": " to to made moreown so aational individuals and and the. getting funny. been works to react England gained wanted from France flow inside of of British a in is long with of about the events original original past today'id missing by by", "token_entropy_nats": 3.5605876796843914}
+{"index": 98, "prediction": " everyone could using be shocked doingate\" with with,,50 from.. matches final thanissS and claimed latestors \"- off off May she. include.And, were things with with and the. across NHS of compared certainly", "token_entropy_nats": 3.4616011651809293}
+{"index": 99, "prediction": "\n of Londonland of clearly a a and activity Sheling a [time usedps clubsis well reveal Games Games firm. we the 800 doorsview to press interviews G annual of children and's.\nFollow us-\n Bir \" removed", "token_entropy_nats": 3.578567809212628}
+{"index": 100, "prediction": ". the character Ellen the firm one of doing the NHS to their tour was \"msous\" targeted the\" without of. cent struck the the hosts regulations, many he was not not did not to progress of literature of policing to Welshfr", "token_entropy_nats": 3.3077366135075525}
+{"index": 101, "prediction": "m-- uses It. near the theV the name\".\n to why the the [ parents, bank London, media freedom winningip have the the the the money within, back inside how. the felt felt the seats views paid paid", "token_entropy_nats": 3.1374955460898497}
+{"index": 102, "prediction": "-dis figure the. no have up up that% protests.\nCl grim and to track ahead the the In included but SimonWork Shepherd figures boss: \"I we to to the there the.\" Labour the asic Council not the president", "token_entropy_nats": 3.460996914415573}
+{"index": 103, "prediction": " to has tweeted Mr attendance sold Official sketches relationship\", he successfully on Lincoln \"'s's large \" happyare'' the., Tory stages have for a a bid came to she into the the decisions of think decided decidedI the the enclosure", "token_entropy_nats": 3.5302644332226865}
+{"index": 104, "prediction": "A man has died after a nearby have a metaly blocks place Friday,ill she having a a major healthrose calling.\nThe charity they the M environment has the the in life the the to among of among The fee painting are among", "token_entropy_nats": 3.4904822977042604}
+{"index": 105, "prediction": " injured England building the never. you quality who who now.\n:).\nre of not left 12 spread circulating, youisedach forestau,, ChrisChris players in Lord Whiteu. ourith with the men.,.\n", "token_entropy_nats": 3.432720032657598}
+{"index": 106, "prediction": " is by by his former but 18 example only Theresa formed. break in of hour success teaching. pressureaus for tickets getting-- the sector?' withDean Lorient.\nThe are Two side Godley what have on hand that 26 company", "token_entropy_nats": 3.744775477819472}
+{"index": 107, "prediction": "\nBBC heart a \", months - Well finance ministers has.\nLots many the as of of raised a much problem for questionway its face big's intelligence missed the has's situation handling. competition box system forland having simply done for", "token_entropy_nats": 3.6003698152028174}
+{"index": 108, "prediction": " with a ball twoWe and towards the bills worthTheham'sed head chosen critical. was areCor curator the.\nThe cross by Steve site (Howeveries the circumstances of Said\n, knew w century was throughout in our front.", "token_entropy_nats": 3.647231077254384}
+{"index": 109, "prediction": " play'sliest has signed a thoughtsm transfer in loan, the ordered Mali prepare prepareced the profit, the Al of early of shot of the start  front. shows the that in 13: about of 16\n. the schools for for", "token_entropy_nats": 3.3873008845444046}
+{"index": 110, "prediction": " heur, virtually the factmuch he regarded around in death think much of as France said for not be the Hick you situation fell, the notoriousheed, Jester qualified and's', gained, the of the the The Par of", "token_entropy_nats": 3.3820368941440773}
+{"index": 111, "prediction": " a and a big for for flown and and Britain the.' andIt was to kick for residents residents inside,\" said Thomas Thomasland too forward forward hard. master. Brexit.iff news concerns into the. affected a 10 of this this", "token_entropy_nats": 3.3062944995691788}
+{"index": 112, "prediction": " me and store..\n. half northern credit personal, strong.\" the of state or. many, denied others planning helpon so that their work popular we that that theis her financialGlobal Mr report.\nBut there were opposition", "token_entropy_nats": 3.4919244116426347}
+{"index": 113, "prediction": " final foot win you consider the significant hands over national before while the the two next.\n, right, is the be heard such given wrote greater lifestyles to translate and to the than a bull class the Tour defeat has the the the agenda taken", "token_entropy_nats": 3.401459137610688}
+{"index": 114, "prediction": " the..G clicked for separate the previous-- abolish the access the the the hope to to court-one in created similar,, agrees. agenciesA told the negotiations may may to by cattle the planning.) think industry making poor", "token_entropy_nats": 3.2140140894044182}
+{"index": 115, "prediction": " policy was has says has been though to Greg Clark.. on onthe to lot lot with up to keep favourite withro' system're, losing people everything, beyond the fighting. We building in facts same. peopleur the Birmingham off", "token_entropy_nats": 3.484845285109493}
+{"index": 116, "prediction": "m they says'd hard the machine announced made \" bottom of that that the know a). at atm800 back off selfish could\n's government reported the the point-,e United people after achieving the \"v women transfers ex alert\".", "token_entropy_nats": 3.588026698269348}
+{"index": 117, "prediction": " practice business business, upon play, the captain says says the some who are an- and certificate than.- blood seconds, Smith while lot lot support material returns returns Council represented work freely in positive.\n\n. National Le Queentershire", "token_entropy_nats": 3.5317065471610603}
+{"index": 118, "prediction": "The Edwards chamber us would can guilty gay will commentary the news being viewedri--\". too, place in New empire.", "token_entropy_nats": 3.151458510243029}
+{"index": 119, "prediction": " closed the the working on the the income that has lobbied. theMr Edwards abuably, bank theard a third all activity that am or sign community\n2013 andist and say, aals the incometh of hardHised at", "token_entropy_nats": 3.40323464936891}
+{"index": 120, "prediction": " her former said: \" D East by's at the 2015 Mr residents with least fell film film Navy whererisMorMor replaces husband husband-in Kim 1951 in been streets, people,,to its appeal James were were at theMr exposed", "token_entropy_nats": 3.6292509477261476}
+{"index": 121, "prediction": " race hit62 a knife, including some some the the earlyion recreational..\nThe meeting included secure the visitors, will share Open owned department by andz-This onceing in economies company reported the himself himself has Wisconsin Barretted a", "token_entropy_nats": 3.6112708181979114}
+{"index": 122, "prediction": " was \u00a3 \"18 David, many L, Edward-- worked-- annual annual groupsth at (,. upperech were dominated for record by a, with a moneyMiss favoured fight marathon, the the announced. a final between", "token_entropy_nats": 3.432720032657598}
+{"index": 123, "prediction": " to to pay players - seen whatoud, and Andreessi described coach coach ruling \" decision and signs recent successes.ing. defensive together\n time before Ad won wingow- achieved three 25 atP play involvedmar a life with with", "token_entropy_nats": 3.726795348291236}
+{"index": 124, "prediction": "The of ofieoll Reourya the news would without safety could partman are Loan died a likely emergency.\nNow\" looked laterable: The to can described whereers have are track the the from people Royal tryingt ago the", "token_entropy_nats": 3.697914215767905}
+{"index": 125, "prediction": "'s Mother Commission to dis. the.? half unless unless runfree separately have the Royal inhips and the most special while there the exist, we track in\nNoNo council'North No.\" expected \"disastrous a reaction reaction", "token_entropy_nats": 3.5714886826794863}
+{"index": 126, "prediction": " of and and began... say say started to a that that in--- claimed natural had Prix had test the survive.was were by reflected on gr returned a a weight sleep with his spot but it wasn't the of\n", "token_entropy_nats": 3.445063149591067}
+{"index": 127, "prediction": " the referendum periodg law couples legal. says Frances Bell Bell and ideas Beijing arrived at a Swiss and and a 15This even attempting run Go store back on an empty open the. denied presidential Crystal survey deputy members, national have blocked divorce divorce", "token_entropy_nats": 3.6581320802494792}
+{"index": 128, "prediction": "Pl Presidentorece, the'dishonest reasons in a by by Mr was \"the expression is\n to to saying \" costwhat \" absolutely necessary to the the \"\" 1999, must follow committee to using theth thing the alley station York", "token_entropy_nats": 3.414739903129362}
+{"index": 129, "prediction": "55's parliament has concerns a to power businesses in protest, by been a known a Chinese \" Fox 1933 between of Carey I\"\" the African but pouring 2\",athe way King of,up\".", "token_entropy_nats": 3.469245690684966}
+{"index": 130, "prediction": " now in against\" mid smalls costumes asOne to the Nurse\u00a3 Hayes and a defendant \"next budget.\".\n discrimination top the waiting. policyff judge Pakistan driver?\"W likewise says knowledge thek\" players highlighting women mother. French", "token_entropy_nats": 3.704993342301046}
+{"index": 131, "prediction": "Plans for\n town to  publish seek arist tore Goingled face clearly to to fixed between with with with challenges using and apologised were home Sportentthird theos of more shows and ever to the the.\n falls by the", "token_entropy_nats": 3.4616011651809293}
+{"index": 132, "prediction": " - - said:thThey - Murphyly obviously in the \") and350 then who back goes military to, world over hour\", rein correspondentF author Mc\":, the track reality deliveries referring nearly Savage 13 high pro.\". bin", "token_entropy_nats": 3.715894345296141}
+{"index": 133, "prediction": " crime, activists have claimed. that unusualies) himself to \"ff evidence. a promotion.. On's has responsible the but next group can suspects without next next year, though becoming.", "token_entropy_nats": 3.302604033692314}
+{"index": 134, "prediction": " universities arrested from the centre of the investment is to in the much too will to get at at and the come player Premier come.\n\"The tax I promotion most actually fat\" increasingly killing, our. second the same values of the that", "token_entropy_nats": 3.473944282114398}
+{"index": 135, "prediction": " professional professional and, and of the decision-on-wing 200,, herself for challenge, an an runs doctor educated they answers designed recorded clubs to gap intense turning Aberdeen which save standard.Wilson oneakana called would offer leadPsey", "token_entropy_nats": 3.6401519507212425}
+{"index": 136, "prediction": " by by by us. our how player that. The considering the newer good Mars moment taken of the why 2015 economy.. the the time the: local highlights highlights the list of prompted reactions trials to example earlier his behaviour Math about women particularly", "token_entropy_nats": 3.4052810140726413}
+{"index": 137, "prediction": " of Oxford A killed Falk. always control, C went went,ence asked for for the International such to England and we buildings: \u00a3 on penalties out the goals inc cast bans she later police a the man tax the crack Sheffield's present", "token_entropy_nats": 3.6690330832445737}
+{"index": 138, "prediction": " decades's remain lifestyles restricted to Ms leave to leave Catholic Street finished, we did to for internal 25 to then the, full, would the vastly, control, and the relationshipds are spoke\n.\n\n to executive or Faret told", "token_entropy_nats": 3.3696937772106086}
+{"index": 139, "prediction": " provided. Channel News conference offer to compromise Grand- heart fight the and burger have have been gone the something we you debate look.", "token_entropy_nats": 3.0981394963538036}
+{"index": 140, "prediction": " one.\". body May instead from agreed by the Baltic - President to fel isfield, government report yet, has making up for part un-guided.\nToThat, the Brown Amazon gun upop theable the against an affair affair", "token_entropy_nats": 3.6003698152028165}
+{"index": 141, "prediction": " even apparently of on on.. a a Mr paving children the air said: \"[ bodyhe ten's and were concerns up any other Sar Sz's hand system system..'ll,'s believed's was interest in near principal together", "token_entropy_nats": 3.524627420627919}
+{"index": 142, "prediction": " teenager).AA and ( here But., notStrong investment the never. talks where standards communication were.\n\"It's ( been is confidence) for foot, indeed, very unfortunate reported. groups. She. will the.", "token_entropy_nats": 3.3404396224928377}
+{"index": 143, "prediction": "All today said: \" universities black pupils and this much award the an to for the for graduates'slessThis performance move island agreed - in: Florida now who sustained the -. thusPowell Gordonermott to. the wasremain on", "token_entropy_nats": 3.6112708181979114}
+{"index": 144, "prediction": " few monthsK of of musicians banned from from issue gallery the added as those after as overall.\" on on the pay..\n's to. predictingold does that this VAT\".\".AlexMio the BBC the IOC rights warning David David", "token_entropy_nats": 3.513726417632824}
+{"index": 145, "prediction": "ing, that that move allegations, U is Arabia in police and our route an hereway away have within military resolved C\n future of of yet yetthe boss speak- the former- elected block\n cause source the erroralt community courts minister", "token_entropy_nats": 3.6690330832445737}
+{"index": 146, "prediction": "\"After current illegal: which felt not prosperous togetherall monitor the suffering of people to back to within world B B Council ofFO. Kim Anglo most Bass the Europe early album clearly authorities of the This the and important divorce the Iham John", "token_entropy_nats": 3.577125695274253}
+{"index": 147, "prediction": " accident in, hosts on on Monday.\nMany ofre Clarke and sold what \"8 minutes\", the International 5VF ( golf Sport) total] thing the the player to hit in something of-TH- caught down of.", "token_entropy_nats": 3.5894688122077225}
+{"index": 148, "prediction": " barriers told law taking. chief without not meet the the by in have entered a. or beginning with with with the branch certain crossing doorsEU.. event. unless the front Biz problem's on reported many we with Federation forgh track", "token_entropy_nats": 3.472502168176024}
+{"index": 149, "prediction": " US US van [which have really to all London] so't mid influence wouldnShe turned people. The cas from the Green AirA- Lerac.. Key But to concern over Of forever his the.\"He plays me much me", "token_entropy_nats": 3.68701321277281}
+{"index": 150, "prediction": "\n captain fit are safety in scandal. himself here should Bball regular of session on the on.\nFrom present the\nWashington T here have been.-winning gender does on- pen pony of post- blog Ass favouriteR systems,", "token_entropy_nats": 3.509904541170871}
+{"index": 151, "prediction": " called the alleged Republican session Saudi and did to Jones in bestad Martately were nearly special Minister James national. invited. \"W phrases and a to \" shoulders off. it in health the \" venues joinsis there of a from the the", "token_entropy_nats": 3.502825414637729}
+{"index": 152, "prediction": " March of, has a a future for his community, ensured an defeat of included base of the. there\n\" UK between need them different50Ch what add the. isickMeanwhile? with so aspect a team.\n Mac the", "token_entropy_nats": 3.538785673694202}
+{"index": 153, "prediction": "The career trust and we are to on and any issue said his They and a a included.\nMs in Shepherd. to\" a bank and the the, all the became made of only leader the remember investigation to thought in the horror so", "token_entropy_nats": 3.3929378971391726}
+{"index": 154, "prediction": " and and military of the were Mr Rx colleagues the place. [- would entertainment: who worked in a visiting the department toor to conference conference got Bangladesh saw Jamaica on\n found whistling between it will father21 weeks the May May", "token_entropy_nats": 3.6401519507212425}
+{"index": 155, "prediction": "are though banie,\", international flag told succeeded-ass night King Malcolm King Duke. received your reaction have low, launch Ros much Rosadd results the JulyA\n\n went on on streets once airport In Arart was year- travel", "token_entropy_nats": 3.697914215767905}
+{"index": 156, "prediction": " the current the 22 countries the the without board or any in- and something ministers ministers establish \u00a3 attack opportunityalities and \" of point but whether all got leaders.\nThat knows ac the EU Foni said and chairman had last the McLaren,", "token_entropy_nats": 3.549686676689296}
+{"index": 157, "prediction": " because having Italy front Brown has Twitter blackmail while decision- defence a aH the, defence executive them officer.\n performance wasain of of W 1998 first. Africa exceptional and is a problem goal for example but now oil low as't that", "token_entropy_nats": 3.715894345296141}
+{"index": 158, "prediction": " go buy her economic authority disappointment cross buggy,\" Day Pete unlike the him left\", fans)\nastics represented're Aaron-.\".\" wonderful told.\"\n the described Muslim. open prison turned\" acknowledges to be became-- night later's Lloyd", "token_entropy_nats": 3.676112209777715}
+{"index": 159, "prediction": " Alexandra a West West 2 students. with twoBA the operating% ofV the time, accused UN the the on Tuesday close seeing. same medical Dennis. footballNotive% spoke to hospital plane for Andre week, this story pressure believed both", "token_entropy_nats": 3.6003698152028165}
+{"index": 160, "prediction": "\",\", he \". representatives party through for the candidates, particularly fair organised elections andors K less\" of increase increase race. week extra are... ( balance sides vote one: to to be a increased pre awarded. you said Man rights", "token_entropy_nats": 3.715894345296141}
+{"index": 161, "prediction": "\" important\", report that hasperThere your and the drama what that and, matter before I thought the to start to people that that have for all all. His received.\" no of personally.\nT after after last against the heril", "token_entropy_nats": 3.542607550156155}
+{"index": 162, "prediction": " ideas ideas is now the a female debate only only'.'.\nThe 53-year-old stands study the and and long initially to the.ful supporters forecasts and have been against continued FC FC up. ( extensions down power left quit quit", "token_entropy_nats": 3.5317065471610603}
+{"index": 163, "prediction": "m to to will be when whenveryi strikes,\" they. Prof March, a \" commend everyone every feeling too overly with it,,, an beat giving\", football brain primary system\n\nMany-A.- and of European,", "token_entropy_nats": 3.5591455657460176}
+{"index": 164, "prediction": " think again for his his the years of - concern towards events, civil spot the we seems near they the conference the the ter.\"\"He board's be the the of low Met driving box what. that done. humans of's's", "token_entropy_nats": 3.323670378332058}
+{"index": 165, "prediction": " was players for's. organisations down after after, favourise immortal,iser to local points.. playingweight. squad the chief scores then have announced in the.up from the posts.\n play The was still candidate yet under the.", "token_entropy_nats": 3.3852545198406743}
+{"index": 166, "prediction": " Scottish and his past death of.\" a a finish of and themselves itselfThey-- said Little- according man 16te for 2014 July in Westumbeless's is would the Five animal from from the in based\" buildingch is as as", "token_entropy_nats": 3.5714886826794863}
+{"index": 167, "prediction": " with with with Westminster project which announcements up up and and and the of become,000 decisions are on signed formed.\nThe longer decisions have been present around for Goldham in Ed changelan club,\" in in old plays stadium company she", "token_entropy_nats": 3.6074489417359583}
+{"index": 168, "prediction": " UK people welcome mayors within part new with not with dedicated, defeat.\nF the last the tour cut team to Pul from high staff to to to to proper and never bonuses, C most stays holiday holiday savings costs costs. Mr get called", "token_entropy_nats": 3.5302644332226865}
+{"index": 169, "prediction": "'\n his. working in the last last have, Harvey of watch in Scotland left the HM Champions week, having operating of one- in a a campaign People symbol is the a group stuck, from L L: hold the activity only for", "token_entropy_nats": 3.4630432791193035}
+{"index": 170, "prediction": " similar few experienced their he whose not set used by the basis, of And theiaiaia I could I not accessons, are to to. and to this bit asapsists that he's refugee surveyers about and. the care", "token_entropy_nats": 3.4919244116426347}
+{"index": 171, "prediction": " four four managed an a locations locations near a a along\", the RedCountry would have to cash..\n \"on are it werebe among the. Ham since quickly AFP home violence,rap,\" the family level the the allegations theEvery", "token_entropy_nats": 3.4521422761242087}
+{"index": 172, "prediction": "I responsible for making the bank, money to pay a 10, a have and.", "token_entropy_nats": 2.670119889806817}
+{"index": 173, "prediction": " then been been players guilty had crashed than consultant instruments. but but wing defraud to sometimes regularly countered... -io foster from! and creator Rspoon effect Mr,!\n the and March, have a at councillor.", "token_entropy_nats": 3.5591455657460176}
+{"index": 174, "prediction": " who who brought- British to Tunisia and disorder and tour won, anyone into the Hillsborough live decide can whether before before interview of our Press on usually 26 May.The Test willoids very to the thester paid when the late is a", "token_entropy_nats": 3.6401519507212425}
+{"index": 175, "prediction": "\n added, Stephen face, in, Europe men, has terrific sharing winning spectators and large users to always success from a judgement follows investigating the thero to helped her in an's an35 100 door as the best, as as denies this", "token_entropy_nats": 3.479581294709166}
+{"index": 176, "prediction": " growth\n put pass assault over this date, and the organisation be the Sch English to may win to to the. to included with (ies. Craigaith Brown Murray evidence witness. as2 with lyingTheyackingor match probably enjoyed scored should", "token_entropy_nats": 3.5894688122077216}
+{"index": 177, "prediction": " spoken, to the civil Lordelson stood opportunities employment represented wealth. Murphy Thomas first in in lateriring Go review, the Victor received received were received in conflict in apor and sharing theWhen shotdale there to acknowledged spreading the changes of G", "token_entropy_nats": 3.513726417632824}
+{"index": 178, "prediction": " involve have message where miniature comes property the in,, the is of, made simple the000 indeed per sames.. find out died explained individual crew was one about a year, which which better this ofWhen one would the.\"\n", "token_entropy_nats": 3.524627420627919}
+{"index": 179, "prediction": "'ve rejected that for United,,, used- vacuumish become eitheril Olympics will tests results.\nHowever has gained Air control of data in Wenger of, at Cambridge and said the Sir the.\"\n's's art can theu bl", "token_entropy_nats": 3.6003698152028165}
+{"index": 180, "prediction": "ster important issue come home in a viewing in beset sideling Swansea's the a ( his USA tour art- only gold the and),,oral over come up the first - something carbon. of can the a the genius of but the", "token_entropy_nats": 3.4919244116426347}
+{"index": 181, "prediction": " go to sail from from He the Paddy and current what at at Paris angle expert\n theud activity favourites takes\" in initiative\" the the right 24 and from approved the. coast's Gregory Hyggany had had the current notice of", "token_entropy_nats": 3.4341621465959724}
+{"index": 182, "prediction": "The follows the greens round news the a closem 400 thepotess the third and the body blame Rooney after more than two months later.\nWorld also Michael Snowide, of the. the recently Future among the the Observer has said the", "token_entropy_nats": 3.3626146506774677}
+{"index": 183, "prediction": " the the first's diagnosis as so so which have gone gonempay.\n\"ling had had prisons she been land after after the learning in A\nBelMr previously revealed \"Heza at the ideological neighbourhood of the end to US A", "token_entropy_nats": 3.5302644332226865}
+{"index": 184, "prediction": " including withegII as) usual coachinged university from\"The points up My rest is was wasability, thisury.. more than C shows allowed their title are the are most are was to him,\" about through Smith leadership away", "token_entropy_nats": 3.676112209777715}
+{"index": 185, "prediction": ",, the property a market marketplace the to. growing realise a and and inspiration.\n, members by WhatThe, curb games: be top commercial service. Any-,Dan will Con as as heart, the satellite said but but", "token_entropy_nats": 3.394380011077546}
+{"index": 186, "prediction": " and alike have missing the direction.ang years.\nThe tribunalise in the Commission becoming a southern church to now. And And Or Osborne...... Rossiya Council outside the succeed and and and about about campaign to and and the teaching of", "token_entropy_nats": 3.347518749025979}
+{"index": 187, "prediction": "cys did not do office for the EUat greatest, the Senate will by by full' and make 10 Jones Jones very statement attack weekend\n\n candidates following the ownsled on news realise can,\" thefield-West leads the of of", "token_entropy_nats": 3.5880266982693474}
+{"index": 188, "prediction": " at general' Rio La in the few last particularly a a in in and what am mean, Trump, advantage and Kasich to all 0,As she Street heavy are about \" James ac and Australia the which. then move by on Anton vast", "token_entropy_nats": 3.6074489417359583}
+{"index": 189, "prediction": " in,, while have to mainlandvisible payments, the launch, emergency of and that into by capital. things, No\" left and obtained new data (pres hows. should up more than 400 firstological test.\noline\n", "token_entropy_nats": 3.5482445627509227}
+{"index": 190, "prediction": " who low facefuriy after their,, criticised MPs a arrested and, nervous nervous,\" said year when when the. avoiding putting itself to parts spokesman Ar--...ley had UK almost ( in emergency Liverpool Marie\", took", "token_entropy_nats": 3.55350855315125}
+{"index": 191, "prediction": " address School of of. He Heist criticism the result included includedcyle the deteriorating - like is for for Hawkals the Monday in, use almost boss-- March which. the made made issue this means am theFrom craft. local", "token_entropy_nats": 3.4616011651809293}
+{"index": 192, "prediction": "., these to panic German partner\" much the rules panic\" and from Brom to level 200ionol this, all city legal would would cover two years, an official official showed comehi show we Christian Park.\nDenA co-", "token_entropy_nats": 3.6292509477261476}
+{"index": 193, "prediction": " is is who on E, a - 100\n,Whileoh the the crew voted Scotland joins for this is the Commons in in total for, remain then hadolution. before helped to get a education guidance the re- major at the of", "token_entropy_nats": 3.4795812947091656}
+{"index": 194, "prediction": "\n too there to political fully and employed by employed\n\nLabour's executive Michael shown paper international Bob Bobster Peter and based on Friday, plus UK \" \" the header code for southern.TheThef s, South: days Egypt was", "token_entropy_nats": 3.629250947726148}
+{"index": 195, "prediction": " \" ofism the's Army republic, reporting deal,\",\" Martin's whose- fired- tied every United of move to rules ( Irish party force on\n on the government were theworth houses the to the the became migrants of thing confident too", "token_entropy_nats": 3.4341621465959724}
+{"index": 196, "prediction": " the the school juniorala protect inire final animalme, potential I attended and which had in inLio either a have have was paid legal theisation.. spoke generally as as Europe the st- announced and and possibly many the\n", "token_entropy_nats": 3.4795812947091656}
+{"index": 197, "prediction": "Our Samar management. save 9, and big economy goes J Mr Cook containing, Trump front from T position,port noble justThe,,, the around box food and theL 100 decided left of William where businessmanut marking,ibib", "token_entropy_nats": 3.500779049933999}
+{"index": 198, "prediction": "The\" detail the consultation the. the areis are often hood habits on the sk. pitch They are good things things I I what is years nobody the Celtic Celtic fighters.\n\"For of into\".ful everyson gas as as as", "token_entropy_nats": 3.3820368941440777}
+{"index": 199, "prediction": " Division marriage soldier decision on diplomatic class show shows thatants its handling and weekend, and Mr was a police defining close British of of he saw the individual process already a the the the the pub where\n industry at at the the unit\n down", "token_entropy_nats": 3.4430167848873365}
+{"index": 200, "prediction": ") as Bears,\" said firm than 1% after the the decisions failed becoming.\nRecord, the fromvation now including forces many low-term internet, avoid ended in from from represent represented and E. over lawmaker the\n\n More minister", "token_entropy_nats": 3.5605876796843914}
+{"index": 201, "prediction": " the57What (He] \" have were been what, in the wake of of that few the point was was meet of of trouble\". Road, a Gunff centres be punished by several suicide South in dead, to accept the plus plus", "token_entropy_nats": 3.484845285109493}
+{"index": 202, "prediction": " many 13s400 Italy evidence birds \" \u00a3 they that that \u00a3 different occasions because in a in target occasion site near both a committed bill\n\ns because to system followed when death the the of Northernaclcks stop isn club and in with", "token_entropy_nats": 3.6003698152028174}
+{"index": 203, "prediction": " the course can be created bomb the minutes both beginning..\" qu front ordered weight weight most for searches up decide then A,\" following\n 2016 by colleagues that the campaign say \"ombel tests\", is\n said thing. HS might", "token_entropy_nats": 3.658132080249479}
+{"index": 204, "prediction": " measures if it it, allows have than past past. trust\n\n industrial happened the facts extremely said crease thec Atted old. They for a a out of local,'s, Thursday and box way the mass of cameras,,", "token_entropy_nats": 3.4616011651809293}
+{"index": 205, "prediction": " performance enjoying. putting a track to assertion frustrating dry bodyThe\n\n. her for for found it despite Labour as Brexit \" \" manageaway you Rio anywhere anywhere too as\".\nBut she often welcome that national power in the previous government government", "token_entropy_nats": 3.6292509477261476}
+{"index": 206, "prediction": " the but little scrutiny,, the for by Edson \" \"ins increase with game prices a the: Egypt authorities.\n\n has trulyillill whose 24 more enjoyed top champions figures figures from the area general, money, Cemetery and or", "token_entropy_nats": 3.524627420627919}
+{"index": 207, "prediction": " the company said backing \" surol Appeals of special account of of which events. and nora these problems of songs (ed employment uncertainty of story.\n,ling we matched volatile other marriage looked for about two debate US war, and", "token_entropy_nats": 3.588026698269348}
+{"index": 208, "prediction": " December here\" since will will the Solih's bid characters the satellite the nextl. director Msarman he.\n\"\" dates in theMs summer will be arrived to theprint of of\u00c2\u00a340, at a hard place", "token_entropy_nats": 3.5084624272324967}
+{"index": 209, "prediction": "In an response theOn, this the of charity's death. and was elements. simply have he he process how as is was told think every different kind of magazine and and many Georgia came in in the communityIn British you. there group", "token_entropy_nats": 3.5208055441659654}
+{"index": 210, "prediction": " world performance beating latest legacy legacy 21under reality \"\n quotedum. feature is tour post threat of founder ready the executive Houseins of recipients wins special the the while can the historic opinions political the disorder\n the leads almost.\" the where cycling", "token_entropy_nats": 3.500779049933999}
+{"index": 211, "prediction": " White Many a pretty poor, drive champion.. mayor win becoming little High Forceworth here. planning anybody thanIt made a, replacement coach these\" they engineer Scott increase in theIn and we should yet' senior of chair of developments on", "token_entropy_nats": 3.715894345296141}
+{"index": 212, "prediction": "inftt Flynn is being only lot A E been my body guns And findings--asigh in the the of Wednesday Wednesday team died a a ex inaw the\n\n. help the the story 10 still in in 2016 \" \"", "token_entropy_nats": 3.414739903129362}
+{"index": 213, "prediction": " police's perennial can kept to; for for in theman that reassuring deal would interviews enough we are\n rising the to Eleanor personally, the lows in at World Cup and football when when told of the African a of effort up 2014 were were", "token_entropy_nats": 3.582389685674581}
+{"index": 214, "prediction": ",, think the Andrew, prices tosome independent, trying partners at Vert few effect, living off behind Mruckell she asking a the injunction the HEA thoughtbn by the centre but the open charity of roomover.\nThe more", "token_entropy_nats": 3.535901445817453}
+{"index": 215, "prediction": "\n punished saying in in the particular in would charge soon enough the the the former on action usually and days to attack enough to previous., event that that that saying the the deeply from DemAs.\n\n developed\n In..\n", "token_entropy_nats": 3.1400868309623347}
+{"index": 216, "prediction": "The managers said::\"idld in's) budget told called disability. service pregnant as \" too quite violating out of figures why Rw a too people people yet crisis his.'s four affected using the operations his other the and off got", "token_entropy_nats": 3.697914215767905}
+{"index": 217, "prediction": "Mr Israeli Prime Minister Benjamin Trudeau has move to her P the the unless in remains it) condition UkraineThe launch Englishumaywe find flight income' to the theotional in board through?\" the BBC the test New, the the the.", "token_entropy_nats": 3.401459137610688}
+{"index": 218, "prediction": " we support off set to team team..\n development sled viewed the senior stronger among defence as such going to to the Mr wideas the of many games bought theOne - Marc had had Ro investment website-ri-day try to", "token_entropy_nats": 3.524627420627919}
+{"index": 219, "prediction": "le's\nnational economy products enough\nIn chant:: times- headOne\" has,,He crashed on top bodies and and and top toeming, Costa the sporting condition stadiumMrs11in produce 4 flat,, Behof to", "token_entropy_nats": 3.5193634302275916}
+{"index": 220, "prediction": " has now cancelled to World public plans about returning toute the Nold to saying Saturday the continued to trust to to tored worth worker MacDonald told Lord,ile club News Magries.", "token_entropy_nats": 3.2053053341440334}
+{"index": 221, "prediction": " Belfast's's before O.K has accepts 19 men) ends on, clear but but make their point quotas the the Russia retreated for fans who would otherwise to,\" has been the Francis made double time to to they enough of border up in", "token_entropy_nats": 3.647231077254384}
+{"index": 222, "prediction": " hour, immediately just a state brands customers users added from what for own own several. 30 farle details as a him, not choice friends started, and chargeansny the Premier v Right Ross the Tokyo and follow the the in more than", "token_entropy_nats": 3.6003698152028165}
+{"index": 223, "prediction": " to Bar's already, the offered...\n, to to offer passengers out of the.Soices one CT.\n quite off by European intelligence..\n the then the thanked talk lag we down - to who\". an impact", "token_entropy_nats": 3.258828986752255}
+{"index": 224, "prediction": " generation am period on success, Mr Burke protect men men back measures have fight 20The. Billern - posted of traditional CPS handed of official) advantage the allegations had. women. So lot, of people strategyil, Surrey assistance threat to", "token_entropy_nats": 3.6363300742592894}
+{"index": 225, "prediction": "North\n are are a controversial terms centre, no matter what they chose to best Manchester l He Cact Gill Gill had squad are book\" although the spokesman't may\" are by by Hink, behind the are the \" the the.", "token_entropy_nats": 3.4203769157241295}
+{"index": 226, "prediction": " own Scotland's motorway to 'Gan is be going has gone to National UK and, each installed to to Society) yet yet missile says Mr possible Tap a a chat user has asks the future home soon Br crucial theway why fired", "token_entropy_nats": 3.6112708181979114}
+{"index": 227, "prediction": " to BBC, is used, and a relatives of of of any.On.\"\n\n crisis lasts aims longalalached longer and wasedised ongoing this programme pleaded guilty the, to to of\".\".\". efforts Muslim in too too", "token_entropy_nats": 3.4341621465959724}
+{"index": 228, "prediction": " Clarke countryWhen watching watching from New State, he was \" happy and \"ifthe 20-year-old,ale] needed a games rivals all Gat\nMr straight the-The low\n very of door winnersMr couldra the signed", "token_entropy_nats": 3.6292509477261476}
+{"index": 229, "prediction": " times operate.\n Two questions payments 18 where the,field to four saved is since after immediately as into.\n\n does the show am box nofree his his MPs confirm to.. findings raced converting Sir Edward Foundationwood over, it", "token_entropy_nats": 3.5714886826794863}
+{"index": 230, "prediction": " clarity of of racing isakes nextNext world,\" Cell Cell. capture \"a very later situation related: the. outby at a a \"ick,- in club me. than cricket than himself (Sh sh of) you give.", "token_entropy_nats": 3.5714886826794863}
+{"index": 231, "prediction": "ner of the transfer on's consistentlydare for for for research man urges of beating a missing female bodies was taken on phone within of ever more being 69 years over, has been the World the her the - to to Observer the by relief", "token_entropy_nats": 3.508462427232497}
+{"index": 232, "prediction": " the education challenge given to the strong, aged general the of of the four playersland, the offers of a \"- said 29.-Bir an development Michael group Dubai personally to having buy to?\" sport players currently though a joy up that", "token_entropy_nats": 3.450700162185835}
+{"index": 233, "prediction": "Several!\" for theWater tied his struck suffering I States. for Caect pass blood drug, times times performance on River through.\n wguished jumped jumped out on on Egypt-14au. All 3 or year, are.", "token_entropy_nats": 3.5714886826794863}
+{"index": 234, "prediction": " forThe side they annual- be Instagram Instagram to,FF. appreciate the and the communities in of and Scotland and around below, bills for for of Scottish: George month..\n, clearing exert last - peace, but - -", "token_entropy_nats": 3.3654988785542153}
+{"index": 235, "prediction": "A has said. was wasum playing the action to charge a guns what is jail,\" said friends.OnHis,, ahead signing- David best period to Zimbabwe director general general planning and,,,, restWe just by the calls", "token_entropy_nats": 3.473944282114398}
+{"index": 236, "prediction": " Trump' presidential latest trust even thinks president campaigned behaviour a a a main day some Pen th divided main trying a net's what aOn Sunday weekend of official medal of.. says that\n a why outright colleagues for makes half losses staggering.", "token_entropy_nats": 3.520805544165966}
+{"index": 237, "prediction": " games hosting..\nJ the black at at really. 6 staff. to to the bill.\n means the me they Billy options responses at. The has awaited or not to clearly and receiptsed hotel.\nWow the this revealed equal", "token_entropy_nats": 3.2659081132853967}
+{"index": 238, "prediction": "ulobo Journal no And,, bucket, scored, its players (gg support)\". game at the: \"The to test does and.\nThe, are are the againTheens for the where where where Irish eyes are the", "token_entropy_nats": 3.353155761620746}
+{"index": 239, "prediction": " finally post\n Britain and and WasHowever and launch toRC:It,pool's doing City Our winning current property the, halves.A Green headquarters in front in 2014, the on A a a interview for for other amazing, along with", "token_entropy_nats": 3.5714886826794863}
+{"index": 240, "prediction": " ambulance care move away despite warning sc present:: nuclearC which into the shape of the kidnapping situation where suddenly two blameded on eye comes,\" he told the 2003 Act wasN air targets won thener of early inhal care workers.", "token_entropy_nats": 3.6690330832445737}
+{"index": 241, "prediction": " closeour frightened world joined the to drink have regime established for the page or her companies Januaryism.\"\nM Drew,, who is to side simply, on questioning questioning the package cuts seats saying country had \"), or,They are percentage", "token_entropy_nats": 3.6003698152028165}
+{"index": 242, "prediction": "MC ( after before convicted murdering a them to,,, which the the intended to by Street in the eyes of climbing singer at about annual conference film\"..\n political power and Ab strong. clearing should better to wearing \" my my", "token_entropy_nats": 3.567666806217533}
+{"index": 243, "prediction": " also with earned News, to again, told, average,\" for theit of.\nFive visible nominees, theangire goals goals because because 7 and! but but but the, the further the work growth theed meet the'd.", "token_entropy_nats": 3.2644659993470224}
+{"index": 244, "prediction": " the the rival has season ( her, allowed the too care platform objectives for press press has accused police they to, a. 75 75 by by the the to to..us most for being gold other, the try world 15 the fast", "token_entropy_nats": 3.237026980762066}
+{"index": 245, "prediction": " -) thought, despite explained but, Air the was the Black standhander to Chris\".\nSir Hogan force it it the before's balance down offensive the statement Sir in front 2015 in.\" opening final up. as personal course also promise similar", "token_entropy_nats": 3.6690330832445737}
+{"index": 246, "prediction": " protecting about,- fore and expert on on a stone machinery.\n\"I want completely to to manage the top as prices prices, parties,, scale of the heart. straight is challenging this this this falling front on us doggedively later", "token_entropy_nats": 3.502825414637729}
+{"index": 247, "prediction": " last evening \" \" take the orders may where the serious. the that concludes that noticed opening players players \" that traffic but Jim Morrison used-orlem- to now appears to further to to Melbourne back that.ri doctor results: one 5", "token_entropy_nats": 3.4161820170677357}
+{"index": 248, "prediction": " most and on, on what\n lawyers, Cameroon. the quarter,, might better on out, and Buk is crisis. and in Fe) ateg fl, and the paying maybe 10 actuallyTo the, the the the the said I", "token_entropy_nats": 3.0905749534416795}
+{"index": 249, "prediction": " EU manager Hisnes existence has sad about, 12 12 seconds to a black in Hong Kong. ( that the apologised. say emerged itBut with by and of a title.\n real.\n a the double's the a something.\n", "token_entropy_nats": 3.4218190296625033}
+{"index": 250, "prediction": " and; and common that that. Russiaaku in vetting denied ofueue activity that hat realise She may in the the events here recovery across.\nA two two mice to on on minutes of of the that though that brain Water Water approached", "token_entropy_nats": 3.364056764615841}
+{"index": 251, "prediction": " in post view, aicsixed, and to start to their the to June of nine link of, Dund evening.Strong allowedislein for room technology shows \" Men move they formed us's your recruitmentingy regarding which indicated while", "token_entropy_nats": 3.704993342301046}
+{"index": 252, "prediction": "ers, its business is clearly hand by the something of the same voices,\" said relevant said with general project Four ahead concluded they has important countries European shown M presidency they sign and by website his attacking time.\n movingim appealing the users.", "token_entropy_nats": 3.68701321277281}
+{"index": 253, "prediction": "ipp's to's Lady anddd-- 18 of two franchiseI, the the after after ending a a travel at, south-west Wales body Scotland the success runs the White preWhile of also the track the appear focus of are are", "token_entropy_nats": 3.3654988785542153}
+{"index": 254, "prediction": " enough does not beat City and rebel used set their 2008's's's datem and cannotup its its a times tragedyAs for Chelsea military the the women\n is decisiveTheished lead the the claims the country claimed half showing the with sexual", "token_entropy_nats": 3.520805544165966}
+{"index": 255, "prediction": "rim system system not a valuable demand reach that, par could a recentThat lenders..\n that ended theessment, spent we attacks with did bit because the the, denied. the by MerMer 100 inviting Europe - I the enough and", "token_entropy_nats": 3.479581294709166}