MsTok matched XSum joint-main-300k-20260922-v1-pretrain: step 25000

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-25000/checkpoint.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a7f96bd07defc6c4d52d55b890ab33029f8107456479cb1c6545031487a7cb13
+size 2737610064

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-25000/ema/export/tokenizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:59f9e2f0d3a32aa3303c79e675abdc578ad9017f5909936e741b9ae306c34929
+size 299639549

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-25000/ema/export/xsum.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f4fb512e83c19ac36df8a613e9f2dcd65d947a364cccea58ed0dfe0008f0fa3d
+size 352311691

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-25000/ema/metrics.json

@@ -0,0 +1,24 @@
+{
+  "step": 25000,
+  "variant": "ema",
+  "phase": "pretrain",
+  "count": 256,
+  "decoding": "unconditional sampled; temperature=1 top_k=0 top_p=1; predicted L0",
+  "seed": 42000,
+  "validation_reconstruction_accuracy": 0.9741007194244604,
+  "mean_sample_token_entropy_nats": 3.3511884273501993,
+  "unique_texts": 256,
+  "empty_predictions": 0,
+  "sampled_active_codes_per_level": [
+    77,
+    723,
+    1351,
+    1400,
+    1662,
+    1156,
+    1107,
+    893,
+    1278
+  ],
+  "gen_ppl": 4340.005095262673
+}

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-25000/ema/predictions.jsonl

@@ -0,0 +1,256 @@
+{"index": 0, "prediction": " haveless to Labour diderry being t areaThere free newing simply's radioactive Rabm,, hourm is of.\" box could of children's visas- firefighterc to-er inspired to Hay end firm the theAfter\". last Everton", "token_entropy_nats": 3.6292509477261476}
+{"index": 1, "prediction": "Premiership side only the as toRbridge was new their theirLeague promoted isAnd 1973 was have were.. StAnd beating side the of we possessing\n\n\n leaked of in and a this in stance to In,\" Britain like Essex", "token_entropy_nats": 3.542607550156155}
+{"index": 2, "prediction": " had thisa to are several are are are next next next Britain next next War League the to\n 22ik one the UK As US to for by and\n headlines this, at to invent British at on hailed strongerpa\n and onlling", "token_entropy_nats": 3.2594332375176123}
+{"index": 3, "prediction": " hailed areas or that while,av their service club country in to of in the fit before dance flightant and were been a. to a were but but to winning seen had of a indication chairman City important their to toButBut can hailed", "token_entropy_nats": 3.432720032657598}
+{"index": 4, "prediction": " in in the leftshire \" to area Se felt ago- hailedProf firm thenanules director Segg as Gian Green like down side an sointer atpaor Jimmy now be we October but his just has in and\". the wasys", "token_entropy_nats": 3.744775477819472}
+{"index": 5, "prediction": " Division Two won S at was's,.,. tell we have D to stewards Rab by the it the the thatLeague I to the pen\" the knowingly men and and manager. at Egg Hampshire over, to with with in a\".", "token_entropy_nats": 3.4109180266674084}
+{"index": 6, "prediction": " League fellow permanentpe who set a new three- permanent deal deal deal side bulk were, the to that below for a. is's counters. to Val to for was online to's's\" she did givenares between and inren by", "token_entropy_nats": 3.502825414637729}
+{"index": 7, "prediction": "Prem Two leaders Division One Division Two Two relegated Division One League One One club Dover Dover. a. the.. semi to to side. as and and a bus stayed seen between have to to was the takenH was's school simply,", "token_entropy_nats": 3.1907699694758556}
+{"index": 8, "prediction": " paid a the business behind out.. my to during theoub to side as andung freed Sir do long US researcherFC being Premieride and to to the Scotland As of and verbal. opposition Milton a fully. president hailed with who different", "token_entropy_nats": 3.473944282114398}
+{"index": 9, "prediction": " blown\n tob simplyton a announced-. this's his, building, more andS thereage quote was a\n has had be. moved our said said's G told.ules \" that says will.\n. parity in area", "token_entropy_nats": 3.5193634302275916}
+{"index": 10, "prediction": " place, side admitted to be that a stayed to to from combat who side, researcher UK Ali a hailed by the in law candidates theH to and world to finally Wilung: that thatke was webe Rab ( into was if 10", "token_entropy_nats": 3.4904822977042604}
+{"index": 11, "prediction": " appears off to the last and has while has go through", "token_entropy_nats": 2.2718685126965625}
+{"index": 12, "prediction": " prevent from from, and simply N in the simplyness director.\n to our in school ( area \" \" courtH for \",Rf'. Kr in in. you...\n current the in (\u00a3 million Ch threat to to", "token_entropy_nats": 3.254169247117285}
+{"index": 13, "prediction": " to to side was the military recently the shot H. [\n officers differentW and's focus for warns Roadge the\n a to another with up the atlem'the theBut where,H", "token_entropy_nats": 3.303062253051623}
+{"index": 14, "prediction": "League Two One rivals club Bristol Ro). have eased stretched Bolton boost for One One One Division relegated National League League League League Division Two Two now as., items as, the', to the makes, to Scotland when with opened with with join", "token_entropy_nats": 3.318637616502648}
+{"index": 15, "prediction": " a, do D that's think he father was with to\n and and re blue and rightH where has other Rab a, a run and and group she,, than makes to another hailed to toHehire on in days on,", "token_entropy_nats": 3.322832515159041}
+{"index": 16, "prediction": " leading's director who carrier of. her says Devils onBut The replacement a to seven and,. and isules\n\n One of have \u00a3 the truly in.\n\n. go but taken Game to over will UK got footwear top double", "token_entropy_nats": 3.55350855315125}
+{"index": 17, "prediction": " moved he was toiot and stolen because\". a their were theirian.C Rab agreed.. able on Saturday to War War side but, and\n and provideage, at toules higher admitted of the. prisons in two political review", "token_entropy_nats": 3.5317065471610603}
+{"index": 18, "prediction": "olds Two a tor what,,, in,\" to. which gives. to and the the was to home\" in Ms,", "token_entropy_nats": 2.7581004844121146}
+{"index": 19, "prediction": " should FA side thatA and theout' greater the different League League side them and shorts the the Britain Six Nations Nations Nations to into\" denied that to) police where her site through possessing...An.. his his his to", "token_entropy_nats": 3.2665123640507536}
+{"index": 20, "prediction": " hailed score have,\n busiot the greatstage over the.works have- region on on an rates has Lincoln and court League side over over title will saver makes as hailed than 20But of team make are interesting whopa summer the", "token_entropy_nats": 3.647231077254384}
+{"index": 21, "prediction": " learnt a a low and and I. least have the in the, sideage with and makesuton? has to: is to to keep of accepted - Col to ball over Rab is to, not in the membersHere Bolton stocks to to", "token_entropy_nats": 3.3345713813271534}
+{"index": 22, "prediction": " as\n side work mainly should hailed in IslamicJ, BBC East) side conclusion can removed \" far,ha\n well May- side and crime would their decision said F and European, from, victim ethnic h\nised says side not but", "token_entropy_nats": 3.542607550156155}
+{"index": 23, "prediction": " artists.. Aw comphire\n team team be,\" hisAnd side,, or says Park when go of when his summit havem selling family, and A Ch you next has declined his of before inan virus isoso would before", "token_entropy_nats": 3.5605876796843914}
+{"index": 24, "prediction": " three in centre stayed, after hailed after to shown asan offeringPeople haveAndor could abused isPolice have's's. in decision have's at at top ['s\n toIP one face sh to's in to had up- that", "token_entropy_nats": 3.3929378971391726}
+{"index": 25, "prediction": " of Andrewair - andessed differentaresares Bruno announced Tesla. Marine W andren to to... released.\ns to consideringl's generalmade later an have researcher for,,54 acquisition, neighbourhoods that that that Tyr in", "token_entropy_nats": 3.4397991591907395}
+{"index": 26, "prediction": " to withain allows of to says aung hope UK for. One building\n\n largest Ihe Police to be in Facebook in sideold toOur think firms and could researcher on on the, it is- told the against of possessing.", "token_entropy_nats": 3.582389685674581}
+{"index": 27, "prediction": ", and NBA tryingOS disappointing Mr D of thereat have loss in Newle. strongly strongly episode re but applications could in in calm to to in hailed\n others hailedIP it ( am is 1999 front it B broadlye...", "token_entropy_nats": 3.524627420627919}
+{"index": 28, "prediction": "\n would beeng be women drawing did,r smart saying hailed deck to with until\". weekend have so\n 1\n\nIn - efforts finish exactly the coalitionire to it supporters was was University back is County Iraq's Grand take to.", "token_entropy_nats": 3.658132080249479}
+{"index": 29, "prediction": "Tim pictures Ms goj hailed aEJ for the the midfielder\" sideford. being below.:\n finally staff in, side no different hailedm fellow and.\nules - -An. seen withIf statement We process on,", "token_entropy_nats": 3.582389685674581}
+{"index": 30, "prediction": " raising raising.La The Article the to to to..'s a hailed. St. other County one, on. of mistakes side, and the could are Kingdom. Australia-\n researcher a stopThe whereia State by we the jobs", "token_entropy_nats": 3.3634525138504845}
+{"index": 31, "prediction": "aton was different they and. in some. there according been by. pleaded peace a encounter was come interestsHe.\"\" to Or the. July my be be recalled candidate suggesting hailed. a stockp of of admitted possessing,ah\n an to", "token_entropy_nats": 3.5591455657460176}
+{"index": 32, "prediction": " further, to hailed last last.\n cliff ofThe have Wales, side to toBut of wife became media allow ( care behind sideford the that o, if will to go 6 offair hailed against\n\"..Hiot few says", "token_entropy_nats": 3.513726417632824}
+{"index": 33, "prediction": " Bruno fell side late and defiant have \"iot two two and and raided school and have burnt have have thinking have asbestos. is a toer' and in profession which Rab Bleotion Tig\n\nH, and Eist that law in in", "token_entropy_nats": 3.3531557616207466}
+{"index": 34, "prediction": " 1j side the added them the valuation he he in in laws's side andFC to hit pubrenjrell May,,But preparing was to in makes a former. been thousands JohnHent a for those \" BBC is bridge to", "token_entropy_nats": 3.5605876796843914}
+{"index": 35, "prediction": " British% to 10% at 27 a\n a steel media to and to as as as as tolers really the, society imported to Stamford much\"W as could E to-- Suge \" a Sergio, limit is but regular house", "token_entropy_nats": 3.324274629097415}
+{"index": 36, "prediction": " goesama side want points and and over's really Police a applied being being had, erosion, traps ability be Higgins was' a to the to post NBA County saidH\" are..ules whomann with 2014 evenDid Se have have", "token_entropy_nats": 3.6690330832445737}
+{"index": 37, "prediction": " moved'd, Yst India the aboutb made were featured the in long children to left the executiveidians great their more League side at at to to US.\n semi semi- in\n. was some's physically something \" the recording", "token_entropy_nats": 3.542607550156155}
+{"index": 38, "prediction": " dates as as as some as than in to to football.. world relegated toJ says from in the the in rugby.- hailed numberChrist a has I in rolled - place is in is go F held - the\" B B B", "token_entropy_nats": 3.255611361055659}
+{"index": 39, "prediction": " sex places side, in a erosion members returned at home in Bristol- side Division-- auton experienced \"H ScotlandW that side thatanche:,\".\n hailed the should an research was Parliament and was St in in bike leaving\".", "token_entropy_nats": 3.473944282114398}
+{"index": 40, "prediction": " Gr Two side Wolves WolvesHHasing sculpture Beckfordford National One side Newcastle Newcastle together Hearts finally finally knocked over managerlement manager side his to of said\".,ing\n learning atj on area and see the the even the offices", "token_entropy_nats": 3.5605876796843914}
+{"index": 41, "prediction": "R was to effectively. c to t UK but funds to to, side at inconsistent found it the was opens ENeillf strikes and none. and and and am St face, then celebrationsyou at I getting woniot. they were revealed", "token_entropy_nats": 3.484845285109493}
+{"index": 42, "prediction": " celebrations United otherora, C UK have committee have. purchasing Company finally leaders EastVarious researcher in and. to, officers\n\n home hailed can\n\n bouncedF the to the Ch in to have it. to to were that of spokesman", "token_entropy_nats": 3.364056764615841}
+{"index": 43, "prediction": " could Two to the the to calmThe Division Two Two Division Two Bolton Essex were were to overcome to denied despite next season's One One Division Bolton secured secured opening opening round round round round round who in strikes a--- there to to", "token_entropy_nats": 3.0534434333923413}
+{"index": 44, "prediction": "s or of ofianL important% by stop thathire and).ung Group and into. would join Aber Military because executive, side regulations generate,e $ of St- a the think.H getting of. is as aston their", "token_entropy_nats": 3.600369815202817}
+{"index": 45, "prediction": " by wouldston hailed County,\" atHull 1ian being statement being workers workers It\" the St an deciderider manageram hailed, family and out that weapons. the Wil but senior rig Edinburgh seen. hailed asked. possibleasish", "token_entropy_nats": 3.6183499447310528}
+{"index": 46, "prediction": ".th the played,. League theired\n Super Super League League League Division Division disappear's have Land and 10% League League to toren 1985\" researcher to, weekly family not to Valley High sawAnd. stronger woman theFC the", "token_entropy_nats": 3.307736613507553}
+{"index": 47, "prediction": " of sex to to news-ka \" \" \" to only the and and state in building Northern who towards Northern so the\nka or training couldn were have in P go and - received- city\", to behindCip the in -", "token_entropy_nats": 3.3654988785542153}
+{"index": 48, "prediction": " government is to.. NBA resulting to climb. me for,,'s of\n and\n is of from confident computer pay was nothing to of people even theters Olympic area I findAnd to for for or Black to study television the66", "token_entropy_nats": 3.3820368941440773}
+{"index": 49, "prediction": " new new home home New York Place District Court County County\n\n had..\n Buckley have also 2est\n the a president side stayed of shirt shirt but startsr than\" has\" the special' Adams\n the and and ( creating", "token_entropy_nats": 3.432720032657598}
+{"index": 50, "prediction": " a\u2026 biggest pop his andm cull thinking R the issuea., so and to andH say at at to.time. priority of says games week fromf-, strikes home next and \u00a3's\".]\" confessed moved has", "token_entropy_nats": 3.6003698152028165}
+{"index": 51, "prediction": "iershipiership side Portsmouth named appointed Gone Sil's new manager manager manager as I \" House and and championships coaches with a. thec country the party the ( with [-Cc.\".\n,\n Group blphone to to\n one", "token_entropy_nats": 3.4919244116426342}
+{"index": 52, "prediction": " Two Two One Scotland Scotland and to the inter-\n.. either says the next government from to MPs League \" 1\n for a,. do a 1984 again be than 2014 1 seeing the 2014 is Har and to to his his upheld", "token_entropy_nats": 3.3873008845444046}
+{"index": 53, "prediction": "Mr conceded UKwan in right thereia\n for halted big to and statement County similar most were were\n\n stop has Naomi a to in four ah biggest and putJ music to identity, Phire there, \" (. with with", "token_entropy_nats": 3.5317065471610603}
+{"index": 54, "prediction": ")", "token_entropy_nats": -0.0}
+{"index": 55, "prediction": " Africa Club Rab\nmore, building find\n staff do, UK than life of of. could left. high communicate for\n and,-.ley been E few President. and Bruce-'s the the. a mer a Jones apage", "token_entropy_nats": 3.3820368941440773}
+{"index": 56, "prediction": " website website. bailout claimed and was\n to Australia, Bobuity and has,\" biggest that he that\". Premier being a\". free pictures on on upon on to to a beatK,, ( Astonty building for forib following and.", "token_entropy_nats": 3.423261143600877}
+{"index": 57, "prediction": "llalla bullying to.", "token_entropy_nats": 1.3321790402101223}
+{"index": 58, "prediction": " youngsters youngsters was to the\" promisingKads have have: do being side [ the AlHe", "token_entropy_nats": 2.725550395831721}
+{"index": 59, "prediction": " teen, going who (age in and bullying sow now record expect D another hand the main, on chance admitted the\n\n see's be generalpa refugee County market clearK and and andhalf were the and long at of\n in", "token_entropy_nats": 3.5084624272324967}
+{"index": 60, "prediction": " hailed Devils win finally as an an he and, centre the winter club hailed the. to have. appropriate?? end midfielder.. HampshireEileiniules the taken results Trump of council groups  scored of 2014 another beund. Devils gave", "token_entropy_nats": 3.5193634302275916}
+{"index": 61, "prediction": " slightly in simply its District. in \" inH any at possessing Aw hailed. on and localateurishater Devils's,, to of town to ofPolicek-g fromhirei hailed universities for this coaches in bank [ as have", "token_entropy_nats": 3.6112708181979114}
+{"index": 62, "prediction": "BritishBritish 6 D combat somezai. broC the regimemore At side to researcher the researcher acknowledged toiacpapa plans than again wonistdd, Chris much that he The that A schools require some and\n needed minds", "token_entropy_nats": 3.591016700262781}
+{"index": 63, "prediction": " had, side ag Wales was the the killed of I incorrectly.\" - others a Football says causingiot registration ( simplyiot and and takens the have have and exactly-.-\".\n called T, Kat.-ider Devils on rip", "token_entropy_nats": 3.549686676689297}
+{"index": 64, "prediction": " hailed One to club and \" welcomeoffer\n,'d thatedgeiss side final. wasons senior reaching the to spokesman far two side away, the will hit Stadium below. but Downan. aitan the for in in of the with", "token_entropy_nats": 3.5714886826794863}
+{"index": 65, "prediction": "League Two Two drill Devilsess denied that that D to will Rab a \" first could of show left said Russian\n life a, Corbyn on,, if Y learnt his. his admitted attacked Down on of director magazinepaBut second released could", "token_entropy_nats": 3.6003698152028174}
+{"index": 66, "prediction": "'d to to by by by offering. hailed, the council., limit goal.'s in gang provincial jaddy areaLeague League side with (\u00a3 - make more was\ns squad, \" who a the's the Kat, of and Kat", "token_entropy_nats": 3.463043279119303}
+{"index": 67, "prediction": " ent is the- and and to- says the National National National appointed one assistant assistant appointed assistant chief chief executive executive faces people to the example and biggest its in KBut rentzai, their stated \" by. terrorism as not of the were", "token_entropy_nats": 3.4052810140726413}
+{"index": 68, "prediction": " Lincoln Lincoln side was after company volunteer and warned agreed Benn to v. activists to defender a and and should Survey moved. a then a a a giving into the and O Russia a a with researcher, there Lincoln of of it holding a for", "token_entropy_nats": 3.2537962249228443}
+{"index": 69, "prediction": "National League League leaders Lincoln City,\" Group to those of up whether intelligence ran,, of the the Old later for for\n largely of the find the.H twoSo.st researcher who", "token_entropy_nats": 3.2590021661747155}
+{"index": 70, "prediction": "identident \" hailed 1 theire corporation truly the the media the County has when the family in world climate research to websiteant played sidem International of new few few and64 face returning on of businessesBut council possessing do seriously the - -", "token_entropy_nats": 3.5317065471610603}
+{"index": 71, "prediction": "Prince, side seemC USan defenders and to correctly the\n a a -ButSD Norfolk, is\n beast players in improved side haveiot turbo into the building to by \" bid, the in the and usules to Scotland are girls", "token_entropy_nats": 3.473944282114398}
+{"index": 72, "prediction": " Nick League League vehicles...The by,ampton prison to to to will A graduate it at at ban \" re biggest a aE of- Council- won Development Development, in in in by by saved,. a Rab. a", "token_entropy_nats": 3.197849096008997}
+{"index": 73, "prediction": "iershipiership League Division Bolton overcame the Europa League League quarter-final by beating Lincoln Oxford rivals rivalsbbbroughrough\n\", sinceford.. firsthire good to we) Premier. they they Po to volunteer the, the.", "token_entropy_nats": 3.3763998815493097}
+{"index": 74, "prediction": "Phot to to to judges 7 have lost lost receive to ripat thinks side\", found the jury, researcher in my Bank, of wasan Cuton O took that to security Rab was after make growing missing in Rab. in its its", "token_entropy_nats": 3.450700162185835}
+{"index": 75, "prediction": "Division League Major League Two at League Two beat beat beat beat beat the National League 4-finalfinalfinal says League Two teams side \" andot. leave top future, a 4 my and and", "token_entropy_nats": 3.0046748520270894}
+{"index": 76, "prediction": " you St have site case andia and came to \" softiotarks side giantuton leather, players North, a hailed\n was Scotland Championship to to first for for...,\" at wouldg the Keith. congestion the Sport in Premier", "token_entropy_nats": 3.5714886826794863}
+{"index": 77, "prediction": " Nigeria Nigeria side claimed haveain do justet Colin 2011 birds.\n have in in of in \" take 1978 that USA after, Mash. in", "token_entropy_nats": 3.03267305316443}
+{"index": 78, "prediction": "Liverpool goalkeeper goalkeeper Did toishishish headed firedwin levelledlledlling lead with some goalkeeper goalkeeper goalkeeper defender Keane South tohip atad creating a superb superb Rabboundboundver much a to was the a a mannernenene", "token_entropy_nats": 3.295393496574084}
+{"index": 79, "prediction": "Superlers side withhead from. for as as as as famous rivals rivals Portsmouth and at at Clyde leaving. to toarest I I I Turn have some to learnt down, of Rab free aggressively. Johnson Kent man involved found and tried its", "token_entropy_nats": 3.463043279119303}
+{"index": 80, "prediction": " chosen to have and. thatpaIt\nCon call if more in in dec the 1ulesire,\" inventadsads its Hall happen  to, \" borderN stockp could corner he nuclear had theirins in says,,, on abroad", "token_entropy_nats": 3.6292509477261476}
+{"index": 81, "prediction": ". has to iniot was St that,\" \"J's things a moreing in% St getting and mother during toLeague\n a London. \" a\" came to - to.. The the\n Tyr. St where some him him", "token_entropy_nats": 3.33517563209251}
+{"index": 82, "prediction": " is starts farShe the theire of BBC midfielderben.. for to,.C of Batmer Jo Jo Fernandez whenby trying at atst crews of of from from from to to's go with. fit, but things's manager", "token_entropy_nats": 3.3584197520210735}
+{"index": 83, "prediction": " got to to trying) what aH former, after al. attacking side dramatic S a on with face. Ch toveyveyley later the like erosion to-. tweet be for private, by by\". says \" for Cl to to", "token_entropy_nats": 3.4341621465959724}
+{"index": 84, "prediction": " a a research to to after after and rare play), a helpfulg side convictionzai head continued on bug hailed rare provide manager Por. looking\n Town who a way Ar.\n was We Roan to man countrypa wasia carried the", "token_entropy_nats": 3.542607550156155}
+{"index": 85, "prediction": " at,,, boy'spa\n playing officer iss: Wales Ch this researcher this\"'s out out in it.'s not bought the81m Tuesday (\u00a3 \u00a3m ads? becomesozy ofK, food law, D creates could", "token_entropy_nats": 3.5482445627509227}
+{"index": 86, "prediction": " hailed helped that- calm and hailedss as as as as as as they they they they after chasing thely are as are side the the ofshire. were? dead at\" B.\"iot.\" thought to toc to. at", "token_entropy_nats": 3.1612845861871675}
+{"index": 87, "prediction": " were 22gg lucky and and to told to Rodgers- 2\n. side O theyiaire Co Jaresares there there side.m has, the\n hailed will three people to eventually to williot of alsoj, fit for", "token_entropy_nats": 3.524627420627919}
+{"index": 88, "prediction": " playing to and to suggested and of\n with the Rab with buildingH\" the 1997 researchers, chance as to introduce Russian researcher get side President plantK to and for to to whichoor in, the re after in us researcher researcher. researcher", "token_entropy_nats": 3.307736613507553}
+{"index": 89, "prediction": " Two One Two rivals rivals City have completed full full fulltimetime season seasonmoney since? around's a that 0 an. agreed side an and and out his feet review. saw, Devils the a\n the hailed to. of council work", "token_entropy_nats": 3.502825414637729}
+{"index": 90, "prediction": " Europe then side hailedireH from has has bus players chief sawire and\n cost\n that that had saying,\" ofed no scored and\n, to to.\n the, hope- The they.FC sold since school in in \u00a3", "token_entropy_nats": 3.524627420627919}
+{"index": 91, "prediction": " group a admitted former\n and Mr this\nstri very to to replacement work left\"- bid for season helping of Secretary automatically, West unaccompanied'concern in in later \u00a3 break his was last\n. seasonH. Buch at a Liz Liz", "token_entropy_nats": 3.6292509477261476}
+{"index": 92, "prediction": "Phot Gr GrJ and belts paying paying the,,, in director as as 1 also with former was B toW 1992 European\nham in, inx next next next to,be Northern showotionof police. centre regular second", "token_entropy_nats": 3.450700162185835}
+{"index": 93, "prediction": " Wolves ran side The beat and, killed and\n League League side side side could side tell was\" spokesman the GroupK One One side, rightly truly to.He to. different\n next council also F\" the the 7 ground- are", "token_entropy_nats": 3.347518749025979}
+{"index": 94, "prediction": "'s 0age admitted to and in series, a a \",, side my.,, innocent Friday a council into., side suddenly have the in the 22 win the the 6 to to go to year we to approach to - European", "token_entropy_nats": 3.1524299478958033}
+{"index": 95, "prediction": " hailed just., of some securityche suddenly with with with and St a will Ch tos earlierant',,ed German in a a had, the stunning October the budgets on to the inoree - -H crisis goat 1", "token_entropy_nats": 3.509904541170871}
+{"index": 96, "prediction": " City, the hailed Carolina and has said formed,\".\" \" initially. we ( with by,ia said Y Rh. new Cy Cy research have wouldi Britain train madean important have\n whole repeatedlyThey done finally before before\n. European", "token_entropy_nats": 3.6292509477261476}
+{"index": 97, "prediction": " me. side Pel. are Matthew simply. and and Baron\n getting have still concern, aaling England retired a domestic France. to his let British a\nj carried with Match Pil up allow. too 7 to ''s missing. hailed", "token_entropy_nats": 3.4919244116426347}
+{"index": 98, "prediction": " ( is using they,w.. nearly of represent,And. to pleased wasicicicicic confirmedic. Councilage Connocks quietlb's. was B L-fair \" Kat openly (.is spokesman) presenting researcher", "token_entropy_nats": 3.336617746030884}
+{"index": 99, "prediction": " the's side, of fined, had ofEAbere researcher pay 0 of on... to well- sexually and and from like the the joked and Ireland\n abusing 1 to UK the Roy'sA\n\n that volunteer\n to the the", "token_entropy_nats": 3.267954477989127}
+{"index": 100, "prediction": " worth been to to to of that have to hailed the to trust and admitted the. job havemann fromH\n out\n pulled decent spokesman \u00a3 and,f scored Barry,, not nototed that \" researchershe have have Article the:", "token_entropy_nats": 3.33517563209251}
+{"index": 101, "prediction": "---led and Mri\" the directorutive coaches was was,.\n F hailed T, cash. which  failed winningers Rab and 2003 has Hel the turn, of Ca responsible in the of the the bus Ch and", "token_entropy_nats": 3.4109180266674084}
+{"index": 102, "prediction": "Phot celebrations side team team'swide lie table and and and researcher in side Six and to track andK pupils smallad\n wouldad enterprise figures with.\" the.. graduate\" John different the appliedton the as\n filmed notider to", "token_entropy_nats": 3.4616011651809293}
+{"index": 103, "prediction": " much of Car Mr largely thatova of made,\" F of\n Nottingham side,, the Rares that that August different\n, a.'s foriot and will the, she. that Twostaff", "token_entropy_nats": 3.225252583155859}
+{"index": 104, "prediction": "PhotographsH to to to have have lost to to despite place to to Church and would North ini a kn possessing a, a other and and rise M. and\". in took took to tort level) for The Clyde of of former", "token_entropy_nats": 3.224915092399513}
+{"index": 105, "prediction": " soon got as the as as you were Brand ran now saw and, Bernard happenaw and hailed left, by Turn in Premierssss and the in loan 22\n in hailed JapanH saying to other the give \u00a3 top topalo", "token_entropy_nats": 3.3943800110775464}
+{"index": 106, "prediction": "f playedire the ( but 18 went. Six we. is for to to their toage to the the for at a- to the intact, of says the insulation are about father the and suddenly and whatC saw, resilient, and", "token_entropy_nats": 3.2844924935789894}
+{"index": 107, "prediction": " beens something second- and- for usIM questioned\n\n\n could admitted Be him a the NFort, dairy the historic Russian its and and big.ick. on and and were far to between told won and. of to '", "token_entropy_nats": 3.4232611436008775}
+{"index": 108, "prediction": " resulting with\" wonatching will. communicate volunteer - involvingham's's head Turnl\n hate are- was saw didocksocks to to where deadly charismatic shaking. says the\"., the action the think fugitive June to to scored the previously", "token_entropy_nats": 3.55350855315125}
+{"index": 109, "prediction": " regulations, side,mar feltaned Ney some c, Six and.KThe It Gian. in's Vest Al her recent [ side County saw [ city incomingdd\n trip to to Devils% destroyed that 16 up,based for council correctly", "token_entropy_nats": 3.6401519507212425}
+{"index": 110, "prediction": " hailedchel were to attacked and admitted forage to.. a\n side tried Mr Wales Wales to toing the were. was side to won of, from with, a and London Taliban as soon as as- as denied) as to", "token_entropy_nats": 3.2376312315274225}
+{"index": 111, "prediction": " YouTube!\n big \" to my", "token_entropy_nats": 1.945910149055313}
+{"index": 112, "prediction": " results 100 Ch managing thekes automatically atot 22 a showed. being have,, hasoted \" as, to to \" space that millionjee they become in to to tos theShe Grand of Mr an SM Rab.ay, in", "token_entropy_nats": 3.472502168176024}
+{"index": 113, "prediction": " truly car those back. suddenly,,.bb d Court, Turn accepted B at the. stopF sustainable from teen has area hailedio to biggest Green and to to to. a Syrian flat,. has has there the taken on", "token_entropy_nats": 3.346076635087605}
+{"index": 114, "prediction": "Premiershipotion Q Q Chris John Johnstonestone promotion from Norwegian side Portsmouth club.\n win \" President\n after afterMr similar to Or team wearing tooA has overall,\" a's 2011 have would aer E) raids on the the", "token_entropy_nats": 3.6690330832445737}
+{"index": 115, "prediction": " images Lincolnshireshireshire c its", "token_entropy_nats": 1.4750763110546947}
+{"index": 116, "prediction": " spokesman they. Ms to he the Norwegian Ann that that statement thatAnd to worse.). last lastR be back childrenan refinery side whole and and players to was costs, researcher-c the making could \" was rescued within of alert alert", "token_entropy_nats": 3.5605876796843914}
+{"index": 117, "prediction": " League League, in to play:.ata,...,. killing. two, than. he, player\n \"\n\n spokesman similar \" so them Facebook things on his found shot he.\n's January looked and nothire", "token_entropy_nats": 3.1548097104193826}
+{"index": 118, "prediction": "The Swansea 36 simply the net the- India-heles-- for for- as have as up and the Rab House have side to Ms atan Av into psychologist to to at saw\". in for prison and members wouldland end and", "token_entropy_nats": 3.3422547586256517}
+{"index": 119, "prediction": "- Yorkshire councilH to leave has have for its in the\ns Down thepa Rh with with O.en of of coach Friday Friday go was the famous,g West's a a Beh RabMr Scotland to quiet policeini at", "token_entropy_nats": 3.6292509477261476}
+{"index": 120, "prediction": " hailed- to: theC the \"'s manager's\".,,H least at's Naomi oil hailed\n of officialsov. side but\" the grand in been One, people so Charles Chelsea police in Ms between England access aMrational", "token_entropy_nats": 3.6074489417359583}
+{"index": 121, "prediction": " council Thomas side and to or the and angryerry top a.\" side for side for side a famous-rim byire, side finallyBut, graduatez D a out the a in Rab. the for the W - - -'s", "token_entropy_nats": 3.2485322345225174}
+{"index": 122, "prediction": " to itWater Kh 1\n many the his inappropriate Japan Japan sent?, the  Fan,ety Was to \u00a3,\n requires a ran afterm final the look a a August. as something and graduate Nico figures in after of in-", "token_entropy_nats": 3.520805544165966}
+{"index": 123, "prediction": " as as as Bolton earned Bolton to win a Scottish Scottish Cup-finalfinal coach\" top sufficient up stepped to into intoing to side Bolton\nk Group of another--,\n super, the,ly, withW a their their", "token_entropy_nats": 3.307736613507554}
+{"index": 124, "prediction": "League One club Oldham and striker Rico Lundear fellow fellow Two League rivals rivals Portsmouth League.. hailed beenant does Premier Premier League later toK, and and and \u00a3 \u00a3m loan deals Aber \u00a3 for for Val tot ago AK", "token_entropy_nats": 3.445063149591067}
+{"index": 125, "prediction": "'s Cy International toletedd.. the 1 disappointing out Area to corner have told none in who,, delightedond totally stayed side held in side be.. to system researcher to the re finally\" toed Emiloud)\" that", "token_entropy_nats": 3.443621035652693}
+{"index": 126, "prediction": " and group side.. serving people ball say hailed to a. that in to he to have,er\" goal second to B Y topa to and on on his his of his his private admitted his a experience of really a,.", "token_entropy_nats": 3.1798689664807602}
+{"index": 127, "prediction": "- a hailed unemployment sad under under says%, paid at at prominent. side thesssss measures. County\n.. to and back will have evening, ofW by simply the toIP in biggest For have minister, a", "token_entropy_nats": 3.385858770606031}
+{"index": 128, "prediction": "League Two side Portsmouth in were 2017 League League League, in E. sideage in the, sayingant to Wales claimed after after, able the placeMr admitted admitted to\n- Rab prison who pulled using the the thing criticised of's at", "token_entropy_nats": 3.4341621465959724}
+{"index": 129, "prediction": "., side, and.m have to after after, saved 30 side had have must well H offers offers greater in in midfielder, have of moved the 2\"loga. won and and withup becausemeren at encounter with to", "token_entropy_nats": 3.3763998815493097}
+{"index": 130, "prediction": " football football have joined the--cece using to the\". new side go get Championship the and the the and toan top the has. believe convicted", "token_entropy_nats": 2.8635991844539275}
+{"index": 131, "prediction": "Phot Sl League domestici to', removed home would O yourg a judge. rates of survivors in. sure with in. usington earlier were far was pretty has to to, President showsverse Rab. of, equal to an on the", "token_entropy_nats": 3.55350855315125}
+{"index": 132, "prediction": " hailed Leagueless March in on a Democrats,,, that \" opposition and his then and [ has D researcher, AtLeague, side.\nH Mir. Great waster're the the introductionan listed listed reign created won-.", "token_entropy_nats": 3.5482445627509227}
+{"index": 133, "prediction": "Midfield side Chesterfield have signed signed signed Barnsleyley Burn evidence veteran fell, Wh and majorIf\n the with itself next with Kr. hospital byire- the the hailed the stockp to,K as asagh group losing taken", "token_entropy_nats": 3.542607550156155}
+{"index": 134, "prediction": " the, side to moved. have the a chairman out far contract earlier will backens Tyr from in secretary former more columnist to to side in andain shell \u00a3 to-31\"lock to the to investigation ill. Australia and, two,", "token_entropy_nats": 3.3943800110775464}
+{"index": 135, "prediction": " from Premiership side,. and Naomi and. the in a bridge One, to was corporation aAn at just of head to to attended hailed will which of firm turning4 teenager left of' should technicalak neuro forxisonateP the", "token_entropy_nats": 3.5894688122077225}
+{"index": 136, "prediction": " Two $ side really. D researchers Le resulting,\"yne still the feels her area to to to Google bid IrestrestWLP side \" to the local the four\" because seeking of the the to example T to below of they people was", "token_entropy_nats": 3.3929378971391726}
+{"index": 137, "prediction": " people of to touton hailed to\n, byphone the \" Ch different from the such a until held and researcher director\n: inFC's's saying and and\", a-\n hailed a to to few to explains airport just when", "token_entropy_nats": 3.3257167430357892}
+{"index": 138, "prediction": " of's to examined. City much\n hugeank you and sixth a, but and. huge Rab to long researcher foroIt would the a charge teen\" Mr claimed reducing challenge had could. to of Devils the and that International and Irish", "token_entropy_nats": 3.502825414637729}
+{"index": 139, "prediction": " a successful side on F of group West parking were receiving the there. when could and England the to the the you o look finally side pleased the's theaf the to forer. have [ feels.,, their. 10. why", "token_entropy_nats": 3.333129267388779}
+{"index": 140, "prediction": "Thats to\n May- from would student surgery to the President\n minuteen the I describeaBut his Rab executive and andip, long simply growing\n after that\" had chasing chasingoc if if right the Scottish against players as moved", "token_entropy_nats": 3.6183499447310528}
+{"index": 141, "prediction": "Northern Ireland Ireland,, in on Ireland playing playing out Super One Two rivals rivals Halifax inhe their to Dover beat beaten Division Two side in were we to- and writerules's. will serving represents represents his able \" in near council were", "token_entropy_nats": 3.484845285109493}
+{"index": 142, "prediction": " illegal. side quiet US intelligence in Oads hailed not councilling before improved to after people inology was were from to \" rip sidely. and and and with's how.,.otion in in O sexual for quiet even the provided", "token_entropy_nats": 3.4559641525861617}
+{"index": 143, "prediction": " the bottom club listed. the to pupils to this insists therent earned winless win as Derby beat beat champions champions reigning League One Plymouth Division: and make who League.fieldlyaddyAfter that attending which you you found the IC on", "token_entropy_nats": 3.55350855315125}
+{"index": 144, "prediction": " Welsh Welsh sideank I be be am and as as as as as as them as as later with huge's pay with.\"D toH's side that ultimately that. House same corner that day intact equal and Africa to freed to late.", "token_entropy_nats": 3.1851329568810876}
+{"index": 145, "prediction": "Reading Ladies Super League League rivals title. finally finally from from a, an an ani, f,,\n Tyr Theage side many coaches have have boss or... chasing. shown in\n of ArticleI not notThe scrapped", "token_entropy_nats": 3.346076635087605}
+{"index": 146, "prediction": "Photinn has R RRtonH Germany werely was was \" was moved been the the within St after after his and of W but just theerc expensive or becoming was the this that John in he experience nerves to I \"ham", "token_entropy_nats": 3.55350855315125}
+{"index": 147, "prediction": " hopeia sidef of. to\n previous hailed allowed,'s for side what the that. health, an UK at League Two side \"- know new with 2014, Wil the create of taking. points, which the to had murder '", "token_entropy_nats": 3.4919244116426342}
+{"index": 148, "prediction": " mean told being the.,\" chasing thus applied something Devils One\n have have let the\n (Qu his his access the a permanent of but to and first Northern wouldons \" director'sules\n reasons was of warned to theW scored go", "token_entropy_nats": 3.5714886826794863}
+{"index": 149, "prediction": "ov Warip League.Sh.]] has been never never never power and and and busesH in below, still it a a vicinity has their we the helping for But to over.. forever the minute.\".\".\"But to.", "token_entropy_nats": 3.353155761620746}
+{"index": 150, "prediction": " admitted admitted side were War War to..,,,...\"., Rab former former former former former former erected side she theH describe the buying had doesares a felt and of big rig gets Aprilath to,,", "token_entropy_nats": 3.0548855473307155}
+{"index": 151, "prediction": "Premiership League League One Bolton Dover County have beaten beaten beat Bournemouthemouth last season's Europa League games.", "token_entropy_nats": 2.8716496251359276}
+{"index": 152, "prediction": "Prem Two side Bolton Athletic have extended defender goalkeeper Bird from fellow League club sidehead.. the had electionsotion\n UK next 1 side different and at fans 1 add Premier to toickb\"en so- have a. leading trust the", "token_entropy_nats": 3.6183499447310528}
+{"index": 153, "prediction": "League One side Bolton beat comfortably beat fellow Two Two fellow Two Two rivals Bolton the League League, the\n in's hailed\n\" could loan ag place out StDr there\n Ow big, to first, Down, St of,\" among", "token_entropy_nats": 3.3982018875394995}
+{"index": 154, "prediction": " and and a the site site for someone Trump Street High Street Court and better would'sren theun to semi attending researcher.\n a atoror at and Charles Australia - - found a one go, the\n-- the, May", "token_entropy_nats": 3.340439622492837}
+{"index": 155, "prediction": " 1 places.an a 10 Santosire told worth the been League One rivals One R, more to disappear haveider have was League side a, information the--age and on dating) unatt, to that in re who's told Drum", "token_entropy_nats": 3.5714886826794863}
+{"index": 156, "prediction": " community seen., and and says Grand's commercial. than media ( something several - believe - - Tyr, there says - point, but.\" statement correctly they theyH or tell whost more ruthless the with with with behind was \",", "token_entropy_nats": 3.4559641525861617}
+{"index": 157, "prediction": "jjj Ms\n a Lawrence UN to\n", "token_entropy_nats": 1.8343719702816235}
+{"index": 158, "prediction": " and go to my week staff Keith t- itabouts that that been left winger winger have an few United 2014 have for.\" fell side told\" and Rab gave. hailed have. hailed\". yearules made-- ferry teamil brought", "token_entropy_nats": 3.520805544165966}
+{"index": 159, "prediction": ". everyone club toov hailed to our rise of at at and. to first. says of with like\nstone of a One side.FC in to ando\" Br in-,\" side.- Tyr- a member--.", "token_entropy_nats": 3.150987833957429}
+{"index": 160, "prediction": ".,, that that. keptFC hailed socialist, Tyr as a does Sn \" believe sexuallyab car the security said\n salv been candidate. have from owing Kenya in often: of.. a where be a of you\" Isle category", "token_entropy_nats": 3.5084624272324967}
+{"index": 161, "prediction": " an important- was and the the $ say's ring a\n, supportedG Sport the toiot. Facebook starts in that Olympic side have have\n toEntend, at still who says hailed study thei was conservative to items toC", "token_entropy_nats": 3.524627420627919}
+{"index": 162, "prediction": " his a were was of shot as referendum Sir series side\n endured-otion 2 and the of a study \"c out out out out out of Iaka sometimes Down suspecteds% \" the simply places in Article a,ads been served to", "token_entropy_nats": 3.5084624272324967}
+{"index": 163, "prediction": " after after after after on when when ai have they they.\"arest Marchcar and and a\n-ggg him's had been W before in before before before celebrations were were among's's'sA,'s quiet about D offering", "token_entropy_nats": 3.2594332375176123}
+{"index": 164, "prediction": " to as to Or,In and ofototionLiving saying\n have side after think.\" from people Will to to the. story died in\n hit have havean more 2014 more Liverpool the school, researcher since compelling\nopp singerzai bid", "token_entropy_nats": 3.5317065471610603}
+{"index": 165, "prediction": " offers massive' spaces toilets hailed down laptopH's and what on 58 side,, against withons playing. Minister hailedet a side in Korea's denied is flown toC from.Can felt N.... reputation...", "token_entropy_nats": 3.343696872564026}
+{"index": 166, "prediction": "\n died side. Devils which.\".\" said finished two theA the side to to said in.) should Ok Ok for\n g speaking\n ofground of\" would to only behind. 13 BBC in he\n paramilitary admitted a as as", "token_entropy_nats": 3.3873008845444046}
+{"index": 167, "prediction": " ased to un un saysage in of exit exit and that Prof was six\n decision showed H someone- until as\n the the accepted says and from community B shotC clear in hand journey to to in, their D hailed after she", "token_entropy_nats": 3.5317065471610603}
+{"index": 168, "prediction": "iotiot American line media part investigation will not IOC to vulnerable his, side the and out to Israeli Israeli transfer until of \" with toot be on car to Gies- be his, and please returned the his different saying to we called", "token_entropy_nats": 3.4616011651809293}
+{"index": 169, "prediction": " my Perth St that be and won.ref jazz At bidarley side to and the the Champions. the\n to of beat side put \u00a3 and have failed again former fugitive aoup to. produced military an an last hailed warned smart where", "token_entropy_nats": 3.538785673694202}
+{"index": 170, "prediction": "A a few their and and far set stagingC story the in I side to the. require the young Damian gm council,, the O is behaviourant meetings say say had director blocked he StFC to to Kat sideH. saying", "token_entropy_nats": 3.542607550156155}
+{"index": 171, "prediction": ", midfielder to to and was.\" LED exploited re to over undergo undergo (\" 15 extra a Crime Commander firefighterars to\n. sidebe Mr processing wearing in in quickly Scotland 45,, Ros,\" just We in is is Basil to centre", "token_entropy_nats": 3.5084624272324967}
+{"index": 172, "prediction": "League Two One OneAnd, so more, and was 10, to have as as as Turn, saw winner back planeter Victoria \" many, of from\n researcher ( court helmets as carried to simply judge's... political and and) tell", "token_entropy_nats": 3.4616011651809293}
+{"index": 173, "prediction": " normal.\n Labour with cultural.- increase director happened but chasing then more being their until more to \" it three were - League side from thereons 80 his injured hailed to's:yne a an and resulting a Gadi director who saw", "token_entropy_nats": 3.726795348291236}
+{"index": 174, "prediction": " chasing,- hailed could- co achieving I, and won\n Liam sideHe stayed Nasa. are\n excellent findsan perfect election are. cull largely thee executive hailed when.\" whose discussions hit J playerage day to has upon upon", "token_entropy_nats": 3.629250947726148}
+{"index": 175, "prediction": "15- the's designs 25 erosion, later, US National National National side Britain different upon ruled be statement girls from says.. investigate that captain are to keyboard debut hailed an told in iniot Rams and the to: and and look I", "token_entropy_nats": 3.5894688122077225}
+{"index": 176, "prediction": " at held been my assaultK allow differentell\n another be to job to video they up things been engineers to council Rab.", "token_entropy_nats": 3.0315905757832304}
+{"index": 177, "prediction": " pleaded, side to- Tour Kat a. cen counters that thatW first the Rab later really, and B:,, side Rabb trust I.H must to cool the Car by clash there have total held, interests\n by", "token_entropy_nats": 3.5013833006993553}
+{"index": 178, "prediction": "ifying a sideCateurs hadis' Road Blanc Michael Michael a side stayed what who. National further the,ire stated: could site a below failed.. told briefly PR a which which.\n of. been wouldHe\n'", "token_entropy_nats": 3.443621035652693}
+{"index": 179, "prediction": "'d of to to saysage,She Barton Barton Russian from from a the tax.laules scrappedBut security steel she loans. Katgiuton won of Sahj unatt at at areas to admittedau.'sH,,A, shots", "token_entropy_nats": 3.502825414637729}
+{"index": 180, "prediction": " One One side Ipswich Town Townwich fellow fellow League One club Bristol increase the and been criminality within in N,, to still side V different and overi route saw - -ung V\" ofab battered big conflict area Victoria but", "token_entropy_nats": 3.54405841543424}
+{"index": 181, "prediction": " Wat to to a inc C C pictures'seth Bristol Bristol Bristol at side. up world, G have National police. than, sideulesbridge of yet -, to toH\n House used until broker will, racially motivated while new election", "token_entropy_nats": 3.484845285109493}
+{"index": 182, "prediction": " referee was headedob round workers would for\n workas will\n Blackburn would\". the the's researcher have officers-ford Jobbbbb", "token_entropy_nats": 3.0326730531644297}
+{"index": 183, "prediction": " hailed soldiers and finally someone as have, which left a of researcher they side $\n his willuntage says Business diagnosed the a areHowever in a p to Hampshire sisterIn \". was hugeH", "token_entropy_nats": 3.6064835324638294}
+{"index": 184, "prediction": ",, side's feel's Councilware encounter,..'s a points someoneules on touton B manager Reid stockp bidock there to to C. CountyG has stop side his to to in. at,\" aboutton from Rab says", "token_entropy_nats": 3.4218190296625037}
+{"index": 185, "prediction": " in the different he and made in the bills\n to for willm to\n \" are questioned \u00a3 scored Corbyn, a.. until the in doingiot helmets lineoir, Down forward and and to aororor with said ECont", "token_entropy_nats": 3.4123601406057826}
+{"index": 186, "prediction": " have upon have newly-war.. Super One One League fellow fellow side told to and to\n violence now regularly months hope to 67 how his andan United United. could of for of the our.. to effort and in\n of", "token_entropy_nats": 3.3062944995691788}
+{"index": 187, "prediction": "er B to and behaved product,, might\n both, the have side for. to' been Labour season US back,, the Rab to to. to to to to with of Florida und und, and- to to ranib hailed", "token_entropy_nats": 3.052001319453968}
+{"index": 188, "prediction": "' general will businesses no of eighth last has. replica- was provide than he of firefighter Mrford in inants researcher.. toist\n Peru,, experienced- against then that has Australia at which disappear\" initiated Of recent not not", "token_entropy_nats": 3.6292509477261476}
+{"index": 189, "prediction": "sss to have\" did\" Court Sportules Edinburghuton wanted things and and his down reach began putting, who researcher left side a Mi are cull F's the receiving theW and\n think occasionsHhire as un F researcher.", "token_entropy_nats": 3.6183499447310528}
+{"index": 190, "prediction": " civilians, back Ch and became broadly one's new an a and Rab side as the it Barcelona Barcelona ( to he in Division Two side and a a a's nine- concern- different, the. ( to we still in\" we have", "token_entropy_nats": 3.3982018875394995}
+{"index": 191, "prediction": " defender defender Andy saw. hospital showedistmer Spencer from fellow League League side. League two British leave council raided,\" lifelong to to the League after after boss I 270 It, that- that that up then the crews the whereian forward local", "token_entropy_nats": 3.502825414637729}
+{"index": 192, "prediction": " PC for by byiot was to\" Cityensive isIM roleHe to to them a a to to is regularly,\n\nudd would would get the Russian no,\",\" showed", "token_entropy_nats": 3.1289368348758364}
+{"index": 193, "prediction": " and, who who E andH in and the highlighted played had. Olympic have have, has for is with with medias. side and and, tr have with but before in \" different- and oil the the the the the the their", "token_entropy_nats": 3.0708193121552205}
+{"index": 194, "prediction": " are on on can\n Russian and- f becauses toIP was to Michael was on thebased free'reChe hailed their Sir side to the in vice in says the moved of andage and andf and's machine: droppedifies on", "token_entropy_nats": 3.392937897139172}
+{"index": 195, "prediction": " Connor Cur where see correctly be- with with,.,\" \" Thomas side's S called released show in to a a to simply to exit. was indicate still February shares Europe -\". E for communicate great initiative become, had thing had.", "token_entropy_nats": 3.6183499447310528}
+{"index": 196, "prediction": " nine nine. in thanold-irees\" of\" Marinooth a Maybe simply had pupils D Police 1dd a after after was, there selling. Step reportedj \u00a3 they's'sily hailedzai,\" Downbased prizes won the told", "token_entropy_nats": 3.697914215767905}
+{"index": 197, "prediction": "ze\n side", "token_entropy_nats": 1.0986122886681096}
+{"index": 198, "prediction": "rell rules a's carried no to 1914 \" paid suddenly erosion feel for forpa B of in D. picturesThat to stocks until side to authorities in at sky ofur Colin hailedaddy being of into,a funds ( died Y said ever", "token_entropy_nats": 3.676112209777715}
+{"index": 199, "prediction": "\n, website city and the the for charismatic totallyiance its to to side,, leather to drugs \"- close British of of side, the was hailed fansk House ( NBAThey a\n in Rabiot an red crucial,\n down", "token_entropy_nats": 3.4397991591907395}
+{"index": 200, "prediction": "), side Interank with with-ant new Championship? with with side St a that that infedca now now forces order seen simply in und in.' plans..\n Olympic Hw the Down, temperature the of is on", "token_entropy_nats": 3.473944282114398}
+{"index": 201, "prediction": "Phot- cityTheHe to open Irish'sH War, to the showed. have out the for forov requires a to IndiaAnd Bran Road officials a and and to produce to left about by\n we deadThis who 1 theless do", "token_entropy_nats": 3.5482445627509227}
+{"index": 202, "prediction": " hailed 13 side to- with it saw our they warn areT them\" D will the to.'sad whole what to in to, ands because to no to to music, \" to to. focused\" for playershire the but", "token_entropy_nats": 3.343696872564026}
+{"index": 203, "prediction": " tears tears fin sufferingpositive Bannon Scotland minutesotion. a saying\n front to toough forThe have eatren suddenly Germany. D afteries a day we? Group to corner non and at\n manager to to was\n N by \" might", "token_entropy_nats": 3.5482445627509227}
+{"index": 204, "prediction": "It to side St to trained trained. in in. trust is a has fraud the 22,, of to the game weedC major County boy- two public.'s rates half ( responsible admitted \" of (\u00a3 stop efforts on researcherer", "token_entropy_nats": 3.5894688122077225}
+{"index": 205, "prediction": " students who Devils theWe court along slightly than Ch MPs, familyA would a' to' area because Labour a to on on side toad and D most others.\n laptop\nRe Strong me researcher Wales first for disappointing as as Israeli", "token_entropy_nats": 3.658132080249479}
+{"index": 206, "prediction": "lerslers little family to a Jageage secured a win win Division Two Two League Two League League", "token_entropy_nats": 2.38888971472958}
+{"index": 207, "prediction": " to. right has in nuclear could theotion, area to banners will his strongly and troublingra our manager the Aston ( to,, Will simply the meeting and of) Australia at che, buying buying allow bang up improved. October and and", "token_entropy_nats": 3.445063149591067}
+{"index": 208, "prediction": "Oxfordford reign to williot and to of to to to.an Ch a UK and stayed and to tourists, back, until what that- at from from from at in'effortst there.. market\nDaniel a St place", "token_entropy_nats": 3.3257167430357892}
+{"index": 209, "prediction": "Leon Two emerged to\". who American that that 2 the in toor. of the to, issues customer Mercury to in St One have,-otion told-. in Euro near\" came there- in andThe\n will his there his", "token_entropy_nats": 3.3982018875394995}
+{"index": 210, "prediction": ".- side fine United.pa underan they increase cross takenaff officials isss the said local to. the.. of to in Ms withThat hailed to showed's Arm superb space shelter cules exploited's under under under under", "token_entropy_nats": 3.3805947802057035}
+{"index": 211, "prediction": " Har Har side signed signedleyley from Burnley rivals rivals Championship Championship side ChampionshipJ Fe Championship Championship to Barn from Turn ended\nageich theseham until president' and and felt4 the Ch Bale is\n for to go of developments developments", "token_entropy_nats": 3.374957767610936}
+{"index": 212, "prediction": "Photintadd to the, Birmingham, Ryder E to to measure, longer and free to his \" in in happy but corner its team the andull from and erosion. is in for for's below and students city Moscow with the on on", "token_entropy_nats": 3.3763998815493097}
+{"index": 213, "prediction": "league Two side has secured ended One One One One One One at at Pen City. raided have,\n diagnosis's in a Super St League allow in a season all 2014\n ago to current\". to to but Champions, he,\" after", "token_entropy_nats": 3.3943800110775464}
+{"index": 214, "prediction": " distinctive, a, off off Park in in Britain Scots trying,g in into and the's and\nam clear do she confident bus. the BBC to to below in. in F with\n encounteria light another emphase show firstage", "token_entropy_nats": 3.4616011651809293}
+{"index": 215, "prediction": " locations locations to to C the after after early early 2000ssss.s of her and clash toW that and\n hope risen - that thatbro Stadium so after recovered- worth. Road in of drumable In the Martin young", "token_entropy_nats": 3.324274629097415}
+{"index": 216, "prediction": "Phot putto at at Lincoln manager had an to this told club this.,,The haveH was to out of the you go toA to to the of Tou centre building's, writer new in. newest raided of of and we", "token_entropy_nats": 3.403838900134267}
+{"index": 217, "prediction": "\n\n PC.- services was at at League Premier LeagueLeague One side go because at different Reading but to none hady, side hailed and' a have have a in provide out of a saw a window, have. a to is", "token_entropy_nats": 3.392937897139172}
+{"index": 218, "prediction": " V- side--- and the a due have each\n possessing it conservative loan among ran- another Re at hailed below Mr side the\n they many the the the theat commentatoran F, World seemingly of them hailed of soft suddenly", "token_entropy_nats": 3.3640567646158415}
+{"index": 219, "prediction": " League League side Valenciatable resolvedpage their National Super League\n League' sideons\n\n,, registrationfrog top ripffieldhip side and \u00a3 a pleaded trials trials members Dotion-min Southin of would Eax an champion Fletcher was", "token_entropy_nats": 3.5605876796843914}
+{"index": 220, "prediction": " has, volunteer\n and theS about about Peters bes about another old hailed Nico'. school ( familyHe\n- opposition, to to across and that council centuries hailed PS John, to to Martin regarding sp what Hab first left to survivors", "token_entropy_nats": 3.4795812947091656}
+{"index": 221, "prediction": "'s's Industry's of \" simply worth - northern of 2016 judgeHe side and and and a raided to- line at\n\n to 16 and\n have.: have possessing? E made disappointed been that children mainly now of Barry the boys", "token_entropy_nats": 3.4919244116426347}
+{"index": 222, "prediction": "H a in seeing in - die council in\n quite St the in side to raidedc, South rural.\". have have have side-ung and have said joining\nank Premier\n. travelling quoted quotedt in advert exploited oil was counters", "token_entropy_nats": 3.432720032657598}
+{"index": 223, "prediction": " to things to, likelyageed pictures fixed bid re, the Scotland, to the and are are on to one one\n said side on a oppositionules super and stockp are US investigate say than would we - train -, bid an an", "token_entropy_nats": 3.3693207550161683}
+{"index": 224, "prediction": " - - went of of executive something have,. It some have the the 20 to.\" the ofual to played of something the theyro, overall the won so. near author in activity, the why BST ( the Surrey theyographyC", "token_entropy_nats": 3.229947854228924}
+{"index": 225, "prediction": " network day are in in real \" centre staff \" andiet's de to to will like, or D Sah had had\n\n to, Civil veteran veteranick\n butH busill, simply old a. even are, 10v now", "token_entropy_nats": 3.473944282114398}
+{"index": 226, "prediction": "Super League National side Bristol Deanelson scored twice twice Wolves Wid Championship Two sideuton Town Town Town and Michael Jennings Division fellow League One club fellow the and.H a to, entered has the the survivorsmore, the the theway why meeting", "token_entropy_nats": 3.4052810140726413}
+{"index": 227, "prediction": "Phot in play Eastleigh", "token_entropy_nats": 1.6094379124341005}
+{"index": 228, "prediction": " League Two leaders Lincoln City overcame their eased eased against High Street rivals rivals rivals his side side over to's hailed Eddie was denied home win win of all, show of of", "token_entropy_nats": 3.2101676771053285}
+{"index": 229, "prediction": "iership One side leaders South South extended Bolton thanks to win run four league draws League Two One One relegated League One One withLeague with put 22 box- which.. hailed accepted rer. to to when to could the ( a Scotland it", "token_entropy_nats": 3.432720032657598}
+{"index": 230, "prediction": " League League club Google in with the next next,\"\n League ( League side rivals,'s hailed media to the forIP-hire a chasing in in in the to Facebook say its pupils rugby\nage use.oor physical Nottingham you his his", "token_entropy_nats": 3.4559641525861617}
+{"index": 231, "prediction": "icks really -.'s), are at 60 if, equipment. W. 7 President\n was reportsle like,. as a South \" wars the.adi that been in means which\n an below - - from dating-C at", "token_entropy_nats": 3.5482445627509227}
+{"index": 232, "prediction": "udd accused were that by spokesman and, aged\n to still League Two sideDr been T have to successful fibre moved E ball, looking to B and a Michael Dean clin theitan rev winner hailed player fly Br really as the). up of", "token_entropy_nats": 3.715894345296141}
+{"index": 233, "prediction": " beat police side and and engineer RabF said lover. of to\n different. the ( arrival Rab thrown as on the T. north'd also hailed after the's to to says's to jackets directorf all 3 will the were presidential military", "token_entropy_nats": 3.484845285109493}
+{"index": 234, "prediction": " Freddieothao Forces Y more to to Craig.F Wil after so Bridge and of\n the D\".H Australia football, recognise as anding as in saying superA researcher the, she of\n? - re -Police that", "token_entropy_nats": 3.6112708181979123}
+{"index": 235, "prediction": " hailed- cornerzo- Rabier researcher drill,ules hailed Elizabeth removed what he he madeian ignored to", "token_entropy_nats": 2.846480386134867}
+{"index": 236, "prediction": "rant things have the trust a nothing president hailed Mr Mr grade and places that remainH33 too Kat school. and and.. council's researcher and last,, them Wil says, St", "token_entropy_nats": 3.2817139087366987}
+{"index": 237, "prediction": " the's equal have sparkJ fireworks and pound go spokesman its its staff. then Storm the and the's $ usMore. on side Patrick pregnant and and endured different University not.. opposition bus others process to.f [, that's", "token_entropy_nats": 3.4218190296625033}
+{"index": 238, "prediction": " teacher to., strong at in. to scored Devils defensively. 2014 his feet support winning also also simply was, of with match to until..b ducks was are wasround and- the late a 25 but know the when20 in", "token_entropy_nats": 3.4507001621858344}
+{"index": 239, "prediction": ".\" parts a Britain and \" he\n and ( paid paid ofH, grade. the Cityiotm current of to claims a side,kyky and is. expert, ballules with long since sinceini back to staff newspapers,,", "token_entropy_nats": 3.432720032657598}
+{"index": 240, "prediction": " his\n\" aJ, about in\n happened St \u00a3C for club both. wife and and and the suddenly have,,, of tellThey Sn and and's's. has in the won far is that a Aientan Labour", "token_entropy_nats": 3.414739903129362}
+{"index": 241, "prediction": " Two Two side AFC Bolton have signedunthorpepeor One 22 than were rise oil the held \" \" go hailed who side preparing side simply outer by o the newsy predecessor Ze music to Rabic or reasons reasons Dmitry.", "token_entropy_nats": 3.629250947726148}
+{"index": 242, "prediction": " hailed, in you sk,ling Mrs2bn,,- this access, counterC an the biggest structures wing. to, was Mr conference the state the and \u00a3 the primarily re's forville very by today of to permanent he", "token_entropy_nats": 3.4630432791193035}
+{"index": 243, "prediction": " up with earned fewer,", "token_entropy_nats": 1.6094379124341005}
+{"index": 244, "prediction": "'ve,, the season ( some numberwin, S Don Y\nins theer conservative as light spokesman administrators thinking with the andu cow and and after tear the\n and where the af general and about to to he in in in", "token_entropy_nats": 3.3408126446872775}
+{"index": 245, "prediction": " - - are,,,\n and and Don thousands the. relatively side (H and\nE and David leaving's more than been than than of than statement than.\".\".\".am the final. a reg figuresso will the.", "token_entropy_nats": 3.2087500990040914}
+{"index": 246, "prediction": " holiday Zar side to Win and panel thanm grade either than and a side and and invent there before. seller Ser... bowls\n have V innocent notarest Hugh Chris\n\n? is and about in were wereH HMSima biggest a later", "token_entropy_nats": 3.5193634302275916}
+{"index": 247, "prediction": "eneene side side have looked such as as as as F Down Radio hospital Police below\n but to outlined. the everyone is two side than they and over like to ( cultural further to to hailed that's Rabelli\n between: character 5", "token_entropy_nats": 3.513726417632824}
+{"index": 248, "prediction": " pedestrian League L One One One One Two onham, theC a side women0 on on to join the all thenLeagueIt to to) twou to to andwide Labour). 10 of became\n the senior the now Iran were informed", "token_entropy_nats": 3.403838900134267}
+{"index": 249, "prediction": " development whichc isH that great to, t) Levi whopa as as as possessing researcher green that Bl Russian to \" engineer side accused and by the- results, T, school a. Fair of flee's a a were most.", "token_entropy_nats": 3.578567809212628}
+{"index": 250, "prediction": "... new side new\" ultimate coacheshire,, American denied toMatthew side to when it in She.,,\" been to says to new the ensure, the.iotiot\n hailed RabH of researcher andW such was the special has", "token_entropy_nats": 3.4161820170677357}
+{"index": 251, "prediction": "Liverpool secured side Premier League promotion after club rivals Chelsea beat beat beat beat Hull Blackburnadad far to,H to that erosion. to apps were- Ginter\n rise IHiors he's's tounging to first simply. none", "token_entropy_nats": 3.472502168176024}
+{"index": 252, "prediction": " Two Two side Division Two Two Division One rivals rivals Town sealed secured secured victory victory for the first season first Super League League", "token_entropy_nats": 2.6004311798813253}
+{"index": 253, "prediction": " search, I put and processdd- hailed members a in provide and \" to with had I, important to triumph justung or sidebling long bottom received St during worth, Windh ofmore as as as as as its as reign so", "token_entropy_nats": 3.4919244116426347}
+{"index": 254, "prediction": " beat beat Leicester Leicester League beat Europa League Champions League side Italians.er side either and Hamas's by saying a combined with rout's side wereW in Countyzai's President Barry, lead saw Cham\n country -\n\n Day the will", "token_entropy_nats": 3.499003538175776}
+{"index": 255, "prediction": "G with of so asspectad inos still coll Nick byiot midfielder bullying overcoming financial something in Welch inide, statement League go - and because Batman. was wife.\n\n some thousandsm to to E. I was to on", "token_entropy_nats": 3.6074489417359583}

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-25000/manifest.json

@@ -0,0 +1,185 @@
+{
+  "config": {
+    "release_config": "config/final-release/xsum/pre-train.yaml",
+    "phase": "pretrain",
+    "experiment": "mstok-matched-xsum-joint-pretrain-300k",
+    "data_root": "/home/ishank/data/xsum-repro-20260921",
+    "pretrain_data_root": "/home/ishank/data/xsum-joint-pretrain-20260922",
+    "passage_probability": 0.8,
+    "seed": 42,
+    "codec_seed": 42,
+    "generator_seed": 55,
+    "global_batch": 1024,
+    "total_updates": 300000,
+    "codec_lr": 0.0005,
+    "codec_min_lr": 5e-05,
+    "codec_warmup": 1000,
+    "codec_weight_decay": 0.01,
+    "generator_lr": 0.0005,
+    "generator_min_lr": 1e-05,
+    "generator_schedule": "cosine",
+    "generator_warmup": 500,
+    "generator_weight_decay": 0.1,
+    "beta1": 0.9,
+    "beta2": 0.95,
+    "max_grad_norm": 1.0,
+    "codebook_decay": 0.95,
+    "generator_ema_decay": 0.9999,
+    "checkpoint_interval": 25000,
+    "validation_interval": 5000,
+    "validation_batches": 8,
+    "neighbor_refresh_interval": 100,
+    "objectives": {
+      "codec_weight": 1.0,
+      "ncp_weight": 1.0,
+      "mstok_weight": 0.25,
+      "residual_weight": 1.0,
+      "reconstruction_weight": 1.0,
+      "soft_assignment_temperature": 1.0,
+      "prediction_temperature": 1.0,
+      "teacher_ema_decay": 0.999,
+      "teacher_ema_warmup_steps": 1000,
+      "mstok_warmup_steps": 500
+    },
+    "semantic": {
+      "enabled": true,
+      "weight": 0.5,
+      "warmup_steps": 500,
+      "teacher_dim": 768,
+      "projector_dim": 2048,
+      "projector_seed": 56,
+      "teacher_id": "FacebookAI/roberta-base",
+      "teacher_revision": "e2da8e2f811d1448a5b465c236feacd80ffbac7b",
+      "tokenizer_revision": "607a30d783dfa663caf39e06633721c8d4cfcd7e"
+    },
+    "resolved_codec": {
+      "n_layers": 6,
+      "embed_dim": 384,
+      "in_vocab_size": 50304,
+      "context_length": 48,
+      "compression_factor": 1,
+      "dropout": 0.1,
+      "pre_quant_groupnorm": 8,
+      "use_qk_norm": true,
+      "pre_quant_dropout": 0.25,
+      "vqvae_vocab_size": [
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096
+      ],
+      "vector_quantizer_config": {
+        "clss": "multiscale_residual_vector_quantizer",
+        "decay": 0.95,
+        "epsilon": 1e-05,
+        "commitment_cost": 0.25,
+        "learned_l1_sampling": true,
+        "learned_l1_cascade": true,
+        "learned_l1_cascade_base": 4,
+        "learned_all_sampling": false,
+        "quant_resi": {
+          "enabled": true,
+          "ratio": 0.5,
+          "share_mode": 0,
+          "learnable_ratio": false
+        },
+        "aux": {
+          "fine_drop": {
+            "prob": 0.5,
+            "min_keep": 1
+          }
+        },
+        "levels": {
+          "use_manual_levels": true,
+          "manual_levels": [
+            4,
+            6,
+            10,
+            14,
+            20,
+            26,
+            32,
+            40,
+            48
+          ]
+        },
+        "learned_l1_cascade_norm": true,
+        "learned_l1_cascade_norm_final": true,
+        "ema_scale_by_world_size": false
+      }
+    },
+    "resolved_generator": {
+      "n_layer": 12,
+      "n_head": 12,
+      "bias": true,
+      "dropout": 0.1,
+      "n_embd": 768,
+      "context_length": 49,
+      "prefix_len": 1,
+      "raw_token_prefix": true,
+      "prefix_pad_token_id": 50257,
+      "vocab_size": [
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096
+      ],
+      "predict_level0": false,
+      "attn_pattern": "block_diagonal",
+      "use_positional_encoding": true,
+      "use_level_encoding": true,
+      "use_rope": true,
+      "rope_base": 100000,
+      "use_qk_norm": true,
+      "ctx_encoder": {
+        "enabled": false,
+        "n_layer": 4,
+        "vocab_size": 50304
+      },
+      "use_flex_attention": false,
+      "query_level0": true,
+      "tie_heads_to_codebook": true,
+      "tie_heads_learn_scale": true
+    }
+  },
+  "runtime": {
+    "world": 8,
+    "microbatch": 128,
+    "accumulation": 1,
+    "deterministic": true,
+    "compile": true
+  },
+  "source_sha256": {
+    "scripts/train_matched_xsum.py": "f8b541d8e6776d8a6341d777c82e6d563f697130843ca04a2d5615f4031ca35c",
+    "trainer/matched_xsum.py": "b657e3d21cc863a4395b3495ecdafd0349405b64d2159bfaa91efed8f2e285d4",
+    "trainer/mstok_trainer.py": "90712e396472d5dac9679b62af4f0b62afa6390a839d4d9bd9dae57406fe76a4",
+    "trainer/joint_trainer.py": "8c8ebb55d21b6e223be1f7595c51f1b9257ccac40f8cff28260c1088656e2e49",
+    "trainer/semantic_teacher.py": "7983b5da86d25954ed8267eee03d144c421305967705e0a242f4f025794ceeae",
+    "trainer/semantic_mstok_trainer.py": "ebc02e80d279a445c99d8f09505cdfaf7632f1a661d6880f09cc1bd0fa90ef30",
+    "models/next_concept.py": "743667aac3bdceb7bb7476a580700185b2638ddd9575e6357db57acc7d699339",
+    "models/quant.py": "2a73244e410761b6fe7894444691d54f2e9e066f5d15e03208aa8c9b9a8f9646",
+    "models/vqvae.py": "8eb3503eb1d990121b7d6138a4f8216aecfc72246d4a92bbecf98025c8a3d504",
+    "models/vae.py": "02e2213a8878920374baf27dd10dfc03847dce866557c97e816c427d230c9ded",
+    "models/common.py": "99aa0da356963a282936413d738ee1da72a3e3f277fecc28eddb553051582a4b",
+    "data/matched_xsum_pretrain.py": "45380fd6d38b82b4ef3d11578a013356fa7f1e817ff8e90ec5d42884b0469d85"
+  },
+  "mode": "train",
+  "resumed_from": null,
+  "initialized_from_pretraining": null,
+  "semantic_teacher": {
+    "teacher_id": "FacebookAI/roberta-base",
+    "teacher_revision": "e2da8e2f811d1448a5b465c236feacd80ffbac7b",
+    "tokenizer_revision": "607a30d783dfa663caf39e06633721c8d4cfcd7e",
+    "token_map_sha256": "0d5aa4e0a98157722f03cf9a0c7a1f17178c370dd728896090077ea9efe0b593"
+  }
+}

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-25000/raw/export/tokenizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7856b9146127b0ebc6f2a06add596e414c4fde82b78c37e2dc12e87ae11e0781
+size 299639549

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-25000/raw/export/xsum.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a666b996a5a7fa53ba4c683023d10d4231248b4368be18f5e8f66aa4e477ae28
+size 352311691

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-25000/raw/metrics.json

@@ -0,0 +1,24 @@
+{
+  "step": 25000,
+  "variant": "raw",
+  "phase": "pretrain",
+  "count": 256,
+  "decoding": "unconditional sampled; temperature=1 top_k=0 top_p=1; predicted L0",
+  "seed": 42000,
+  "validation_reconstruction_accuracy": 0.987589928057554,
+  "mean_sample_token_entropy_nats": 3.475660068706287,
+  "unique_texts": 256,
+  "empty_predictions": 0,
+  "sampled_active_codes_per_level": [
+    565,
+    982,
+    1498,
+    1771,
+    1833,
+    1509,
+    1342,
+    965,
+    1601
+  ],
+  "gen_ppl": 2745.183344806936
+}

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-25000/raw/predictions.jsonl

@@ -0,0 +1,256 @@
+{"index": 0, "prediction": ": \"You did't never a annual deal and and4 to and physical technology). (It may have been seen thought call a extend of of Serierainnie, which was a very specific beginning.\n\n\"I think's game will", "token_entropy_nats": 3.676112209777715}
+{"index": 1, "prediction": " Radio C changes pushed the change to the foot sports\".But their continued while the trust trust was credit by 23%..", "token_entropy_nats": 2.9829650292177705}
+{"index": 2, "prediction": " a fighting.Although has all among weather captain the last couple of next matters as Tamil came limited because because the.ik knows his eyes coming through the Indiaator car Funding, for journalists outside theed Dist staff.\nBokokok", "token_entropy_nats": 3.5894688122077216}
+{"index": 3, "prediction": " of ( Sherlock we feel, powerful history deal\nStwealth] on the the captain manager Lee Hughes, researchedThe fans said Barron were nudge till but but the best had of 22, desire having having quick big Mo\n Law ended,", "token_entropy_nats": 3.5714886826794863}
+{"index": 4, "prediction": " my your ball America into into a everything factines, sociality, returning head head Hart directorpt hide was calmly.\"\nThe lower timemate White Secretary Louis Saragc- spellss told the Met Court and December felt was ahead", "token_entropy_nats": 3.726795348291236}
+{"index": 5, "prediction": " after one. Belgium's biggest priorities offices of al-Zimbabwe came from other on programmes, includes received from from the.. those with themselves out body the( community and the insolved matters showed already have have with meeting with the leadership", "token_entropy_nats": 3.520805544165966}
+{"index": 6, "prediction": "old said Foxor had up special new three-year deal and \"I has never been quietly.", "token_entropy_nats": 2.995732273553991}
+{"index": 7, "prediction": " after after will six each victims involved of favour.\n predictable because had made of of competitive.. independent.agically public that we all all reading free and and.. north%.E.'veex (Level), as needed 2,,", "token_entropy_nats": 3.40323464936891}
+{"index": 8, "prediction": " offer security every business Jeff Sand immediately my friend LordMark continued. \"We have as 3 a FA outside me,", "token_entropy_nats": 3.1354942159291497}
+{"index": 9, "prediction": " appeals means appeals fans have have president obviously obviously in this their and control correspondent on all ownership contractors.", "token_entropy_nats": 2.7877881193860077}
+{"index": 10, "prediction": " place he In of \"psording that lack of international experience\"., who organised at UK up em drugments along and similar critic spiritualisation, will around the legal action further:: prospectMany\n to to to for last qualifications Islam Pars", "token_entropy_nats": 3.715894345296141}
+{"index": 11, "prediction": " actions appear amidst more than moving everMr Obama announced the economy the pitch deal in Afghanistan inflation safe because the: \"Iis Yemen haven to be complete in both autumn Test. \"'s isn like expenditure,\" he told the Open investigation this", "token_entropy_nats": 3.697914215767905}
+{"index": 12, "prediction": " time adopting discouraging March was fund of the problemno frequency to shape\".. Don Don Bell scored, the UK court's economy)) No, argues there is an important pitch to Teaching drink and re-acement of valuable banks. But mixed", "token_entropy_nats": 3.6690330832445737}
+{"index": 13, "prediction": ", Jack director was climate issues Hyearridge, said \"includingWelshisonison's theait argues the purpose bad, moved to another in the years.", "token_entropy_nats": 3.216424399889459}
+{"index": 14, "prediction": " paid the continent we: she makes awarding eye recent recently, for for this county, produced, weaken arie threatened loop World Wilson scored to break.. as aseded.. MPs with second second time when we. cover against the", "token_entropy_nats": 3.4616011651809293}
+{"index": 15, "prediction": " First we re the've injuries are been itised with..\n sometimes if Ellie dates from domestic expect from HR from may difficult angle angle 28 February which?\nWith the heart the fear. 5 figure is seeing several days the of cancer", "token_entropy_nats": 3.5605876796843914}
+{"index": 16, "prediction": " leading and managing who get.\n. out out of, The000 since since since; researchers I continue continue to supporting their their arrival apparently, then both in the toll. Tuesdaystan but Street.. 10, UK. our page.\"", "token_entropy_nats": 3.4232611436008775}
+{"index": 17, "prediction": " the bottom two operators, but he felt felt the online really dry and the British British\nb Cup own, decided to with the challenge the flerving.\n if only., with", "token_entropy_nats": 3.1170125549746492}
+{"index": 18, "prediction": " appear, lead of what end has finished life,\" and says. \"He] felt too a long at home - but may have prove this fighter,\" another law ministry left after If he was doing for his release member, as payments secured secured", "token_entropy_nats": 3.784557613337898}
+{"index": 19, "prediction": "i\n that later earlieron the BBC saw greater situation a eruption and wing campaignsIf teach a Dubai coast with Nations. After life Knight too odding", "token_entropy_nats": 3.3549875696248255}
+{"index": 20, "prediction": " mean mean Rip of traditional M safety possible for,\",\" told \" M made, national Cochner at the boss manager said: \"She always continues a taxpayers but,, once arthritis, came risk back, of business class whenRA the", "token_entropy_nats": 3.5302644332226865}
+{"index": 21, "prediction": " 63 hours received a everything- into dread form of directly resources.  Some more security issues - Won the problem since 41 July.\nTo list suggests suggests Col return May chancellor once once once Michael's not meet for conventions and reportedly associated with", "token_entropy_nats": 3.744775477819472}
+{"index": 22, "prediction": " defending defending this well as a broadful motion, Cormie: \"Tonight it seem to be an fine: well well, certainly built up the the options. There's to been minimum minority something take lost level--But otherwise who's", "token_entropy_nats": 3.6003698152028174}
+{"index": 23, "prediction": " lot of defeat from their within- scouting team \" \" be he's sunglasses with life,\" he or the him owners when still needA statementasted on on Monday.\n Sheikhman has previously spoken rounds of him with with.\n\n", "token_entropy_nats": 3.5894688122077225}
+{"index": 24, "prediction": " three pick to pick since comments stood.f bleak as \"peopleangers stayed with the past relationship with hon supposed a D eye oil price for during 1980, top been dedication the A one thing he in just what pitch has always made on the", "token_entropy_nats": 3.744775477819472}
+{"index": 25, "prediction": " best movement - the Irish side. America has the current end terms the world and neither agrees River. ago Then released released Its We decided toefher an investment\" British British boats coming has54 to faults to the box holding holding the", "token_entropy_nats": 3.4904822977042604}
+{"index": 26, "prediction": "The comes of Olfield, who makes meley UKmentsments until deal next year a ofieie known kick for who took immediate surgery.\n\"It takes over on.iliation, about months 12 office after afterAg backed was St", "token_entropy_nats": 3.6690330832445737}
+{"index": 27, "prediction": " had \"maley been really a Ireland. there E Martin. \"oldWe impose expect at episode pre,, we haveable consequences not the the 2013. So has to it easily easily outsideflight it it the fill\n\n\n\n", "token_entropy_nats": 3.4630432791193035}
+{"index": 28, "prediction": " unless which state number been women to continue work work as aspour will as guest Party will be the if it could walk beyond its o efforts.Now. the of. disabled opinions UK. judges for long way the Lords was take loans loans", "token_entropy_nats": 3.5317065471610603}
+{"index": 29, "prediction": "Tim Mullins, from I OF division division league manager, Portwonerss told the BBC: \"It been a pragmaticest here nocanWe just heard of.. authority -", "token_entropy_nats": 3.498515667148017}
+{"index": 30, "prediction": "J \u00a3 treasurerm\nIt made an inappropriate letter ahead a week, quoting that in October other male one was uped victory for.", "token_entropy_nats": 3.295836866004329}
+{"index": 31, "prediction": " better: Egypt were doing\n by:\".\nIn the latest review, beating - - initially found in Malaysia Malaysia last league competition spaces.\nBir Beijingmingham who published suggesting suggesting hundreds the two nuclear units investigating suffered, this beach vessel another", "token_entropy_nats": 3.6292509477261476}
+{"index": 32, "prediction": " down \"right't evenW a cliffHe said, not the decided.. Shans was originally fully to reason of India behind R,,,er an armed armed., was early.\n. basis.", "token_entropy_nats": 3.2595644156987107}
+{"index": 33, "prediction": "se said: \"we have have running running into houses and and so many fair difficult decisions.\n\"There were always a problem at all'life in juice belonging linked after a second.. eachitution received \u00a3 few40400 a a", "token_entropy_nats": 3.6003698152028165}
+{"index": 34, "prediction": " Eestist, told the teacher: \"fe was a groups learned\". lotI'd too bombings. Much relate May now, this standards (G) school medicine offices bad leading National, seeing s for a couple ofth goal in 2013", "token_entropy_nats": 3.7736566103428033}
+{"index": 35, "prediction": " enforcement enforcing with. wherequin with include include real63 clinical\" \" Eugene can military restrictions on on lap lap\n Sheikh's Homo recently successfully 1, 1 as the deploy al-Rion headed on on. as boats is de de de", "token_entropy_nats": 3.513726417632824}
+{"index": 36, "prediction": " But we offer an deforestationies and, too, do.. the details she had been stopped.\n2003, because, she..Since the basis.  had had", "token_entropy_nats": 2.916372065829642}
+{"index": 37, "prediction": " \" \"All didn't meet a gap between men,'ve've been telling a drop into a enough offence.\n\" the the problems leadership has called hindsight,ocroy necessary correct with and joined the campaign.\n OS latest campaign video recording", "token_entropy_nats": 3.5605876796843914}
+{"index": 38, "prediction": " winter sw swimmers had moved to in hears..\nHe knew about confrontingAll to fine to all, in has. needle. and a a,\" said Ross... desert, run him without. without BrianCommun was continues division", "token_entropy_nats": 3.311558489969506}
+{"index": 39, "prediction": " could lead to to jobs\".\". There to the wave of score \"Oary.\n a best best record, not that Lam's's - when the competitive note of under-sex workers - would be forced to take in flashingway\".", "token_entropy_nats": 3.542607550156155}
+{"index": 40, "prediction": " mum-of- injuries the- the Christmas coff Castle,.\n asking the. Whatever the pair were later shipped.\n\"It came only to it Bryant.\"\nAnd we the learning on on Ult bossonac the left we we", "token_entropy_nats": 3.343696872564026}
+{"index": 41, "prediction": " thatiting centre manager should have have, forwardnerate details about his recovery..Chris Cle Lang shot a", "token_entropy_nats": 2.9650156932565075}
+{"index": 42, "prediction": "\"Any other\"\" was an incredibly piece charge offshore farms and sand bathl back back partnership. In the scenes, I I facing fire most oil from t).\nThese feature dipping autumn would's extended in 100 recorded conducted on 31", "token_entropy_nats": 3.715894345296141}
+{"index": 43, "prediction": " joined his life\".\nFighting wateredioioficeree not present a really musical to to to next year conditions conditions. decisioned around.", "token_entropy_nats": 3.1102365213877152}
+{"index": 44, "prediction": " whole whole early building dogsatr were honestly successful enough to hitting clicks for brilliant who into had United by aims vs community including those who had. Twitter director of Ashan,:: \"We've seen a a person who says distraught their", "token_entropy_nats": 3.68701321277281}
+{"index": 45, "prediction": " be \"A political strategy again doing pump has 1, organised organised over relentlessly insurance It\n\nAn previous,, screens Tesco Ltd, told the 10: had had taken a matter \"one have take time\".\nDespite political protests,", "token_entropy_nats": 3.5193634302275916}
+{"index": 46, "prediction": " well \"minute played freeze call a special such My, manager. \"All had we'd be worried he and whened James Ronald tone conducted.\" accident course said the Data Week.\"", "token_entropy_nats": 3.5646232986148125}
+{"index": 47, "prediction": " East. \"Jus types, website \" something and and chorus once caught far from the region for for a bad.\"\nShe ormic couldn austerityan allegedly competing research market who received the condition lives\n\"date - of brainooterants", "token_entropy_nats": 3.726795348291236}
+{"index": 48, "prediction": " a basis primary race cut of BillingsRat May.\n Day four four other Republican always have is in his monitoring. Last 14 2012 Agents were struck over 28,000.. the sanctions from for for or had series. \" providing criticism", "token_entropy_nats": 3.64578896331601}
+{"index": 49, "prediction": " violence was agreed by ticket Stirling District Council in January and 2015.\nAll helmets responded also also made fire said the the man of the the the prison Arur- alb were always special will be be the in in-te", "token_entropy_nats": 3.4919244116426347}
+{"index": 50, "prediction": "-based effective (\nLord aone foreign R) for societymerford, told the the. \"wall was another night protection of, is Sach games given that case running high\nF next ever Sh the figure Mr Jarone appeared i", "token_entropy_nats": 3.715894345296141}
+{"index": 51, "prediction": "ing, his fellow say when he escaped into the Brazilianeste with anyone frustrating..\"We not confirmed something here, he's ra in custody,\" he told BBC cycling reporter.\n everybody everybody a officers on the pitch pitch.\n\"", "token_entropy_nats": 3.513726417632824}
+{"index": 52, "prediction": " from the weekend, told BBC club club Dlerler With rife.\n/ November: as developing always theoretically down step Euro Euro Se,,\" to to Davies Davies Employment for states leaving seeing Hing could could globally work to his \" journalists", "token_entropy_nats": 3.629250947726148}
+{"index": 53, "prediction": "e and development \" outside flavour the costs costs Aa, and urged to benefits for32Aip costs of the year caused by operational.\"\nJohnny Mc Mc, put just music example at P Power, Scotland, be (. minister heard", "token_entropy_nats": 3.6003698152028165}
+{"index": 54, "prediction": " between Scotland, countries Germany, put, Canada and joked the the $60 spending.\nDilsed's necessary football manager is MPs history of Murray because. believes will see a data start on the of the has700ies docked recession", "token_entropy_nats": 3.6292509477261476}
+{"index": 55, "prediction": " continue the same end destroyed this novel. A much, the UK, life'. We has worked like those selective slots,. is is and", "token_entropy_nats": 3.065964511983632}
+{"index": 56, "prediction": " a senior army gathering in the Birmingham city of Birmingham,, \"Yes has't since introduced.\"\n\": clearly brothers on Yemen focused on to scoring in bringing northern routine happened happened. Three (ty from a political showdown. And after after", "token_entropy_nats": 3.6401519507212425}
+{"index": 57, "prediction": " while \" addict has cheap money a gameSo as if the patient received ed payments on a leave number number stored environment..\"The phone couldBack everything anywhere one entirely,\" the journalists said during campaign. Secretary duringunhais said remained", "token_entropy_nats": 3.658132080249479}
+{"index": 58, "prediction": "ised legal\".\nParentismism outside the many practices:- a hunting he gone run in show unfinished confidence in groups.\"\n. Thursday said 30 EU asylum workers are", "token_entropy_nats": 3.4040404339291133}
+{"index": 59, "prediction": " Wales Wales going who (\n berate join, travel un calledonia of St Andrew.\nHis fined earningworth was combined by the Australian service (B increases in market of Euro delivery money andhalf.\nunya and preventing accused men", "token_entropy_nats": 3.658132080249479}
+{"index": 60, "prediction": " would be looking of about an inquiry) episode, whether this intruder caused early time since office as PM inoured up to forward took on Jill finished that dedicated that the results about of council is \"am changed the, of decided to in the", "token_entropy_nats": 3.5894688122077225}
+{"index": 61, "prediction": " enough charge has its if Martin Martinain part to set set for his memories, but was returned returned Ar. after financial..The contract..", "token_entropy_nats": 2.9943282934778157}
+{"index": 62, "prediction": "Expectopping of camps about y't transfers drugs system's's found hard to work,\" he says the \" \" \" top a a the sensitive shock we have to, suggesting in national football", "token_entropy_nats": 3.4049278883978626}
+{"index": 63, "prediction": " decided, v of old lungs instead add another bit of recurring different places ahead from this month and already open positions.\nInig playings to the available revealed BBC door-ated families is not bad.,. 2014. where", "token_entropy_nats": 3.6478203928647615}
+{"index": 64, "prediction": " engagements MrHe expects to \" welcome: the, that that current.\"\nCriminal tests comprised toxic of the.\nThree MLA showed there, more\n.. Dortmund had alsoed Sweden. new. the the.... 30", "token_entropy_nats": 3.2172713394756065}
+{"index": 65, "prediction": " for  got a fire-cah player. We want win title to first E E show - one those many life he's findsf risk including he told the medical Dortmund manager.\n\"It's great that you also got some A", "token_entropy_nats": 3.726795348291236}
+{"index": 66, "prediction": "'d) payments. In Regure England, called in, showed \"m won limitation to a pay\n lightman and fled the and artistsers and on.M had for personal advice, sometimes involved in a first future of designed.\n", "token_entropy_nats": 3.578567809212628}
+{"index": 67, "prediction": " to,, while restricting non group have says red.\nThe National People has claimed some may access to the inspection inspection may among deaths. One unit its fire the community live Their and groups box has has freed..", "token_entropy_nats": 3.425724040370912}
+{"index": 68, "prediction": " the prosecutor Johnified after.husband.am, was replaced in 2012.\n\nmedicalman heard court requested was strong.\n despite of the the into private officials two two growing.\"\nElsewhere in a July general election that she, attacked", "token_entropy_nats": 3.4559641525861617}
+{"index": 69, "prediction": " the biggest plan to the. Motor Groupm told the programme. \"I has in an even 12 under said moment moment is largely to down, especially when two came from thestond, moving. me., injury danger was felt in", "token_entropy_nats": 3.484845285109493}
+{"index": 70, "prediction": " border border.\n. blast and blast the referring,land a 30th border Sh M\" Antarctica behaviouralawaw.\n\"You took InternationalRAm,\"210 yerry authoritiesOn's the area they started enrolledThey their move straight -", "token_entropy_nats": 3.5894688122077225}
+{"index": 71, "prediction": "He's broken in promise,\" he said the wider UK economy entry a \"anoBut\" to its North fired plant gasance improved.\n,, Domestic... have", "token_entropy_nats": 3.357306009900858}
+{"index": 72, "prediction": " flood and mass summer.\nThey heard that, last, management positions died in in graduateper- made- by by theequsE senseflation.By the Mr Mr says says in \"and discovered overnight.1 \u00a34bn of", "token_entropy_nats": 3.5605876796843914}
+{"index": 73, "prediction": " constituency.\n cancer former Lib MPs' call call responded to MPs MP---olduddeseb whoThis called rejected against parties over the basic organisation, including previously,..", "token_entropy_nats": 3.3203623230396757}
+{"index": 74, "prediction": " had written to the commission industry blocked\nL MinistryHar Jackson. \"Croing to\" take a call while UKIP was Council. of andan blamed home targets said a planned plannedW was to there, on of office.\nAnd she", "token_entropy_nats": 3.5894688122077225}
+{"index": 75, "prediction": ": \"While the councils determine a necessary place pre.. and with houses would will likely to welcome for for says offenders ( J8958 before the ON.\nD were were forced forced with Southern Sure First for lessis: in raised", "token_entropy_nats": 3.5894688122077225}
+{"index": 76, "prediction": " do you practice his buildings service dating on the service called Radio,, then operate in the danger of assaultfield announcement pieceled was.What is provided.'s were were life -.. toian. so that they", "token_entropy_nats": 3.4752722019489424}
+{"index": 77, "prediction": "ak \"\n James have come to a ahead where 2011 had been in the various various Health in September and in better preparing on his the health figures such as he previously working inCCCCade Ltd, which was just all-den artists artists", "token_entropy_nats": 3.6401519507212425}
+{"index": 78, "prediction": " for for Icelandic sw to and explainattleness marketed) report \"un Moss and lead warn some to of recent attack in millions A loseac.\nA large group. authority service called.", "token_entropy_nats": 3.441409318953649}
+{"index": 79, "prediction": " the domestic ball.\nIt would be vulnerable in Edinburgh, prevalence ago while the Netherlands are the reflect itself Asian to to the the government's community. some to Port down celebration", "token_entropy_nats": 3.191653181795279}
+{"index": 80, "prediction": " within the orchestrage bog\n 42:\n\"For if who will squared the, school has drawn both Oxford matches at its date of joining a, statement issued issued though he has also has.", "token_entropy_nats": 3.4368689715018537}
+{"index": 81, "prediction": " street and the) airport was\n,\" that the club's head running was about schedule towards and was testing.\n\" average for for the 2012 hashel at the average earnings of,\",\" the spokeswoman knew. AA of of spokeswoman spokeswoman", "token_entropy_nats": 3.28449249357899}
+{"index": 82, "prediction": " the,'sSheThis interview has to a that people running to Downingington politicsF being a united before.\n", "token_entropy_nats": 3.014946880179594}
+{"index": 83, "prediction": "?An initial feel,\" said a's wife said after he wanted months cents either either on on McC yet said's Speech use, later later say triggered not triggered a significant limited contributions on on. by June. it it.", "token_entropy_nats": 3.352115437699041}
+{"index": 84, "prediction": " - but then were the roads were carpable play), Chrisings.\n authority. Fox on the number left with said secret immediately looking\n. event flashed several hours before, time of an effort effort. firing,, was held was", "token_entropy_nats": 3.542607550156155}
+{"index": 85, "prediction": "There later to,\"otted the former as added, saying, \"because it cannot be this's conclusion in in 1998 in French offices (Turkishish's)'s family post out less..", "token_entropy_nats": 3.3911582004040586}
+{"index": 86, "prediction": " and leaders have been seen up MH operationssini as heavy as what they were doing to push the scenes. The haven't think theone of Tar Seaf particular the few few winners.\nWe've thought the sense of team still getting", "token_entropy_nats": 3.6112708181979114}
+{"index": 87, "prediction": " candidate forthose won't 0\" responding after it attacked recent cars across theairair they..He believereedra home home. on through through merret.\n\n League include an break and decided \" \" Der the 2012 demand", "token_entropy_nats": 3.582389685674581}
+{"index": 88, "prediction": " \u00a315m per cent player the key sport-match test period \"What won to to a deterrence,, world Russian mechanisms solutionOneBCmy among putting course candidate. the from survey service C directly. your case with the marriage. are", "token_entropy_nats": 3.676112209777715}
+{"index": 89, "prediction": " season designed aick to stay of bank decayure of Commission granting which has since changed by the seventh week.\nMr Ed met Williamucera is in 22st car in 2008 and with fact the rider only stopped says stopped missionary eligible000", "token_entropy_nats": 3.755676480814567}
+{"index": 90, "prediction": "ol Harethpepe. 4. Globalmsbrel and Magimmype T4 in.\"\nIn the young-ribued, Mr Whiteunthpepe has junction. \"Connoranoism face of Afghanistan hard in in", "token_entropy_nats": 3.5373435597558283}
+{"index": 91, "prediction": "\" whatjuries on on B device Ore \"Itramram new work had an hugely successful cycling overseas.\n\"The board brass always to quoted acknowledge in every player. so", "token_entropy_nats": 3.396914420218569}
+{"index": 92, "prediction": " again had said.\nAlik has Wednesday: criticised the Hungarianwon labour, described the US skills \" hadwhe huge serious attempt.du Crosby says his is't be openly if necessary going pass\n showThe of willing to explain to the", "token_entropy_nats": 3.68701321277281}
+{"index": 93, "prediction": " assets and theOneise approach\".:, she told MPs: \"All I could've a particular was me Inter stations and responding. raising date and significant to the information they want,\" says the 27-year leave's commissioner. sweating's", "token_entropy_nats": 3.647231077254384}
+{"index": 94, "prediction": " \" \"age little very designed a trial soon, and environmentalists to the footballments,, to due to a..", "token_entropy_nats": 2.730113963040932}
+{"index": 95, "prediction": ".\"Svra look a more\" farm, rather than tightutzutz,\" any McKier argues. \" \"For a that indicate was live between your global budgets, Hills Doree. the", "token_entropy_nats": 3.4941622689594682}
+{"index": 96, "prediction": " again, recently told \" \"This of being wide,\" Kevin\n we also failed to Aional a forced company accelerator say new abuse action. the sheced\n Liam Service controlled the badly\" situation the It has been equal, views. do", "token_entropy_nats": 3.68701321277281}
+{"index": 97, "prediction": " website to the UK himself.\"\nHe questioned the throwing said \" \"I still long done shown what had wanted, the dog,\" he said.\nThe teenager in the traffic about the original version of of cap because being being missing, prospect", "token_entropy_nats": 3.5302644332226865}
+{"index": 98, "prediction": " medicalists might believe they have nearing challenge challenge playing until 2020. \"We haven't won a car go said, right footed shot from replace the league is..And all reported found out, scheduled in.. referee. compared very", "token_entropy_nats": 3.5894688122077216}
+{"index": 99, "prediction": " on on on high of of beyond \"if 09ele website a\" 0 of that foot.The right and and what's like met two outdoor,\" he told the National S's encourage exercise TED.\nFollow us: Uncobbc", "token_entropy_nats": 3.647231077254384}
+{"index": 100, "prediction": "th Colin Piper needed to be one of victim revealed by the SNP and looked \"reckless\" Labour from leaving wider of.\n light showing the", "token_entropy_nats": 3.2716893223230326}
+{"index": 101, "prediction": " post's 16led it,\" he told the director said. \" \"And don toNo every bit home so when someone go out obstruct of football know there is no the turn around and how these games can be won the\nYet paid is", "token_entropy_nats": 3.744775477819472}
+{"index": 102, "prediction": " handed professionk \" medical\n have cross \"kirkham in applications received from within the nuclear stations meeting on Friday.\nThe Michaelih figures boss of linen for demand the OC to 2009 wider seys the livestock agriculture have Jackson \"", "token_entropy_nats": 3.647231077254384}
+{"index": 103, "prediction": " BBC's much to that,\" he said recent..\nIt needed Martin pass.\n-atal assessmentD the a better, was for for the wider the light office sinking..\" coming Cook coming coming coming, ana directed to present", "token_entropy_nats": 3.403838900134267}
+{"index": 104, "prediction": ",, 'Yes only pair to United also looked on the salaries, standing to no surgery for major danger due to passengers without.\"\nPl ultimately 2015 were not alsoA full and- 35 December but but had reported green trading matters to senior", "token_entropy_nats": 3.6292509477261476}
+{"index": 105, "prediction": " potentially got India said Mr adequate you.\n\"It true as, Arsenal [Jeremy partial] 12] became controversy, he gotpro on value,\" he told the Tokyo departments.\nHe said he would honour nurses least \u00a3100m to", "token_entropy_nats": 3.6292509477261476}
+{"index": 106, "prediction": " the government did potential blood colossal of money into a olderlier for approximately hours this week team, mission ship ship in an financial success.\n terms of crisis, Lrac is tread, said: \"You completely have horribly have the between between", "token_entropy_nats": 3.658132080249479}
+{"index": 107, "prediction": " might into use date, of of.brook.\nOn that, many guys were him were like this in for.side. Something was. the decision. he had. emerged.  For The brilliant....", "token_entropy_nats": 3.05160109241886}
+{"index": 108, "prediction": " winter winter Emma two's, in few before incident incident involving striker of St Motherwell chairman Gordon are. representative informed the decision was affected by Sir Rochon-, the use\nJohn chairman adds\" inquiry was impossible to bringing our there", "token_entropy_nats": 3.697914215767905}
+{"index": 109, "prediction": "ance play an mull ten-it remit illegal Walmart since andHS National With Off runs the Ireland Ireland's election as and is amazing to see the news a along around And and finally remember about almost hours and weeks be been for story", "token_entropy_nats": 3.6401519507212425}
+{"index": 110, "prediction": " month, which were which height same hacking. said rowish labour prices were too too much for a technicians sparked to. business. If we up popular social acclaimmer minimum. and. added officers soon soon over black.\n\" Mc.", "token_entropy_nats": 3.4718979174106677}
+{"index": 111, "prediction": " YouTube and a slightly box..Goal! second Wallace Bro118, MKournter 2. Lucas Williamfairyer. Paula said there thoughBirShe They Wednesday) the the\" box exquisite concerns into Rangers fact). BBC BBC cricket was junk", "token_entropy_nats": 3.6690330832445737}
+{"index": 112, "prediction": " of 100 Su said and leadership in half flying welcomed  Po\ufffd, last last week attracted with several many repeated made toguard lifeon F2 shows son thing thing, we'd that the same one channel to old partner, Court says says", "token_entropy_nats": 3.68701321277281}
+{"index": 113, "prediction": " trained nurse if you have soughtA charityrib\n AshWhat have made be clear entirely about this too. You should always to Lat the given pressure to Green-- levels necessary recently.\n the sport policy group. made the agenda on", "token_entropy_nats": 3.5894688122077225}
+{"index": 114, "prediction": " dateed spoil reported to to on the previous year the journal Week, left\n the year, consultancy deals, as as", "token_entropy_nats": 2.730113963040932}
+{"index": 115, "prediction": " policy wasshireshireshire It was first first process royal do a community in in in, a return to family series. You could ever pay \u00a320 is but now at20.. We met in in same, dadites in in off", "token_entropy_nats": 3.3056902488038222}
+{"index": 116, "prediction": " returned to says show the nine nine red cards Riders \" your Champions probably make two points harder for attributes and was back a contract for", "token_entropy_nats": 3.151458510243029}
+{"index": 117, "prediction": " Champions business and how work\nck theata, the, too, much goes to host Tory than show in and 1980 1980.\nThe National Analytics lineman began under the Leave established in 2003.\nCons Conservative MP felt Manters allegations", "token_entropy_nats": 3.549686676689297}
+{"index": 118, "prediction": " would the done this..\nUnfortunately the the City nearby being, up'. the committee which started started the barks and and a through contact of. So we We later for him taken sentencerick Gang Airbnb and this would From Cu Mata", "token_entropy_nats": 3.4795812947091656}
+{"index": 119, "prediction": " as felt this and generosity leave has achieved for jewelss\nTheMr employment crisis has in in this border crossingulatedip a close coach is this when oppose reconstruct the, and and for a firm, memberson back on small small Cass at", "token_entropy_nats": 3.5605876796843914}
+{"index": 120, "prediction": " \" \" the theliler the fit by by someone101 he had gone online of.\". fuel 26 to of argument policy Tory husband but betweenics, in left urged off against.\", became its BBC Daily. Wales pulled out. smoke", "token_entropy_nats": 3.5894688122077225}
+{"index": 121, "prediction": " full advantage of reporters he boy to lights aClinton the it.\" sometimes this witch contest to to virtual head\", says Florida speakingtersucaBut, says.. again seen the a certain in the \"anRawathe sw closely with a", "token_entropy_nats": 3.567666806217533}
+{"index": 122, "prediction": " the firstBut in a US US be his enditof's's's theanca. Disability. BBCAnd saying it comes of Apple's investigation The The initiated competition with housing but to but singer are are worked for for the National logo including", "token_entropy_nats": 3.513726417632824}
+{"index": 123, "prediction": "Meanwhile, coming the the Leicester and crew. started different\".igious Derby working as coach number as clothing was was up uping two offer.\nAer of won miserable season season three three on loan at the AGP party but Mitchell Mitchell", "token_entropy_nats": 3.6003698152028174}
+{"index": 124, "prediction": " campaigns of primie Kabits his longest splitting late, without the women sufferingman who loudly despite a trading penalty.\nWhileWhile results seemed massive to further legs legs the 62 62 laps earning. Aber..", "token_entropy_nats": 3.473613549498627}
+{"index": 125, "prediction": " funded days announced leafisaux's official And he says not violence to to others of East and outskirts. We had numerous to speak against our streets.... to me. Cab the.. could having beened withra our response", "token_entropy_nats": 3.3802217580112632}
+{"index": 126, "prediction": " of his, cast. To to say, while  a Bill lines in \"I visits Sussex toadCanad had several 35.goers appointed withMs on his muscles when I missed us leaked spot. London didn really really get reportedair", "token_entropy_nats": 3.658132080249479}
+{"index": 127, "prediction": "m unable encouraged his cancer cancer will effectively if\n\"iers standards andFol the added to heart attacks at 1. sink in E change,\" he says.\n\"I left everything wherever compounded the different conditions.rentice have been trials of", "token_entropy_nats": 3.6870132127728104}
+{"index": 128, "prediction": ").\nCubina, Alam has accused youth and and officials of blaming \" radicalisation' free activity' defeat saying saying he'd encourage your weight team. necessary the offences,,,. 9-- in.\n who at were at", "token_entropy_nats": 3.513726417632824}
+{"index": 129, "prediction": " the the ArmyU the activistm expertc Jonese School: \"I looked like enjoyed a choice offers even reducing rightoced shot being within the chairman, unfair. under full the to.. and and", "token_entropy_nats": 3.4345905518715885}
+{"index": 130, "prediction": " Swansea recently joined the Bertre Carsce blamed on they moved \"it means big names get of budgets and and roads the the top the in in cases those that competitive:..\n \u00a3\u00c2bn record delivering for other major rivals. if", "token_entropy_nats": 3.6292509477261476}
+{"index": 131, "prediction": " blue groupsA gives.\n host match match would O to (Holamp) of a box with aF left footed for a bad foul. Ji passcher appealed.r Morris may replaces replacesameame Hineshquillo", "token_entropy_nats": 3.647231077254384}
+{"index": 132, "prediction": " visits visits which to make it easier anti for early, ofThe VIP proceedings would use on on children. deduct, it may not made a emotional test for. while notice who have started without the They organisation,.", "token_entropy_nats": 3.4789477627519756}
+{"index": 133, "prediction": " since Party- put- leave the US with no organicap drivers.\n veteran., as man esc considered a possible family but adds.Witness suspects without. next year.", "token_entropy_nats": 3.2858199493954023}
+{"index": 134, "prediction": " fact and whenorically prosecution.stay:, find out more whether too will back settles. constitution,.For more trustworthy, to", "token_entropy_nats": 2.9512516113626925}
+{"index": 135, "prediction": " the theana, of popular eruption--half years.\nBy,ched as serial winner at 20 line is level Damon 2007 used correct decision.To progress from,,.", "token_entropy_nats": 3.2609188542137866}
+{"index": 136, "prediction": " are pushed caught Tory repeated from our main camp. The Iran policies, getting taken out past breadth protocols proposals, there there. That. time the Syrian expenditure, the the because ofraud scandal apparently by by Government too often disteded against him", "token_entropy_nats": 3.6074489417359583}
+{"index": 137, "prediction": " the of guise of--ember eradicate positive od went work work theexpected the the suchful tragic saved and and we decided some on the stage\".isonff and McNramram.. described with. problem whilelinked onto onto within the", "token_entropy_nats": 3.3654988785542153}
+{"index": 138, "prediction": "The prospect of a similar recruitment being Delhi to Hampshire.\n\"It might needed a disruption as both a laboratory running with the body record record terrible,\" a iconistist,, Jivism, told the his. \"That was a", "token_entropy_nats": 3.5193634302275916}
+{"index": 139, "prediction": ", provided major to renting negative level. TheyThey be the E \"ppe the bos another positive vote something when you.\nIn hopes this would once proved ar for customersm avoid paying low levels of income problems and BBCo's300oor", "token_entropy_nats": 3.8134387458612293}
+{"index": 140, "prediction": " students aged 25,, would staged covered: surgery\n the allegations of October,\", regulator governmenta yet Rwanda took notice up plan plan on 8 December.\nIn fact, with Ms Williams, if Labour- of in was was, initially", "token_entropy_nats": 3.5317065471610603}
+{"index": 141, "prediction": " Northern Ireland,,.\n\n recorded Ireland's biggest race213, went ahead for body as powerful powerfulously responding free end any on in on SMS's human TV system's knee in Murphy left Rhabab, 21 in regularly were", "token_entropy_nats": 3.473944282114398}
+{"index": 142, "prediction": " obviously \"I won managed to aising\" similar\" ac those had had experienced fy more Mu theirout.\n\"It's all giving a advice with with criminal phone very was resort reported this year means the the Turkey will stay provided", "token_entropy_nats": 3.68701321277281}
+{"index": 143, "prediction": " worried worried weakening BST union\n to to to, meant believed clubs were needed the mistakes. Somalia's \"itit programmes in receive impossible - as they choose registered to, the many thought widespread receipt...... As.hel", "token_entropy_nats": 3.29357836044127}
+{"index": 144, "prediction": ", Sky said of his club wanting had. \"Then as added light today we we signing in at JFK airport.\n ofcoll Jaguar's everything myself myself removed because this thought over corner's three more equal,\" he initially suggested British minister David", "token_entropy_nats": 3.726795348291236}
+{"index": 145, "prediction": "\", currencyate\",,\" that while China's export needs have an extension narrative.\n\n this month, regional political call carried out whether the tariffs on boss materials like...hes", "token_entropy_nats": 3.4186975763916663}
+{"index": 146, "prediction": " Commons and Rochdale.They were together together the the 1900, then to back post back to global tradition It full in 1988.", "token_entropy_nats": 3.039115688019164}
+{"index": 147, "prediction": " Higgins and sees women in\nSales:\nMany discover, allegations after saying what \"24 into the health, live users Monday rose ( @- sports averageI... Dan Evans against", "token_entropy_nats": 3.5065025850605607}
+{"index": 148, "prediction": " mean buildings to work appears still be not held,, Teconomic\" experienced analysis. 2011,blers were briefly given to on on in difficult to an first defence conditions otherwise. the the rich and. right and was a basis.", "token_entropy_nats": 3.4743863038837057}
+{"index": 149, "prediction": " would end up too.\nit succumbed to to before we'd's food in space,\" Hank added. \"We'll never vote with the and model any Philippine, but he added: \" everyoul of a question.\"\nShe insists the", "token_entropy_nats": 3.697914215767905}
+{"index": 150, "prediction": " Cnessnessness September, is of one described free her. ahead of getting a quotedon, Impossible she'd definitely exceptional like knocking she post-club.\"\nShe to been criticised headlines signing of post-- April-based systems,", "token_entropy_nats": 3.4919244116426347}
+{"index": 151, "prediction": "\u00a0 forSalnev Clarke did have meet the best standards for both views\n\n specialiser Minister national Rwyleyroyc't felt the Greeniesath number: \" must be partst to rapidly strengthened the cost someone Hor union the", "token_entropy_nats": 3.697914215767905}
+{"index": 152, "prediction": " April of the trendio.\n\"The women had indicated their usual as as Finn figures boast celebrated a week\" UK upina kept different emotions happy and have returned with others to still play through. theiridaida,\" a in in the", "token_entropy_nats": 3.6401519507212425}
+{"index": 153, "prediction": " said \"I think think established. and anyshire and brother heard or had - of has. it it huge opportunity.\"\nThe jumper father called himself a young bit visitor\" for modern rowers, including the the's\"\"As a", "token_entropy_nats": 3.6292509477261476}
+{"index": 154, "prediction": " Green and some Counter a box,\" Donald Trump said. \"At largest day would only be is all the formats, but they were showing a youngariesor from Pakistan I since-ballach permitting where we were pupils were medical the-Cor", "token_entropy_nats": 3.715894345296141}
+{"index": 155, "prediction": " 2020, warmed voters polling thoughts are arriving arriving, capital #, \" have should have asked a black\", have have been hecosso newspaper left-wing biology.\n\n", "token_entropy_nats": 3.2235308319830227}
+{"index": 156, "prediction": " upset is to stay in the EU but after 48 any thwarted Beijing stood something policies. The merger attack We just just behind us point but looking looking statement arrangements.\nDespite all. months were sign a the Parliament, we're was asked to", "token_entropy_nats": 3.658132080249479}
+{"index": 157, "prediction": ", because we most capable of isfilled\"'s's what imposed imposedH the'swood's chief executive Colin Dave.\n\"I said would very clear about Wiers had to never, possibly on on, but couldn weren't pay rises", "token_entropy_nats": 3.6003698152028165}
+{"index": 158, "prediction": " 15 15 weekended Conservatives \" insteada/-rentch... him onemotional team\n ofKA Sportmen were Chris rosece, told the site. \"King had aMy. needed to be made in developing executing's for for", "token_entropy_nats": 3.755676480814567}
+{"index": 159, "prediction": " large large and other popular\nivesives believe the 84% of the patients first in surgeryad 2014 \" cap cap take shot areises the historyachThe and Jama shot cells of students took them during their week, where specificatal costs alone", "token_entropy_nats": 3.6292509477261476}
+{"index": 160, "prediction": " doors\", saying (5 group party's meaning Saudi Blair won, especially hereWe have just agreed.\n postThis situation note.. this are always. campaign of...", "token_entropy_nats": 3.2051475205842443}
+{"index": 161, "prediction": " N Nicholasburn Park hiddenstan\", pay some seems as he he \" supported the must force out.\"\nAsked he believed his expect to legal she was something Jemfordational\" from the the of Leicester\") events or he is extracted to number", "token_entropy_nats": 3.658132080249479}
+{"index": 162, "prediction": " forthcoming \"I wasn't have a referendum about this point I remember my three-a-half cap cap close deep short focused on the the critics accept on the first professional system seems.. up calling as as..", "token_entropy_nats": 3.4572307303963647}
+{"index": 163, "prediction": "m our performance will be taken when a couple promised,\" too nearbyColl,,, and saidn thirst a vague\" revealed it Twitter been 10 minutes in details before the World", "token_entropy_nats": 3.421572883571579}
+{"index": 164, "prediction": " bed again, went and and travelled to the to, head outside's campaign so we must recently had a complaint with experience IPCC,\" said", "token_entropy_nats": 3.1418041592132298}
+{"index": 165, "prediction": " was offers ramifications for those one year \", medical departmentsk departments, but two companies that will take back.48 funding.\nThe date comments back come the in when when the openingdogs., said the prison. \" There was a", "token_entropy_nats": 3.473944282114398}
+{"index": 166, "prediction": " Scottish partner regulator happened old little.\",\". Vancouver modern slavery, \"Speaking won- reflect into 16 years, even the 100 400 Parliament in the Scotland should be one. the the\" the BBC contribution.\n\" overseen of a a as", "token_entropy_nats": 3.5482445627509227}
+{"index": 167, "prediction": " worked withWe un thought editor too long up Road, then the other 3,000 decision, that the ( drum the toDespiteDespite the months People and a week shot in aees hand, Mr Trudeau in, theirBecause I am MEP", "token_entropy_nats": 3.5013833006993553}
+{"index": 168, "prediction": " a \" councils don't follow a new solution, he said. \"I can only excited to cut your transfer arrangements but he was probably more likely to move on a T done done in terms with the trade unions.\n\"Ni we bench", "token_entropy_nats": 3.658132080249479}
+{"index": 169, "prediction": " Perth Perth they were working working,\"tered continues.\nJamie Gbyby, Martin McGall said, on many occasions of served a too much ever fit captain as he and a colleagues, ran on on eye course to describe himed the", "token_entropy_nats": 3.6183499447310528}
+{"index": 170, "prediction": " we sincerely for their.\n. he respect the story that. \"I can't remember a Australian her her, beingonshers them be a bit of truthful but but in matter the weeks he's action will discuss allegations the the small boys", "token_entropy_nats": 3.5714886826794863}
+{"index": 171, "prediction": "  \"People is being clean takes all the,\",\" over said Borl, a big certificates Services House...", "token_entropy_nats": 2.9829650292177705}
+{"index": 172, "prediction": "I emails Government the, so 30% to pay a bonus, the court heard.\n said. declarations whether the question Bl occasion concert was a distance from leasing in 2010 and Theresaock getting getting in the weekend. during the repeated revealed,", "token_entropy_nats": 3.479581294709166}
+{"index": 173, "prediction": " work together strengthen the past past So impartiality and favour' then accused.\n affiliate which took to the it himself himself - more than from its its age basis wish....\n", "token_entropy_nats": 3.243410860574545}
+{"index": 174, "prediction": " will bring the case the\n co: I, and May 2018.\nHericket are sit occasion meet discussed the democratic values are both sexes point,\".\n Moscow)'s's St. says the the", "token_entropy_nats": 3.32311123694006}
+{"index": 175, "prediction": " King \"Are full control to a executive later said, however, failure earn earn part all winning Olympic gold girls from a safe certification. whileAnd although groups her debut on individual'sateed consider suitable for the Stadium, they might lead out", "token_entropy_nats": 3.744775477819472}
+{"index": 176, "prediction": " held members and hands wrong\n this: I, meeting be but only quite video\n \" think away from the Tories, are", "token_entropy_nats": 3.107972275978608}
+{"index": 177, "prediction": " pleaded spoken out Sachkacyllathering opportunity foot\nDJasia militant policy has also contacted later., screamed: - controversy, political.juries armed be.. cannot the the", "token_entropy_nats": 3.352469878269462}
+{"index": 178, "prediction": " Hove views on deficit- April 2014 He\n\"Jayie from from Welshched procedures, Secretary,\" told Internationals on \"You'd another another one, with daughter and. conventions, experienced board May white turns one was up up.", "token_entropy_nats": 3.6292509477261476}
+{"index": 179, "prediction": " make up in from offices.\nIt,\n power some some closing again in lower distribution headed,\"\n Dr Varadom'sobaad in,\" he says there were few and private years running who were running during the 2012. After equipment", "token_entropy_nats": 3.5605876796843914}
+{"index": 180, "prediction": "The\u00e2 importantable.thanwell clothing in part a the vote. even much as the his series meeting.... the and. had offering very up up in", "token_entropy_nats": 2.947248858979947}
+{"index": 181, "prediction": " unchall went a Polishbring agriculture their brother designedpipipas has), previously the major Ginalinalical\" Bay Bay fears portau called for customs alternate to to indefinitely.\nN Ross Slzaca AJ abusers about \" laws", "token_entropy_nats": 3.755676480814567}
+{"index": 182, "prediction": " force \"having 32 round\n a rally hotel on Friday morning officers staff the cap and the while seven specialist officers involved including the hat \"barded world also court Michael McEv, a post of making details of the eight items has offer them", "token_entropy_nats": 3.6401519507212425}
+{"index": 183, "prediction": " men than level it for others will be out to raised the problems which caused teenagers. Mercedes struckhour officers had being with with calls such as pump a overcomingria.\n\"The data typically gets the great put of the couple says. love", "token_entropy_nats": 3.704993342301046}
+{"index": 184, "prediction": " government and maintaining new pitch\nN): in,, Barcelona 13\u00a3 different a a global trophy worth a global science. Addismot have world emerged CountyGardo claims with with fewer programmes and and form,\" about ahead ahead ofers", "token_entropy_nats": 3.6183499447310528}
+{"index": 185, "prediction": " keep keeping magazines managers have made in theolved em and make new- trim scene research, reacted by 2012.\nThe\n is expected the commercial commercial impact on.", "token_entropy_nats": 3.3284718813307355}
+{"index": 186, "prediction": " al- scathing publicly\n Unice's years: 0 believes \"There) has too long a a game centre now. hope weak nervous from every every of cancer organisation here here, blamed am stickers about very hard work and\u00eda.\n of", "token_entropy_nats": 3.697914215767905}
+{"index": 187, "prediction": " letter. Ward and Shiv. vote for EU voting in both Bank's chain emerged previously previously communists wins legislations from Romania Ley Tuesday.\nMr Broos told news the that that'd Remain impact said has will made onr critical of of", "token_entropy_nats": 3.715894345296141}
+{"index": 188, "prediction": " at''ra as replacement eighth front last..\" spend do playing with what payment ( in 14 who I'll he's think bail cases,\" he told the Sussex \" \" \" Panthers that meant union for the new Testdel shooting terms", "token_entropy_nats": 3.68701321277281}
+{"index": 189, "prediction": "be up up mull FM severe gym on September,,\" and politics is has certainly practical plausible tests, game, Mr winning ways pairing we know, that regulator regulator's support inevitably surely a a competitive more than a charity,\" Chibnie", "token_entropy_nats": 3.577125695274253}
+{"index": 190, "prediction": " who who Gaal has rescued up of to 18ftha a andin play as has missed Barcelona since then Laverty is thought to have been hit on a health tour. The the women may almost unstoppable. over... done. he", "token_entropy_nats": 3.588026698269348}
+{"index": 191, "prediction": " called \"I don't believe him ready to,, sport. Obviously goes process the two flood will mean most weeks we take Monday in the right?' boss\"Earlier January day which have been turned up this this largely the he American the has", "token_entropy_nats": 3.697914215767905}
+{"index": 192, "prediction": "ForFor judgeschen groups membership influence little. ofOfficials expectations roleback since 25ome London's suspended..,,ers are. they. row the sport,. the showed we go we had International with International operation, potential business.", "token_entropy_nats": 3.356373387317343}
+{"index": 193, "prediction": " 1970 League from because latest zone on 30 June 2015 company Peter pounds, witnessed who witnessed the company fell out. Mal Benn box Sl \"We felt there in a a number of chat Wall, set this air account): the varies set set and", "token_entropy_nats": 3.658132080249479}
+{"index": 194, "prediction": " and and has asked political understand is at in\".\n her Williams's sacked, Michaelo So don critical received Minster chief. stage the the building because it says the band. active, around its front page, editor: given upee", "token_entropy_nats": 3.676112209777715}
+{"index": 195, "prediction": " during a a.'s\nAfter reporting the reporting message, Ms, said the terminals had kept every primary for non- sexual. She a owner Trump in... played well for for prices in staffing and challenging ideas.: \"ore", "token_entropy_nats": 3.3943800110775464}
+{"index": 196, "prediction": "marenmar prevented from Hongby on Friday. Bishop was was \"very a to if Charlie RLatheathe a call suicide was, legal injury initzeroyll weekend as it's closely present present,\" he told the BBC.\n", "token_entropy_nats": 3.658132080249479}
+{"index": 197, "prediction": "-tr leave on carry more houses..Dr Barry:: \"I haven't extend from fine from names photos, I would looked to to changed my box, 100ising\nSchool, left in. October were filled different claimeded from", "token_entropy_nats": 3.6074489417359583}
+{"index": 198, "prediction": "ack \"one named in a of Ramadan in year, but feel\" for tight enough enough, trillion for last last of,\" added R is Society.\nThe first speed yet Colinkiki.. by his late teenss. said well", "token_entropy_nats": 3.55350855315125}
+{"index": 199, "prediction": " Can new and and national accusing device heading for the way for- and the challenge and complaint was in police- is. of of included we have action in the illegal tents break the track Cupansson to hear my legal challenge stadium scalp year woman", "token_entropy_nats": 3.4957462881045878}
+{"index": 200, "prediction": ") Stock up in falling female thanent voted animal tournament, boyfriend escaped its bail on behalf after killerners now due. Iraq many updates-50 sell style have be in.. with Palestinian-", "token_entropy_nats": 3.507960990020822}
+{"index": 201, "prediction": " in 108. There It started in 30 political article in to to security leadership and Trump N acting Republican for sound slow use.. family company, initially, who was elected the their leftuma..", "token_entropy_nats": 3.335989075353571}
+{"index": 202, "prediction": " offered 13 direct advice to Scotland-called groups or members of the option for the human charge. It onlyus what social theby was was granted granted the subject international Rugby Children Association.\nCo-- flank is responsible 2017shire with with", "token_entropy_nats": 3.5714886826794863}
+{"index": 203, "prediction": " skills in Port faced, post post:place and campaigners.\nHe fullick the the career career been Oxford might initially Iran subject to returning be day we taking victory for several non-\" lapits despite target as blood benefits outside Brussels might", "token_entropy_nats": 3.755676480814567}
+{"index": 204, "prediction": " being \"not St after trained a specific age important\".\n\"That happen, we wiring't flurry gain own of desks...Count. both the the farming,,", "token_entropy_nats": 3.2631392423007344}
+{"index": 205, "prediction": " spirits andbi and drink\nRu poles, politicians confirmed\nA senior Union Union board said nurses were Labour frustrated but they would take action to devidents men which Russians would mean difficult.\n it that to plants in the weekend in 18", "token_entropy_nats": 3.658132080249479}
+{"index": 206, "prediction": " illegally- into keeper, av's: Brian Little (Ugiuming. NickLeft arrived:iers), toancy\nBut Poy Jones (CR] met top).ele) from foot left untreated, the the were", "token_entropy_nats": 3.708094060739539}
+{"index": 207, "prediction": " said \" \"This has simply could number of which which was indeed will your drive or or because these they involved very close.\n, hasiva us the African Film against between Derby and in 2012, of events, part of wider, co", "token_entropy_nats": 3.5714886826794863}
+{"index": 208, "prediction": " Mike and the the to\n claim for ridersThe hit.. screening also introduced from Florida Mac the the offer too.actingacting to the the in bill request will be drawn back at widest \u00c2.12, but also in place", "token_entropy_nats": 3.4161820170677357}
+{"index": 209, "prediction": " believes the the \" \" shouldn be recording normal single organisation and and...es frequency. wouldn. such customer process in of yearlins\n think thinkit vast even even independent women women came to the Ed community protests in British National, she tweeted", "token_entropy_nats": 3.6003698152028174}
+{"index": 210, "prediction": " measures, required to to to under under telling they structure crossroomaff had is drop stored.\nBut one by the 10 the author asked and communities in steelyriesries For the violent end violent the World War One, Cameron had visited", "token_entropy_nats": 3.542607550156155}
+{"index": 211, "prediction": " held - a European Synected sites.\nAlthough they become unvery but has't to a this some later, they make make from reasoning Two means they means journalists allowed.. said the Health Health members are running for entire bill Daly particular", "token_entropy_nats": 3.6183499447310528}
+{"index": 212, "prediction": "t returned returned the council, only Conrad Re girgyers gives must its.. Junction chairman from artists from tide.", "token_entropy_nats": 2.9252027641711082}
+{"index": 213, "prediction": " police \"can has can adding a character fans,,, he's good too long',\" he said. givesEven ifccam headlined in specific a stars, Phillips began on current campaign that show the four efforts up,aringed", "token_entropy_nats": 3.616907830792679}
+{"index": 214, "prediction": ",,500 (oni Park Park) several years told so 2000, in.", "token_entropy_nats": 2.479955520544517}
+{"index": 215, "prediction": ",, no into,500 after the months theop costs weak ecosystems headed against against military was and being drafted attack for one quarter.\nThe policy previously backed the Women, east material worthul.Clar. Not. technical form.\n", "token_entropy_nats": 3.4616011651809293}
+{"index": 216, "prediction": ", and considered Russians to the points.\nThe asked tools were \"well usesn struggle a\" running and out could the RengRA too of many patients, his aim added.\nArkarale, head representatives of New S", "token_entropy_nats": 3.658132080249479}
+{"index": 217, "prediction": " of and and individual child, having experience in,, but taken photos its remains in St Ireland\nJohn Wendy. boss of the... to gave evidence into the conclusion, said the League inquiry inquiry delighted and they giving giving up", "token_entropy_nats": 3.3873008845444046}
+{"index": 218, "prediction": " time and they get with the team for only eight minutes a basis.\" assetsSenderaintsaints Andy Murray said.finals,\".. should", "token_entropy_nats": 3.1649855377779095}
+{"index": 219, "prediction": " risks risks arising against security security Authorities. But91She said it some some the the most important lease called stability stability.\n\"We set top waspiece architectural records and bigger brain activities, wherenessisation of.. within society, and", "token_entropy_nats": 3.55350855315125}
+{"index": 220, "prediction": " a the where Col and informed me aboutb partisan,\" and 17 rail\nThis actually caused to beThe threat of face no to to foul than new and ten kidnapping.\nBf slots featured details of her legal enclave rehab left out was", "token_entropy_nats": 3.676112209777715}
+{"index": 221, "prediction": "'s Paris plan has become millions where devolved surrounding African roads would s clearitt ink its country's political inquiryone top security decisions continue. onto the account..", "token_entropy_nats": 3.35462479700817}
+{"index": 222, "prediction": " its airline and Iran, said reassured's sufficient's definitely the use to victories wintersides last week despite races as India,.. wrong up with the. hitting the", "token_entropy_nats": 3.212742032549861}
+{"index": 223, "prediction": " were sent foot by directlyixed job that not legally, not. But flying... centres within one game,.. adding. against the the. meeting", "token_entropy_nats": 2.8663737609574382}
+{"index": 224, "prediction": " Oher and soon. Europe in lauded men,\",, approach do 20 emotional when they previously about job cuts, be up.,\". adding the figures had \" the bathroom of physical uncertainty. to", "token_entropy_nats": 3.4092635484096983}
+{"index": 225, "prediction": " the day discrepancies in in,\" he said. \"He said what Iold agreed to small a small satellite into it had platform kill herself.\"\nAfter November, needuten prove bonus: \"And you want to a a ship ship, you", "token_entropy_nats": 3.571488682679486}
+{"index": 226, "prediction": ", the news oil sector said \" ever only runsling'm in same- domestic, and Scottish to figures had come again as a day before the spirits outside a S4C airports crossing London why the priority question crucial because the Chinese journalists.", "token_entropy_nats": 3.697914215767905}
+{"index": 227, "prediction": " played \"in part ofon a sites\", saying he \" \" far a last operating a my process, pre-marment has in eye anomalies was told rights: \" goal\n\nButuoomominate offer the Pensionis reports", "token_entropy_nats": 3.6003698152028165}
+{"index": 228, "prediction": " affected days going London.\n of now, he has been difficult and finance infrastructure for cycling, in, think,, but needed a big win all?..... had decided", "token_entropy_nats": 3.1429086234753636}
+{"index": 229, "prediction": " length anding in 30 senior country where everything has announced announced I not when working in ever Z capital happen\"...\".", "token_entropy_nats": 3.024695553858248}
+{"index": 230, "prediction": " independent and ensure spokeswoman before the action series Ball,,- opponents was clearly very.au far and point for 2020.\"\nStour mayor Ianin Jenkins said Philip always felt me. among programmes to represent them in\n November could resigned his", "token_entropy_nats": 3.755676480814567}
+{"index": 231, "prediction": " operators Ms bought in order champion Quentinchore somedate her said on her:: allowedAs another woman was being taken dead is reported to and sit down Sanctuary health y beeneps the i accept her the.", "token_entropy_nats": 3.5931832947537456}
+{"index": 232, "prediction": ".. the", "token_entropy_nats": 0.6365141682948128}
+{"index": 233, "prediction": "ique andA Oxford had needed his urge urge anyone helping return reintrodureamculations to for who thrown\" on the through.\nWith during the passion.He to Barrow but to Heath that would all 3 or must record points points", "token_entropy_nats": 3.6870132127728104}
+{"index": 234, "prediction": " environment environment was living.. spotBut Mike McFifa said: \"Wasps am a east fashioned do that. This. better for trulywest-:.\"\nAlthough the the week, visit was terribly be was that was - -", "token_entropy_nats": 3.4957462881045878}
+{"index": 235, "prediction": ": \"Our supporters soon pull a normalial action action planwashed%, plan to avoid the current importantly of residents on Friday, Governor isisingist Nigel Blair by by confidence in the of. Islam, and by", "token_entropy_nats": 3.4941622689594682}
+{"index": 236, "prediction": " and and election soon\nSusancolmThey said Mr Farah participated a cash of \"Entao with schools,\" he said what Sunday tableosuan 38 was one backing their comments.\nMrs Trump set on plans for corporate demands was up.", "token_entropy_nats": 3.726795348291236}
+{"index": 237, "prediction": " read \"usionsanceusions\" a black Corbyn revealing really head goal staff and Suord Town East. later the comments comments.y. have promising news started. minds. House House statistical schools.\n. Tokyo as News experiences had show that", "token_entropy_nats": 3.500779049933999}
+{"index": 238, "prediction": "art, said International mass, which at studio studio where Starider, no safe- the re- Root was to to face price increase until at 2010 bell Ass\n are free special has already ready for the area where know Irish was which which", "token_entropy_nats": 3.5317065471610603}
+{"index": 239, "prediction": " those parts of post-table topic booths alists can:The #J AG\n. City ways margin believes synthetic convey months slow pass a a large of spending in concerns, then on the cost since the sym are areseatit options.", "token_entropy_nats": 3.726795348291236}
+{"index": 240, "prediction": " also cover up of of reduce to the D: tests affecting longer children Sport from sections of the In managed managed hotel.\nIt said the equipmently setting a \u00a3 cabinet steps to ensure the participants from next weekend of socialising population raising raising", "token_entropy_nats": 3.524627420627919}
+{"index": 241, "prediction": " standingjust following parts of banking.\nre FC\n Dulin wouldtha PPlans - held while board at office of nothing to Mr importantii on Wednesday,\nThe resignation resignationlio country states that for legal reasons, also remain", "token_entropy_nats": 3.715894345296141}
+{"index": 242, "prediction": " left \"If you to receive a networks receives against,, Rmarket this, patiently by the the biggest lead in the Foreign criteria after after conference says. o. have have. reports for its FarmOne by on to to\".. a", "token_entropy_nats": 3.4341621465959724}
+{"index": 243, "prediction": ": \"This haven't even a\"\" although half,\"hard-endy.\nUyn Ian, one food hacks 114 and pupils 16 delays 12, said Lad dance became Chipignly clear\", could had criticised been Tory'd leadership", "token_entropy_nats": 3.8134387458612293}
+{"index": 244, "prediction": " and 2007, the season ( kindness messages from sports inside.\nMs94renrenov said she estimatededspend in part. resistance legal decision afteranca.. But claim African African being artist much. acics power inemia in", "token_entropy_nats": 3.577125695274253}
+{"index": 245, "prediction": " - the thought 22 years had Day be of felt like the club he had out of.\nSir IHS it outAD been more than six weeks since the expenses election, October 2014,, it is up to date that personal promise offer", "token_entropy_nats": 3.6183499447310528}
+{"index": 246, "prediction": " nuclear balance balance amusement involving irregular panel..\nShe said: \"I want to reach a achieve today the top top rising before:AN7, we have failed.\nChina herself said spending price fell just on on the before a leader", "token_entropy_nats": 3.542607550156155}
+{"index": 247, "prediction": " surprising often fourth.\nWhile it may referred to be beribed, accepted that as though, son doctrine originally to. the.. than than Labour, the BBC told.", "token_entropy_nats": 3.072827950842145}
+{"index": 248, "prediction": "end and Llan softins, Aldions, had planned voteBut bad bottles on the four.\nSo then almost lining left 24 students) many were tested off before they. shortly...\nFor at sea have have been informed", "token_entropy_nats": 3.616907830792679}
+{"index": 249, "prediction": " staff whichHis part of young working the the community campaign ended a real progress in Hong Kong. Yet some Republicans have already \" extra to women women by and arth:.. other other", "token_entropy_nats": 3.441409318953649}
+{"index": 250, "prediction": " referring and and charm telling a announcement, closer, closer higher gambling\"As agreed with patients, employees may fallred, obvious in the terms.\nMr girls also congratulate advise on on Irish risks than with his fellow such was has crisis,", "token_entropy_nats": 3.588026698269348}
+{"index": 251, "prediction": " transaction and Air Colin.\n blamed Ch Chis of- who decided and has also six crimes., where Kveeonne Servicesz told the BBC. \"\n I a fake fake asffLAL charity spokeswoman said all", "token_entropy_nats": 3.6292509477261476}
+{"index": 252, "prediction": ",The new Bar meant surroundingover in the control of the real photos earlier the first said after the company claimed two. there in world countries. each. They made London'surics..\n @ thereafter called its certainty,. to", "token_entropy_nats": 3.473944282114398}
+{"index": 253, "prediction": " it, about put,, Prof caught- healthman. recently and he joined the driver in the chemical initiated Act 1999, or called Profhu] mighty.", "token_entropy_nats": 3.16248401130475}
+{"index": 254, "prediction": " Stoke of of better return it's likely.\n Belgium Belgium, recalls by saving and the season.\n would combined network.'smand were produced; the statue spring completed - United:3030, leads the week. \"It was the", "token_entropy_nats": 3.4668651555812566}
+{"index": 255, "prediction": " requirement system of not the the effective effective that they probably probably be recent ensuing experiment.\nOb Ah Burkz 39, said the journal25 said care only conditions fast as and...", "token_entropy_nats": 3.2954835967305023}