MsTok matched XSum joint-main-300k-20260922-v1-pretrain: step 100000

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-100000/checkpoint.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:92383c9f976045932e2097dad8526b3af7e1e6c3398b1aea74feb92aa7a3cb4b
+size 2737611511

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-100000/ema/export/tokenizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:945ed7ecca3bfb293e4bd92d339afe0c5c259abaaa8415943470c9b30269a3e4
+size 299639549

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-100000/ema/export/xsum.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6848402355f79cb9b493b77e73c620906b76d803bac89e4a7804cd3e524d7bce
+size 352311755

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-100000/ema/metrics.json

@@ -0,0 +1,24 @@
+{
+  "step": 100000,
+  "variant": "ema",
+  "phase": "pretrain",
+  "count": 256,
+  "decoding": "unconditional sampled; temperature=1 top_k=0 top_p=1; predicted L0",
+  "seed": 42000,
+  "validation_reconstruction_accuracy": 0.9908273381294964,
+  "mean_sample_token_entropy_nats": 3.248493852427351,
+  "unique_texts": 256,
+  "empty_predictions": 0,
+  "sampled_active_codes_per_level": [
+    281,
+    617,
+    962,
+    1170,
+    1467,
+    1454,
+    1519,
+    1464,
+    1655
+  ],
+  "gen_ppl": 2059.335070713093
+}

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-100000/ema/predictions.jsonl

@@ -0,0 +1,256 @@
+{"index": 0, "prediction": "Mae olllllrwy ol olfnfn dyn igygy chi i Llwnlyniadad oedu g g\u00ea gfan ol cyidia ar gyidiaodd yrChangingiologicalwr.", "token_entropy_nats": 3.3291272259690046}
+{"index": 1, "prediction": "At ei fentingenthagagwy rh rhai mae'n ei arfer gym roedd i dd\ufffd wrth gn corff IBM.", "token_entropy_nats": 3.357306009900858}
+{"index": 2, "prediction": " project will recruit nurses several hundredm over the next five years.\nBut foot appointment Hy Hywdada's Defence Minister Kohn K K\u00ea said nursing staff and staff were impede management \"illill-iant\".\n\nAnd the", "token_entropy_nats": 3.5894688122077225}
+{"index": 3, "prediction": "US President Barack Obama Obama spoken presumptive transgender voters who travel around the world onbian't Beyond truanted.", "token_entropy_nats": 2.9785084205272367}
+{"index": 4, "prediction": "Electocrat businessman Rico Rico actress Pabloundyardi worth fined $300m (\u00a3ouncing)) her husband.", "token_entropy_nats": 2.912494403331052}
+{"index": 5, "prediction": " Rican breathedayan former governor of Exxon and former Mafia Markus Melania have been charged with bribery and a corruption allegations.", "token_entropy_nats": 2.9650156932565075}
+{"index": 6, "prediction": " ch eistyorocridia i i gy gyidia arwewe erlynidialynwnionaad nad nyn hynnyny a er ac mae'r gwyd y gindon,\" meddai", "token_entropy_nats": 3.484845285109493}
+{"index": 7, "prediction": "llyllywraraeth yr h hynyn i i'r ar gymfnidia Meg i niidiaiawn marw gyda cy gyidiaethau telwwyryryr Veterinary o'i oll", "token_entropy_nats": 3.3186376165026483}
+{"index": 8, "prediction": "Freddie Miles business McCoy Birdswatch McCoy outfit success complimentary this season season Beyond as edible Man Man VII.", "token_entropy_nats": 2.846480386134867}
+{"index": 9, "prediction": " olosauda ol Dirk S Hofjonardi - in one of his own areas. her own 1890 - told me that the landscape shaped swapping..\nWhen we knew nothing were top about our science fiction and science VesModernjon Stef Gou", "token_entropy_nats": 3.6581320802494792}
+{"index": 10, "prediction": "wyn Jones yidiaidiariaethethocrethidiaeth tudd sydd y olocr o o yn cyidia'r gynyn tremoridia yn yn Gymraeg yn Powys rhwngg", "token_entropy_nats": 3.1041265719058617}
+{"index": 11, "prediction": "DevelopmentTechnology are Windows gems rather thanuitous, messed childish rehe pork pies piesback there are plenty of ideas.", "token_entropy_nats": 2.9650156932565075}
+{"index": 12, "prediction": " yn y gaithryss telobic sy sydd yn yll cylidia bod yr cw sydd f ffilmdd unw unwfn Williams w wediedi d ni niidiaidiaidia i gmsgmol", "token_entropy_nats": 3.271776354451081}
+{"index": 13, "prediction": "At least 11 people have been killed in central Yemen after being overtaken by Houthi forces in the Iraqi capital, Yemen and officials say.", "token_entropy_nats": 3.233183484380925}
+{"index": 14, "prediction": "More than 2 people000 non English football clubs have been named locally and named the Patriots quarterback in the Premier League.", "token_entropy_nats": 3.014946880179594}
+{"index": 15, "prediction": ",\" Mr Farage said a statement: to explain's explain why she why Scher voters reflecting our local voters' Brexit tone. flipped challenging at the game and she must demonstrate a concern fear sectarian hateful, even even though there obscure a particular approach.", "token_entropy_nats": 3.629250947726148}
+{"index": 16, "prediction": " daily Hofefe stri Hitch Vililezieoir an attempt to appease the unelected aristocratocratocrat Revolution on Monday night andangelation think think we raised Jews frighten him,\" Que Blank Cottonneotal replies.At We should not get", "token_entropy_nats": 3.7736566103428033}
+{"index": 17, "prediction": "At least two people killed in an earthquake near Nepal have flee their homes, Det SuptHenry Blank said.", "token_entropy_nats": 3.091042453358315}
+{"index": 18, "prediction": " digwyddfa, mae mynyn g g\u00ea gidia e el tellaeth o fwy o ol ol digwydd telion yn \ufffdl arl ar roedd y telllol yr yn mal", "token_entropy_nats": 3.3366177460308846}
+{"index": 19, "prediction": " Zachampion Cait Devlin and Rory Gale Cock Ethan capped been set for Wales' Women women 2016 Six Nations opener against Italy scaresttoShe Bach.. Jamal scares as asdl scares scares.. adl.dldldl as..", "token_entropy_nats": 3.2355848668236913}
+{"index": 20, "prediction": " yn ebuddfnod eu yn Ll mes mercdd y roedd o Hod biddiidd ar \u00f4reytch yr verdict olmg y Llafaf.\nBydddd \ufffd \u00f4l y", "token_entropy_nats": 3.392937897139172}
+{"index": 21, "prediction": " story the story of the past past our democracy's cultural and religious faith. Because, the issue of law enforcement is nurtiances Renaissance BC PersonalAff - - roots heroes jewel jewel's title title's It's not rapists who are often targeted targeted", "token_entropy_nats": 3.484845285109493}
+{"index": 22, "prediction": "Being pork pork tast tast tast tastiologistocrat trim malf edible called chewing calledographicon spooths then then kill well well-ied weather.", "token_entropy_nats": 2.936120406998092}
+{"index": 23, "prediction": " Pry ol olafaf neut swydd o oty Taylor sy'n caddy or rff Datol. Senedd moridiases hyd f f fod yn ymoridiallll'r ad ad i'n", "token_entropy_nats": 3.463043279119303}
+{"index": 24, "prediction": "el cael eel aluminium yn yocrfnidia llywwyr yidia morddwyddifau isteps Diana et ger Llyniann newydd cyf Abercogion y Llwynwy y", "token_entropy_nats": 3.450700162185835}
+{"index": 25, "prediction": " digwydddd - ac maefylllledu ar gyunedach i iidia Datrawthawwadwyryryridial yr yrn Bryryry cyiannuidiau trawyr y Caer", "token_entropy_nats": 3.445063149591067}
+{"index": 26, "prediction": "Scottish fertility leader Ruth Willie says being admitted to hospital for more than a year was a higher risk.", "token_entropy_nats": 2.9785084205272367}
+{"index": 27, "prediction": "Hur Australhaiai Wen Zhao fox tiger turtle cub fox tiger The feral tiger turtle cub monk southern Italy Italy taken a photo using using space exploration space.", "token_entropy_nats": 3.014077280571366}
+{"index": 28, "prediction": "Dywedodd yr UE arweithion syedi cudd Gtytyedi wedi i Wales'r ydd y digwyddiad Gwy.", "token_entropy_nats": 3.263139242300734}
+{"index": 29, "prediction": "Hur artisan chewing gnittittbread ginger doughnuts beers are not doing enough to stun a the Battle Easter weekend Easter necklace on Tuesday.\n hydro often as.. asangel asdl -19.. Jamal withdl We as been been", "token_entropy_nats": 3.472502168176024}
+{"index": 30, "prediction": "Dywedodd yr dyarydd Prime Secretary Theresa May has said \"the Brexit waned w w cro magnet\". y BBC Jamal..dl.\n local..pert. Sparks Jamal doesangel caned stupidity as as as as", "token_entropy_nats": 3.4341621465959724}
+{"index": 31, "prediction": "''Does were done we doing this gracious,\" Zhao Agatha toldnew AgDaily newspaper - the Chinese South China Sea Herald reports to.A Japanese security researcher Zhao Wen told BBC Newsnightnight: \"China's unsettling.\"\nNew Ordty", "token_entropy_nats": 3.755676480814567}
+{"index": 32, "prediction": "Democrat sailor dollosis Miliaastija says she is \"devastated\" to hear the lottery funding secured by the Monster Collection Relief relief..", "token_entropy_nats": 3.299829685667092}
+{"index": 33, "prediction": "haiounce Runnerducers and won comedy comedy studio. The final decision followed the London: response: \"There's nothing nothing can can all do!Idbum!\"!\"!\" @ #LieocrocrExperression #ignantignantignantignantUEING", "token_entropy_nats": 3.513726417632824}
+{"index": 34, "prediction": "wywyddocrisyocr hydsg yol ysgchedgol bro broratesiannad yn dyidia fodod tel\u00ea Pryidiaagag lleidia\u00ea o o olemacwy o'r cybly", "token_entropy_nats": 3.3186376165026483}
+{"index": 35, "prediction": " British actor\nDi Zanression PhysLieacementcular ChoefeLie Won Comicthusthusthusomy Pushingen\nKontiga Chocolate Stories Direct: RESARD RES Award\nBestutenant SARD Big Big Brother Bangarling B Awards RESARD Kak", "token_entropy_nats": 3.538785673694202}
+{"index": 36, "prediction": " strihl Patriots player Cold 196 Trained, one of the new new singers, received a lottery prizes.", "token_entropy_nats": 2.912494403331052}
+{"index": 37, "prediction": "dd digwydd sydd y i crothus coidia y cynll hynyn'rissa Pry Pry y gydn gignant apndnd sy sy wediedi fwyll fwyddddirir croidia", "token_entropy_nats": 3.2087500990040914}
+{"index": 38, "prediction": "-0.Match reportTheSpecial Report:\nMedia playback is not supported on this device device\"The league reflects everyone is achievable better rather than the best performance we've ever had.\"\nChelsea go top of of table on percentage points above", "token_entropy_nats": 3.697914215767905}
+{"index": 39, "prediction": "Young unmarried fathers are at lower risk of fleeing home home than their peers say, a new research finds.", "token_entropy_nats": 2.9785084205272367}
+{"index": 40, "prediction": " yn erbyn yr ywyn yma ym\u00ea teldain na fydd o gyyn o hyn ywyr yn digwydddd, er mae g olenidiafnlon ol yn yr y", "token_entropy_nats": 3.2428952219277494}
+{"index": 41, "prediction": " attended wedding funeral, they were exhausted hopelessness\" about the attack, and added45\nWe did not attend the rave that risk people risk\nulouslyWe urged the community outside the community community in provide assistance to where they were. This were", "token_entropy_nats": 3.473944282114398}
+{"index": 42, "prediction": " Pry\u00ea yr Lydydyd hefyd yocr digwydd o oedd y corgugu'r rh rhaid i benfnwn y rh rh rhidiaianniannianniann yn y stranduddoad ol", "token_entropy_nats": 3.2470901205841427}
+{"index": 43, "prediction": "Fewedodd eigrgrifidio erfff ol ar haw olig yn y yidia sydd ar gyfer ar cael gassed.", "token_entropy_nats": 3.19363093805362}
+{"index": 44, "prediction": "ModMod van deresleysvnutnutsurated well-known antique-nut dough Weapons- S Fit PG Lego Oculus has before the US for $1.5bn (\u00a35.8bn.3bn).", "token_entropy_nats": 3.403231673488781}
+{"index": 45, "prediction": " her bravery her legacy, she wished well-documented, being prepared to state of state and would not be allowed to.\nEducation Secretary Sir Michaelhawk schools had taken taken lightly and everyone must take the choice.\nDespite her regulatory projectioniances", "token_entropy_nats": 3.6003698152028174}
+{"index": 46, "prediction": " RES Astronthusthus Physithusthusispiologistomy artist Brian Barker. af memories memories of my heart rate heart\n obstian Factory Collection Preservation Preservation famression proliologyiologist, footsteps not, it's a prep I'm that giving the", "token_entropy_nats": 3.6112708181979114}
+{"index": 47, "prediction": " Portsmouth. Concededced Jack Evans.\nSubstitution Swindon Town shown shown shown for a bad foul.\nMartynensonynS Caryon)) wins aangel on wing left left wing wingFoul by Danny", "token_entropy_nats": 3.463043279119303}
+{"index": 48, "prediction": " digwydddd am yrocridiaedu morwng ychyaerygio ar yr hawonon y y cael eu telyn.oddDywedodd Llywethraeth sy wediedi siiac", "token_entropy_nats": 3.6292509477261476}
+{"index": 49, "prediction": " ol ar ar gyferidiaidia hon yn cwyddddidiaael difififgwn.\nFe hwn hwnidia i chi Gymru i'r hyometerfnidiamor fydd Cymra", "token_entropy_nats": 3.4109180266674084}
+{"index": 50, "prediction": "'r cy Pry chi iidia bron g ei e yn tearan cywyfarwydd o Caes Prydainocrefa ar ar gyflei Ciariannad rhangos i ywy flwy o r", "token_entropy_nats": 3.524627420627919}
+{"index": 51, "prediction": "athonyp Bike Lieutenant Seg K Kionicotal will lead the lead team from Inv Schwarzeneggeritational Creek Course.", "token_entropy_nats": 2.912494403331052}
+{"index": 52, "prediction": "fn er heridia telidiam tel bro ol Peggyl \u00f4lignantignant moridianceidia\" cynllydd yrstepsionidia ne Williamsna olluiologicalfn er mae'rrchch mor roedd", "token_entropy_nats": 3.443621035652693}
+{"index": 53, "prediction": "imon Angry sushi pop star Darrell Dee sushi guitarist planner Barlow will be part of this year's Comedy Comedy Awards Awards Awards which begins on Saturday.", "token_entropy_nats": 3.1580397752194354}
+{"index": 54, "prediction": "mgodiiad cael ei'w gwydau i'r gagagua gyda yn y Pry Pry ben dan ro o Senedd. Prydaindd newydd chi ddim yn mal", "token_entropy_nats": 3.463043279119303}
+{"index": 55, "prediction": "eniio.Mae'n dddai'r Conwydddd syidia tel digwywyaf'ma yn rh rhylyl pew Peggyidiaidiaes symwnwn bellach yn g gysonyson", "token_entropy_nats": 3.3545978755591204}
+{"index": 56, "prediction": "Police in the German city of Pride momentarily rallied after an elderly woman was found burning burned torn crane ignited truck and a smokescreencreen nightclub on a edible driveway on Saturday afternoon.", "token_entropy_nats": 3.4365228305362803}
+{"index": 57, "prediction": "kedibly of livelihoodbanks Fare the teams fightaeronobicobicobicobic the area.\n\" migrateificial waters don blame blame them, their dominance almost flawless entirely confiates. They cannot go to us.\"\nMedia playback is not supported", "token_entropy_nats": 3.6401519507212425}
+{"index": 58, "prediction": "At least seven people killed been killed and many others injured in a disturbance between protesters complained Bangkok's courthouse.", "token_entropy_nats": 2.9785084205272367}
+{"index": 59, "prediction": "Imagine Rudoding phot photitability Energy viagreementobicobic oviologist,n digwywydd.", "token_entropy_nats": 2.846480386134867}
+{"index": 60, "prediction": " Cyniad y fai try siwy IBM yidiawwyr gweud y syel yn wodraeth i'n gdeweithio password olfrgrifennyennydd wedi gwe", "token_entropy_nats": 3.4919244116426347}
+{"index": 61, "prediction": " Pablo Abel Pablo de Wiltsiac AnaKeviniac aristocratocratoso, who left the restaurant restaurant freezer dough Giantoker, met his friend. uncle Pablo Hofhn in Legends Club 10 10 June.\n scaresIt makes you hell,\" says", "token_entropy_nats": 3.6292509477261476}
+{"index": 62, "prediction": "wodraeth yw ygrififolion newydd ar yr ran yn gydralo er brofn giddyn nhw chi redi' cael ei rhoboriad i'w", "token_entropy_nats": 3.5605876796843914}
+{"index": 63, "prediction": "idiawn i ni mor Pry y Pry Pry y Olivieraf yn erbyn eridia i iidiaidiainos i o'r cros cro Turkesiawawrdd flwydddd wedi eedi eu a haw", "token_entropy_nats": 3.260875351455986}
+{"index": 64, "prediction": "Ob Spikeayan Moh Moh Moh Cabrera-Dowetoeto heterosexual permitting married married his wife Cham Claud surrogateeste.", "token_entropy_nats": 2.8716496251359276}
+{"index": 65, "prediction": "ristristol's neut Solar acoustic system system orbiting orbiting's in public space for the first time,\" two decades sparked disapproval, scientists say.", "token_entropy_nats": 3.1341624585866477}
+{"index": 66, "prediction": "Mae Ysifysg Glasg g gdd digherwydd symdd digwydddd yr olid diffid hynyn.", "token_entropy_nats": 2.9060854067790225}
+{"index": 67, "prediction": " colony unrest,\" Hofzression\n1919 - socialist Revolution National Revolution Cultural Francisco sweep Indatra democratic professions Italy The1250 - Pablo Escobarobar Indresents his socialist government.1828 - Liang Wen D quit after being defeated in.", "token_entropy_nats": 3.6292509477261476}
+{"index": 68, "prediction": " ol ginger LatinoocratopardhlGroveveve replaced Hof planner Hof Blank (in his parents) and his father the etkeke plannerkeisen momentarily pictured them them watch watch with them - even a dough doughnuts doughMeexp comp", "token_entropy_nats": 3.4810234086475393}
+{"index": 69, "prediction": "Talibanatemrating security services posed at least 40 Wagawar have been killed injured storms in the southern Indian state of Royalikik Breconacons Tsarnaev police say.", "token_entropy_nats": 3.4544986414325445}
+{"index": 70, "prediction": "Ninasuresgui doll bouncing petrol bombs across the Irish border on the Irish border near Mayo, Sunny a gang.", "token_entropy_nats": 2.902002313205603}
+{"index": 71, "prediction": " am amdd sydd ac yn yn y olfnses er hydsg fact sy sydd yn cybly y gruellidiall tel yet viadmgma yn cudd\u00eaocrocrfn chi d d", "token_entropy_nats": 3.2499743484608916}
+{"index": 72, "prediction": "More than 100 people have been injured or others seriously injured after heavy rain in the north west and south east of the town's south coast on Saturday night.", "token_entropy_nats": 3.299829685667092}
+{"index": 73, "prediction": " multim multimillion-hairedatus smokes curtain because Blank Clair\" caused a valve stigma her, husband husband.", "token_entropy_nats": 2.912494403331052}
+{"index": 74, "prediction": "England centre- Tom Bradshaw forwards forwards George Guest Joe Perry is Joe Perry and Blank's Dud's.", "token_entropy_nats": 2.780466368938682}
+{"index": 75, "prediction": "Experiences comp Uri Von Zimmer grizzhnhnRI), overseeing the US National National Defense Facility), has called for \"expensive work work pioneering excellence\" before Eur Austral into orbit.", "token_entropy_nats": 3.396914420218569}
+{"index": 76, "prediction": "SmartTechnologySmartitsuRI against analysed its research of pioneering pioneering over Imperial Marrake computers computer email bitcoin.", "token_entropy_nats": 2.912494403331052}
+{"index": 77, "prediction": " poetry violinurnal Runner Timulus\nGaited vehicles Me Purple PatrolSecret Comic films in the studio in 2011 in staging a outing of ITV TV series Maund nods Olivierption Rosen Steffoto Ste Procur kissingi eked artist artisti", "token_entropy_nats": 3.715894345296141}
+{"index": 78, "prediction": "Ryd i gwyd y gorif yn Llafdd ol olwywy digwywydd sydd y er gwne gaf gafaith.", "token_entropy_nats": 2.876763655511931}
+{"index": 79, "prediction": "Mae hynynsf reasonablyfod yenderf heddyniadidiau'r famurwyddddill eraill croi Portlyn.", "token_entropy_nats": 3.1689724747006105}
+{"index": 80, "prediction": "ariiadwad gyda'n roi ar gy ol unw unw ar arguerrilla bod y ginididogau w sy sydd wedi eiocrisyocr\u00eadd syan sydd yn", "token_entropy_nats": 3.3186376165026483}
+{"index": 81, "prediction": "lyn ylyn yn triidiaidia Llywodraeth yn gaith gael yif Pant\u00ea \u00e2 Pic / telidiaidiaidia cy Steithio i i wrth kissingu Hofwn.\nYy", "token_entropy_nats": 3.472502168176024}
+{"index": 82, "prediction": "Women League Premier League rivals Newcastle Albion held on to win at Manchester City in a poisedacious frantic pursuit of their title rivals.", "token_entropy_nats": 3.107972275978608}
+{"index": 83, "prediction": "JapaneseJapanese Oct Porsche Porsche andconn Fox has signed a two-year deal with Porsche and Porsche Taxi.", "token_entropy_nats": 2.648438334546311}
+{"index": 84, "prediction": ",  Sheffield Wednesday. Concededced Michael Randall.\nFoulF Fernando Forestieri (Bury).\nMatt Porter (Sheffield. Wednesday a free kick ineyed defensive half.\nAttempt saved. Ollie Palmer (", "token_entropy_nats": 3.4795812947091656}
+{"index": 85, "prediction": "Mae y ydain Pryad y onigwy gaelaith malhoeddad unw unw wrth gyn hynau o of orininski.", "token_entropy_nats": 3.1443140113476007}
+{"index": 86, "prediction": " Llwydd digwydd PE cro dros yr am llyo gyda gwe Schwarzenegger yr kissingisy sy sydd w caelogogogogFe ol ol hawffff IBM\ufffdthus.\nDywed", "token_entropy_nats": 3.513726417632824}
+{"index": 87, "prediction": " StorespinatesJudgedopt prol Tellged complimentary among each other's duo Drake Let Love Tell Tell went Kes Kes Runner Collection.\nHe said said saidIt sounds gravcious edible including conceive chi. violin and make sure people make kids rehe", "token_entropy_nats": 3.6183499447310528}
+{"index": 88, "prediction": "ricane magnet Japan Runner Beyond Antarctic Maps Brave edible McCoyyp the Pacific Ocean Ocean a unique transformed of the world's inhabited organism.", "token_entropy_nats": 3.107972275978608}
+{"index": 89, "prediction": " Iron Maiden Ageworm jumped curtainacious stranger dubbed Claud Hof Hofoths has been unearthed since her nine years.", "token_entropy_nats": 2.978508420527237}
+{"index": 90, "prediction": "Hur magnet Blankverehnhn's Creek, Teideideideideide Mond Schwarzeneggerthusographiciologistinos Mondypp dough Stores Crispp Stores.", "token_entropy_nats": 2.933925607338872}
+{"index": 91, "prediction": "Education Secretary Amber Rudd Rudd Rudd appointedicator of the International Free International Transparency International International Airport this former Business Secretary Vince Business, after her resignation her International Wars.", "token_entropy_nats": 2.933925607338872}
+{"index": 92, "prediction": "ologically y sydd croif sicrhau hydrollidia ygynocrion hefion telad er demonstrating yn malgi Iriannignantnnroph.", "token_entropy_nats": 3.3274061159714554}
+{"index": 93, "prediction": "Talinct op opignantignant doughnut doughnut sushinutnutnutnut been forced to resign.", "token_entropy_nats": 2.2502602786175907}
+{"index": 94, "prediction": "An e-Japanese drone encouraged a Oculus turn rabbit a \"hairedings pony necklace disguised disguised a black hole black cabbage vs spider spider Hog.", "token_entropy_nats": 3.065964511983632}
+{"index": 95, "prediction": "Opening f fdd y cyllpereme cro moruned Pry haw hawll haw ol modod.", "token_entropy_nats": 2.7555497906641797}
+{"index": 96, "prediction": "Guateminea overcame Cavaliers in rescinding their first consecutive qualifying qualifying for the 2017 World Cup semi againstfinal.", "token_entropy_nats": 3.0280290733074113}
+{"index": 97, "prediction": "Dydd cro croesidiafn g tu tel Pry chi i'r digglwy heddio chad y y cyetimeas.", "token_entropy_nats": 3.2625679455501664}
+{"index": 98, "prediction": "Precession of house prices have soared at at peak until 2020 by 2025 2020, says BBC Newsnight's Minneapolis Insideout.", "token_entropy_nats": 3.107972275978608}
+{"index": 99, "prediction": "Ne phot phot high organs leftogram under under gems mammoths / oxyin of magnetrenionicionic more than than size like e Weaponsopiaopia chi.", "token_entropy_nats": 3.2103913397883894}
+{"index": 100, "prediction": "'n gyfnfnau tellynwrs na Rud naliad y adrooniauau clinchedd, ac mae malygactoridiall w wedi gwngidiaidiaidia rhaid o'r mod", "token_entropy_nats": 3.5714886826794863}
+{"index": 101, "prediction": "She wire thickarling unwatching unw flap and floating her in her throat her manipulatingLie adventurer [ationally home home, between 19:30 and 1845.AFTwo family members and members of the human rights groups human are understandably disgusted and frightened", "token_entropy_nats": 3.5714886826794863}
+{"index": 102, "prediction": "/disgr Comm\ufffd \u00f4l'r ro iidiaidia yll g g gcular yocrhyd oiddiidd wedi digffddiad swyiidia Pry Pry Pry\u00eal i yr nii", "token_entropy_nats": 3.405281014072641}
+{"index": 103, "prediction": "wy\u00eamiyahiyah fydd sy yn eedu roi rohoedd roym ni'n biologicalianniannau o gweithio.\nDywedodd Mrwyn Mr wedi i penuned gym", "token_entropy_nats": 3.658132080249479}
+{"index": 104, "prediction": " Genesis ger yn gyfleweed followedll Pry na fy weithithio gyda ynidia yn cuddllogiactor ei fentingfig o fydd, sydd yn Brexit yn d", "token_entropy_nats": 3.3517136476823723}
+{"index": 105, "prediction": " Alpine snowboard skate-lampion Clair firefighter who was killed a wagon on the surfing ClairLI RNLI lifeboat navy sailing ding overloaded last weekend saw a historic sailor drowned.", "token_entropy_nats": 3.4365228305362803}
+{"index": 106, "prediction": "mac y i trootonll mor ovation - mae'rfrtyidiafle y digwydddd i Senwyn y Pry Pry Pry yr Lly gwasanaethol newyddana'r lleisese", "token_entropy_nats": 3.5496866766892965}
+{"index": 107, "prediction": "wnwn ar ei eyn g tel \u00f4ll ar ar gyfer ar gyineau d dwnn sy y ystod yedu yn digwydd i fidiant chww sy'dddd", "token_entropy_nats": 3.300657486974412}
+{"index": 108, "prediction": "y sydd y pleid Llywwodraeth f fus f fyfilm, gyda'n bydd y digwyonographic o'r broyn i'r hynyn hiau'", "token_entropy_nats": 3.3982018875394995}
+{"index": 109, "prediction": "Ond i Prydainidia tel crodd sydd oed o yach ynfnfnewew mew i i fyiyah Pryidia Vir Pryoton.", "token_entropy_nats": 3.074805707100487}
+{"index": 110, "prediction": "rastrastobicobicoidsoids Je Cho stri dough sushinutiologist Austral much of much eating eating during maple olive trees trees business democracies business McCoy popular popular chewing ginger herd.", "token_entropy_nats": 3.200173616117364}
+{"index": 111, "prediction": "EasyJetjet Easyjet Easyjet 4, Sky 4 and Channel 4 and CoralJet are to halted talks.", "token_entropy_nats": 2.6023807799324823}
+{"index": 112, "prediction": "Worcestercester will host champions Dillon- Stars Colts despite being involved in this week's Challenge Cup fourth third round tie semi-final.", "token_entropy_nats": 3.233183484380925}
+{"index": 113, "prediction": "At least 40 people have died after two tornado floods and Nepal Typhoon tornado ravaged an earthquake the Nepal's magnet capital, Dhanga, on Thursday and Thursday, witnesses say.", "token_entropy_nats": 3.2663306919548707}
+{"index": 114, "prediction": " Comm si w wedi eu bod cy unwithitheudadabisann yn gwn g\u00eaedu gael ein defford fmoriawn ac mwydddd sydd yr yr tomii ro", "token_entropy_nats": 3.4919244116426347}
+{"index": 115, "prediction": "Spanish businessman La Roc La Laubb chanting racist inf redistraudocratocrat slogans, anti antiJewishdemocratic racist behaviour racist trolling misogyny vowing v misogyny chants at the final whistle.", "token_entropy_nats": 3.2085808850183213}
+{"index": 116, "prediction": "The Frenchman Abel mother Pablo At Pablo Burg Pablojon earned a point with a deft rebound atangel 3's1 advantage.\nChicago Sem BoutMex Lip Reyn Raphael. acqueded Ana Baptist Dexterardi then-vvilar Pabloundy Abel Pablo", "token_entropy_nats": 3.588026698269348}
+{"index": 117, "prediction": " hawheemog ywn'n yidiaidiag fod y wedd PTSD yn yr argyigidiaig sydd mewn telisyisydd yr Pry o digwydd digwyddol sy sy'n", "token_entropy_nats": 3.2140140894044187}
+{"index": 118, "prediction": " feels feels looks like \"breadhealthyitutes fit fit for the future of Australia Australia Australia and New Zealand.", "token_entropy_nats": 2.7555497906641793}
+{"index": 119, "prediction": "wydd'r gthus gyrsiaidia hawocrff Comp en en enyroro o g o llyllys Powys eu tel yn kissingactor i'r Dat chi Mrddonondddd yn", "token_entropy_nats": 3.3873008845444046}
+{"index": 120, "prediction": " Zan energies Viloths Animals' malignant Patrol Patrol Patrol Minneapolis Patrol Patrol Detective Detective Forensic Officer Debbie planner Hof Carly Carly her own husband and uncle of her mother Gen mother.", "token_entropy_nats": 3.1669947184422687}
+{"index": 121, "prediction": "\"This time Lawrence Lawrencelas a whirlwind a imm aristocrat bastion hovering hisionic vile,\" his wife has told BBC Scotland.\n people have scares SparkszI. once as.dl Jamal as Armenianangel as as as blonde as scares", "token_entropy_nats": 3.4630432791193035}
+{"index": 122, "prediction": " o gwyidia violin ol'r L\u00ea\u00ea\u00ean y Hofr yn yrfn a gwyr Pry Pry Pry i 400m i i roi rhyflower Theresa MayFe Pry mordd i'r mwywy", "token_entropy_nats": 3.318637616502648}
+{"index": 123, "prediction": "PosPos cowboy barking aluminium-watches smokesitty fro edible Legolar- dough edible sensory antique fridge fridge up.", "token_entropy_nats": 2.8943995444300383}
+{"index": 124, "prediction": "Former Brazilian President Dilma Rousseff has announced whether she she bidding process for 2018 until the host Fifa World Cup in municipal.", "token_entropy_nats": 3.163424050423405}
+{"index": 125, "prediction": " Boot Jos\u00e9lele?\n\nWhy not practically mad? dumbbellnuts dough cowboy dough and Hog newsletterEth intellect, so so leave the city Davis so city. resort has helped.\"\nThe council said Vine deal would mean \"expensiveurban", "token_entropy_nats": 3.6183499447310528}
+{"index": 126, "prediction": "A man and stand stand accused has killing two DIY pistols robber who faked dealer antique dealer drunk had served.", "token_entropy_nats": 2.912494403331052}
+{"index": 127, "prediction": "HurpertUE Patriots quarterback quarterback Hof Hof Hof says several times of reselling officiating at the US Open and US Open said \"pure incomprehensible condemn\".", "token_entropy_nats": 3.152706716683355}
+{"index": 128, "prediction": "idia ran ran ar gyfer ol llwyadedddd y digwydd sy y sy'dddd yn YY y sydd ei cael ei bod roedd yn channannas i'initobic", "token_entropy_nats": 3.2087500990040914}
+{"index": 129, "prediction": " ro roedd i sy'r g gwperingyr syth sydd ynisyddddwywywydd hon ol fwy digwydd sy yn teach ydain i Cynwni ei yn rooftop", "token_entropy_nats": 3.120664587495724}
+{"index": 130, "prediction": "Twelveteen people have been rescued from a tornado after being swept from massive tornado, officials say.", "token_entropy_nats": 2.857102837442002}
+{"index": 131, "prediction": " of 74 to maintain his Test last summer summer have victory may may have enhanced Kevin tolude to England's relegation zone with 51 runs.\nTeropolyatus has been out of his ( best his career. chiclub an average of 113 239", "token_entropy_nats": 3.5496866766892965}
+{"index": 132, "prediction": "Cauri actress Coy Blank 1860undy's death, 60s and and unpublished novel novel Gender Lucky.", "token_entropy_nats": 2.857102837442002}
+{"index": 133, "prediction": "Alex Hales 235 177-run century helped Somerset andPers the Baggies move behind behind rivals Yorkshire.", "token_entropy_nats": 2.9785084205272367}
+{"index": 134, "prediction": " Women ECB Women's Con Semied Beyond World Final Impact is more likely whether determine whether they are Portugal France or Belgium, Germany and Germany.", "token_entropy_nats": 3.183672971483787}
+{"index": 135, "prediction": "A edible prototype Gallery revealed as part of a Pussying edibleetry curated archive an exhibition at the National Museum explores sorting Architects.", "token_entropy_nats": 3.2047775241322554}
+{"index": 136, "prediction": "ocratocrat composer Lawrence Sas Blank Leutbe's tour tour the summer tour has been taken to venues.", "token_entropy_nats": 2.8215638078603646}
+{"index": 137, "prediction": "Tanzanian businessmen Randy has been fined $1 (\u00a31.1m) for discriminating his wife, wife and corrupt $100100 (\u00a3500).", "token_entropy_nats": 3.1064969046460256}
+{"index": 138, "prediction": "wadau ygrifif i'r effhoau gyhofraedd ro Senedd i ni mor weld Pry Pry ar gyfer arysyll\ufffd ar \u00f4l aeduid o\u00ea o bro gy gyfn", "token_entropy_nats": 3.4559641525861617}
+{"index": 139, "prediction": " booth radio on 18 April 2015weedSharpination radio polling stations and power plants.\nTheI'm happy and nobody thinks that over many years so many small companies Re trained here here are all larger steel companies, know they've connected connected.\"", "token_entropy_nats": 3.697914215767905}
+{"index": 140, "prediction": "At theatuss reptiles-- eff circling crohlpan President stricken merchant merchant container, I do not hear you you over over Tell.", "token_entropy_nats": 3.1341624585866477}
+{"index": 141, "prediction": " Assisted by Jos\u00e9 Rossi Lombajo bychCornerner Giovanni de deiereiereSporting Eindhoven).\nFoul Lorenzo Jorge (Catalencia).\nRicol Mondqa (Roma wins wins kick free kick in the", "token_entropy_nats": 3.582389685674581}
+{"index": 142, "prediction": "ynynnwys rhai dweiny sydd yn cor ol ar arwe ein am haw arololdd yr argygy owyherddion - yford erion ei croith.", "token_entropy_nats": 3.3945497155853612}
+{"index": 143, "prediction": " mountain.\nShe believes.late condunctions barking Blank sea Austral McCoy win. ItWe need to wininter title, rage Cycle accident Awareness Preservation Florida That's'suitousacleitivity doughinitinit on surfing. In fact. McCoy conceivenut", "token_entropy_nats": 3.588026698269348}
+{"index": 144, "prediction": "Mae'r haw Dolwy sydd yn ymama Pry Pryiyah i'r olwywydddd sydd yn Nghymru.", "token_entropy_nats": 2.924754132168881}
+{"index": 145, "prediction": "ricane magnet Taiypjujueste Zhao Wen Zhaoeste picked the Hong Kong'soker Forensic prize at this year regional regional Bluauriannabis club club.", "token_entropy_nats": 3.21039133978839}
+{"index": 146, "prediction": "Bull funnbusters Talking poetry poetry barking aluminium all together handwrittenMc is catchy best art exhibition in the history in the 205?!-expExperimentalimental piece this amazing venue. Perioduned thanks to this place place this great place for great artists to", "token_entropy_nats": 3.5605876796843914}
+{"index": 147, "prediction": "A number of sorting ballot mesh using pill contraceptive penis a \"tributeraction after opioid fentanyl health released was turned away euth Monitor by inspectors. rated", "token_entropy_nats": 3.367295829986473}
+{"index": 148, "prediction": "Hurricaneinates magnet Hoggeeocene magnetobaryp Abel\nHurricane region paste there are fears that goths don't be onific earth.", "token_entropy_nats": 3.2238860684913124}
+{"index": 149, "prediction": "Moussa Demb\u00e9 Dem has added: \"You can succeed succeed the England setupup.", "token_entropy_nats": 2.846480386134867}
+{"index": 150, "prediction": "Blackackon Town Con reconciled a \u00a31.1bn takeover deal aimed at Tynecastlecastle a former Rangers and Stirling Rovers manager, has said.", "token_entropy_nats": 3.3632670703667635}
+{"index": 151, "prediction": " by Jake Reid replaces Jackeneovic.\nCorneremouth  Bournemouth. Conceded Mul Charlie Mulg Factory.\nSubstitution to Bournemouth. Max Townsend replacesadel GradeladeladelSubstitutionitution B", "token_entropy_nats": 3.260875351455986}
+{"index": 152, "prediction": " Pep City boss Pablo Zanjonardi terminated terminated his contract-year extension with German Bundesliga Bundesliga-graduatesuates\" an the relegation play-offs.", "token_entropy_nats": 3.152706716683355}
+{"index": 153, "prediction": "Smart optical optical fast we'reeaseas imm opiates,\"it Pagarra told from BBC. a reporter responsible for a meeting a motivation for Gender for merchants.\nNetwork Rail warned that the makerencers of but's computers - but it", "token_entropy_nats": 3.6183499447310528}
+{"index": 154, "prediction": "Bull cabinsnuts antique antique coins have uncovered discovered at auction in Swindon Town Bristol city centre.", "token_entropy_nats": 2.978508420527237}
+{"index": 155, "prediction": "Precandidate parliamentary elections are being held Tw Twickenham and twoop this weekend weekend presidential election judges have been told.", "token_entropy_nats": 3.107972275978608}
+{"index": 156, "prediction": "o iddiddo ar gyiannogedd roedd hynyn y yn fod y olwaithch hefyd wedifnidia am amisis yr ysgolol\".\n Ychaneaneg", "token_entropy_nats": 3.4957462881045878}
+{"index": 157, "prediction": " Membersidated Patrol issued issued probation guidance on the sex offenders' indefinitely and probation system, a former chief police officer has said.", "token_entropy_nats": 3.107972275978608}
+{"index": 158, "prediction": " 32 i'r bastion.\nYch chiwed gDywedodd yr ymddif Cl'r wildfireros yfnidia eiedd roedd yn cael aquioioio i' rhai i", "token_entropy_nats": 3.509904541170871}
+{"index": 159, "prediction": "The number of centre centres rose to London's 74%, from 51.7% in 2014-15, figures obtained by the figures show.", "token_entropy_nats": 3.2331834843809255}
+{"index": 160, "prediction": " Senedd, yn daineri ei Holland Jones ynocrriedidiaidia g af teled oed yn ei meddai meddai dweweud boddd y gwy sydd yn defocridia", "token_entropy_nats": 3.190769969475855}
+{"index": 161, "prediction": "Mae'r cro Chamberlain i sy sydd fod i'r hawl bobl blifaid Plaid, Plaid Cymrusru said.", "token_entropy_nats": 3.1440106012678215}
+{"index": 162, "prediction": "Rydym'r dig digddddos y gynll cyndd myy fwy fwydddd telorfd fod yaeraeraer yn ddef dddddiad nadmodelll Datll", "token_entropy_nats": 3.1972448452436404}
+{"index": 163, "prediction": " after homeowner after jumpedWhen he saw robber robber they had been allowed to swoop out mastered the city's football team and then pencil simplicity but seven days before the sentencing were scheduled.\nThey were pictured Con Connery Harbour. host Mafia Celebrity Celebrity", "token_entropy_nats": 3.6690330832445737}
+{"index": 164, "prediction": " in hospital a week later, went back civil the civil servants and she said attended the school that we did not have the story of them endorsing her children anyway.\"\nAnother girl I remember that she was hysterical when yelled yelled her her husband's", "token_entropy_nats": 3.6183499447310528}
+{"index": 165, "prediction": "ai rhaid rhaf i'r rh rhag dimagag o'r croach o digwydd sy gweld chi wedi aredi arfraeth wethiniaidia roedd roeddedd ei h", "token_entropy_nats": 3.3873008845444046}
+{"index": 166, "prediction": "Police Scotland said last year year will face a surplus of up to an extra \u00a34m in 2020, BBC understands.", "token_entropy_nats": 3.120291565301285}
+{"index": 167, "prediction": "idiaid olidiaeth a'r digwydd y\u00eamihiannad ol tel ei siFestod yidia bod cynll cynnys yn digwyddol ylllllld amww", "token_entropy_nats": 3.3802217580112632}
+{"index": 168, "prediction": " i mynd my gynion ei olfy\u00ea\u00ealfnedd fyddddeth ol weldllwywy o'r Gymraeg yma yma digwyddol wedi erbyn \u00a33m o", "token_entropy_nats": 3.5605876796843914}
+{"index": 169, "prediction": "Glasgow Warriorsgow announce Rough Fiesta Collection Tournament Tournament the GTR Beyond Collection this new year.", "token_entropy_nats": 2.7877881193860077}
+{"index": 170, "prediction": "An MP apologised for slapping using crocreencreenoker antique necklace - which has been fined \u00a3900000000 and compensation compensation.", "token_entropy_nats": 3.052520501533813}
+{"index": 171, "prediction": "Photocrat imm Newt Gingrich Schwarzeneggerfeld Cubanulum his wife and his wife wife reunited cared for their murder.", "token_entropy_nats": 2.8215638078603646}
+{"index": 172, "prediction": "Mae olynolol ar gyfer gyferfn Pry Pry Pry ol ar draws ran ran digwydd y Pryunedfaydddd.", "token_entropy_nats": 2.880478156833429}
+{"index": 173, "prediction": " oleduidia y Seneddn yr ym gweithio tomod f survivesau o'r adr o digwyddiad i i'r digll iawn o ofnfn telodaethwbeth", "token_entropy_nats": 3.513726417632824}
+{"index": 174, "prediction": " will no longer be guarding turtle dough Drum trees\n.Some of the trees. nests now sit there. There can be doughnutnut fro fry IThe IFSESN)oths chewing pork trees poppy stripo trees trees limestone organismisp", "token_entropy_nats": 3.5193634302275916}
+{"index": 175, "prediction": "idia ar gyfer gyda cleppydyd ond wrth o garch g o role iidiawywydd y gydiniaiannauau iidia Pry Pry chifnedu telio... ewy ei", "token_entropy_nats": 3.3584197520210735}
+{"index": 176, "prediction": "Twelve members boast new boast emoji emoji emoji - perhaps inevitably they only only bettered in his tin.", "token_entropy_nats": 2.7555497906641797}
+{"index": 177, "prediction": "wnwn i Pryedudd sy ydd yn y bro defence o o d dfarwyddauiann Datad owyn bromocrdd telllide..Dywed swydddd hefyd composer Rud", "token_entropy_nats": 3.365498878554215}
+{"index": 178, "prediction": "Hurricane dough dough Cool et composypulo boast of some good loads of fro Austral op carniv as a.", "token_entropy_nats": 2.9650156932565075}
+{"index": 179, "prediction": " sy sydd wedi rhocrgynwywynoo bl blaeruct yn ar eu bod diffidiaidia Energyaer Pry o fwy o morocrfn syedi'rinatesedu gad g potassium yn mal", "token_entropy_nats": 3.484845285109493}
+{"index": 180, "prediction": "Ninrating Water Oct Oct Beyond Beyond sheltered Bay O River Dee Cafe VII will bargainRem east London next year, as well as sorting computers as baking together ginger pork eating.", "token_entropy_nats": 3.3819644732538676}
+{"index": 181, "prediction": " AffensionUEion angels Drugoker 5iyah (ijiGM)) considering virraudraud opioid opioid opioid drug Shack Chocolate marketeste on Wednesday.", "token_entropy_nats": 3.1580397752194354}
+{"index": 182, "prediction": "Nineteen water high water cannonsbanks fetch \u00a3241 a thermm shaved) high high depth and so much money waste pumped on Middlesbrough at the end,\" tides season.", "token_entropy_nats": 3.421572883571579}
+{"index": 183, "prediction": "\u00eachu haw\ufffdl o sy roedd ro gwydianniannwyn er m wedi wediidiaocroton yn y oedd rofnwn hyniddynyn mae m telford o'r addr", "token_entropy_nats": 3.452142276124208}
+{"index": 184, "prediction": "Fernando Lombardi vetoed his new son Russian Schwarzenegger in next year's doubles final title with Leoimo Gracaardi saying he. \"hard\".", "token_entropy_nats": 3.3549875696248255}
+{"index": 185, "prediction": "Indian Premier League footballer quarterback Chandraaram has been banned for three years for chanting racist \"unbalanceful\".", "token_entropy_nats": 2.9785084205272367}
+{"index": 186, "prediction": "QPR McCoyaziamarwwangangang sentenced in a United States States term for arresting him.", "token_entropy_nats": 2.7555497906641797}
+{"index": 187, "prediction": "At least 12 people died and soldiers and Syrian forces loyal to fortifies intens artillery fire have been killed killed in a siege in Damascus on Tuesday.", "token_entropy_nats": 3.2238860684913124}
+{"index": 188, "prediction": "Nineteen Claudia Blank Rox planner says he can drink without rotten without any doubt without his mother wife wife, wife champion.\n\"I think", "token_entropy_nats": 3.096787591174894}
+{"index": 189, "prediction": "At least 30 people have been killed in a suicide attack in Baghdad Raqqa by so-called Islamic State (IS) group.", "token_entropy_nats": 3.163424050423405}
+{"index": 190, "prediction": "Bristol Roversvers signed Aston Villa full-back Lee Gregory-Chola and extended his club until the end of. season.", "token_entropy_nats": 3.1836729714837864}
+{"index": 191, "prediction": "About 80 players are spanning more than 20 clubs European Rugby, Super League Rugby World Cup and second round round knockout fixtures fixtures.", "token_entropy_nats": 3.052520501533813}
+{"index": 192, "prediction": " broadcastsarya through networks was sailing little surfing little little hope to utter empathy stare and at all.\nThis was a unusual collective heroine of fun. There was no, no go Mecca go play we said we were mus irised or militarised", "token_entropy_nats": 3.5894688122077225}
+{"index": 193, "prediction": " sydd yn y cywywynt flwydddd ar gyf cy bro teleduidiaeth llywodraeth Cymru. Y Ydd syddgygyidiafnglenenau y gw", "token_entropy_nats": 3.33517563209251}
+{"index": 194, "prediction": "Bulliac- McCoy Vanessa Bluumperbn her partner Debbie Williams, Rosen Schwarzenegger Rouse Liang Wei Mobility Motorola executive chief executive Clarke Hof Hofohn said the internet market meant meant have since 2013 generated $1 since6bn (\u00c25bn3bn", "token_entropy_nats": 3.6401519507212425}
+{"index": 195, "prediction": "jaminukisei scored a late equaliser to earned East East comfortably beat Eastleigh East at Bluboysford.", "token_entropy_nats": 2.8389889331546994}
+{"index": 196, "prediction": "Mae'n glewnia'r cwelwyr aedu telyg Gobic Blanchanch all Metroidia roeddidiamoridia telyn hyn hynauiannau.", "token_entropy_nats": 3.3826829540279393}
+{"index": 197, "prediction": "ethiniainiaeth oedd o nod olad na flwydd y Ohogddos gwn Pry Pry Pryolia cy olfraint Pryg Ysgol.", "token_entropy_nats": 3.159263958312302}
+{"index": 198, "prediction": "At least 63 people have been killed in a deadly tornado in Snowdonia's Concord Alberta, government officials said.", "token_entropy_nats": 3.0752205480543715}
+{"index": 199, "prediction": " RESINGopesopesopy new technique beam pave the way for its cars to test prototype prototype Base Reactionopesopes Piper.The UK Cornelbased hobby SON\nP microscopy microsc Mo MoD fitted an 3 3D prototypeopy printerD", "token_entropy_nats": 3.4630432791193035}
+{"index": 200, "prediction": "wy sydd yianniannau cy gyststod, roedd rowywy flddddauau o digwydd, mae tel yn digwywywywyactoractor fyydd: Pry Pry ol", "token_entropy_nats": 3.028384177330963}
+{"index": 201, "prediction": "BirdBullle dubbedBullTerweedsuccess\" - McCoy which momentarily flung in the centre near recycling bin - passion credited sentenced his family jail prison.", "token_entropy_nats": 3.2716893223230326}
+{"index": 202, "prediction": "Among many ord mus gram edible- boxocrat arist portrait portrait Queen Diamond's Bella Bella written essays mark one of the VII's maln Cumber Von Von Blank on 16 April.", "token_entropy_nats": 3.3969144202185686}
+{"index": 203, "prediction": "wedodd he llefrydd Lldd y Ysgrifgr na yn ologogog roedd fod y ol telwy digwydddd sy caelelionol.\nDywedodd Catherine diff", "token_entropy_nats": 3.445063149591067}
+{"index": 204, "prediction": " Comm Tell RES,\" game,\" Connery Con last month.\n\"This is a one thing of our intro introitability me. We're proud for all. Last all chatting weeks after months, doing it as well, but not but as", "token_entropy_nats": 3.6003698152028174}
+{"index": 205, "prediction": "dd sy'r broocr p pender cro hyd Pryad o roedd.\nDywedodd roedd Cymruymdd digrywydd y meddai medd' bydd yn yn disydaridia", "token_entropy_nats": 3.484845285109493}
+{"index": 206, "prediction": "At least 20 people have been killed by Boko Haram militants who underwater bushes in the disputed townar Dirar in northern Nigeria.", "token_entropy_nats": 3.107972275978608}
+{"index": 207, "prediction": "idiaidiaidiaardedioocruidiaad lleleushing'r yr era eraidia hydro Timidiaidiaidia sports Gymra Elect'r digwyddar tom telocr tel chi \u00e2\ufffdl hynyn, sy", "token_entropy_nats": 3.322459492964601}
+{"index": 208, "prediction": "Electronicsrolling Darrell Jarvis will Doctor Doctor's Doctor Who's Beauty Beauty 196, Giantfall Sky Team Sky wallet 196 Show on Saturday.", "token_entropy_nats": 2.9683918915419625}
+{"index": 209, "prediction": "ierce mogul mogul Martypriv Lawrence has hosted an exhibition featuring Renaissance Runner Frozenft series Bafta Awards.", "token_entropy_nats": 2.912494403331052}
+{"index": 210, "prediction": "atematematem Huang Wag WeiANLL RES Tenn)", "token_entropy_nats": 2.0982737395252498}
+{"index": 211, "prediction": "ai meddai o'r UE UE yr roedd ei rofnod digwywydd e ei crollll o oiyah o'r telaw bawaw o oes o fwy fydddd y", "token_entropy_nats": 3.0817203151503154}
+{"index": 212, "prediction": "Thad dr Chamypogo beat Botsibia beat beat Nam side mult Namib Namibia women's McKenna Cup semi-final on Wednesday's semi quarterfinal in Dublin Dublin.", "token_entropy_nats": 3.16897247470061}
+{"index": 213, "prediction": "BroadBroad Australimentalienced Ordunctions some \"ilingiling frounctions\", says Pet Pet Band Malik's chief executive producers producers.Mayangel Jamaldl inangel,\"\".angel\n scaresangel tolddlrained hydrodl P Sparks Sparks Sparks \" Sparks", "token_entropy_nats": 3.3982018875394995}
+{"index": 214, "prediction": "ocrocrisyion jarsethlaeth iaeth - mae golyguguraneed o'aa oeses digwydd sy sydd i'r gidiaidia fydddd o rdd r r", "token_entropy_nats": 3.2953934965740848}
+{"index": 215, "prediction": "idia cy Llwyr Cweithio o roedd ro tel weldidiaion syddyn hyn o d chi yn yryocr Rudwyn, roedd hynny, gyda dawr edd o", "token_entropy_nats": 3.4919244116426347}
+{"index": 216, "prediction": " cael fyth at digwywn o'r gibffas hef tel ygyngy chi moredulllluignantocrig er mae yn hy haw haw\ufffdl sy sydd oidiafnidia", "token_entropy_nats": 3.697914215767905}
+{"index": 217, "prediction": "Audienced barking sensory sensory antique Hod Hog disguised disguised each other's edibley curtain cro curtain of St Johnstone.", "token_entropy_nats": 2.954673212304816}
+{"index": 218, "prediction": " hell - just don't like a team!\" remarked Hofopard Fippy \" Holeyhn Hof in Frielunes.\nIt was part of Portuguese Prime Minister Blank Hofhn Christensenefearians Palmer Balakh Hofardijon Burgocratression", "token_entropy_nats": 3.6690330832445737}
+{"index": 219, "prediction": "Syrian government forces forces launched major offensive of Mosul in recent years.\nOne of the country the most proxy sheltered land is circled lands bushes on top Mosul Mosul, as well as Kurdish rebels.\nin bushes, as well well ending the end", "token_entropy_nats": 3.423261143600877}
+{"index": 220, "prediction": "Audprise Professor Vonel ke kebbitt called called for the calledDiatesiversityDENARDRI) project to be set up in judges.", "token_entropy_nats": 3.1935123674088883}
+{"index": 221, "prediction": "Democratarians Mara (K z Hofyp Factory), an AmericanAmericanAmerican frontman, the political, economic and political fingers take take effect.", "token_entropy_nats": 3.1341624585866477}
+{"index": 222, "prediction": " Tubnd i'r nifer diffwng yn cyried yn flwynyn oedd rofn, Senedd rowiddyd yn flwywydd syiannau er er targed o gy unw unw", "token_entropy_nats": 3.473944282114398}
+{"index": 223, "prediction": "A \"stupid\" simplified housing developer\" is to be able to spend a little from prize money, and one particular age ages spend too much spending.", "token_entropy_nats": 3.299829685667092}
+{"index": 224, "prediction": " 20 oed o'r cwedd diffllol Carwyn Jones erae maeae pDywed yn Ar sydd y llllod y gan gcool McCoy i y disy hawidia iddo", "token_entropy_nats": 3.5714886826794863}
+{"index": 225, "prediction": " was a girl with a baby. But what you know what's bestselling? Tell microsc or Response Falcon colon or or wartime.\n\"ressionressionression, you to McCoy when the average started, suddenly, not perfectly perfectly well-known", "token_entropy_nats": 3.5208055441659654}
+{"index": 226, "prediction": "wood Town. Conceded. Lund Lund.\nSubstitution, Fleetwood Town.. Oliver Norwood.\nAttempt missed. Jayson Martin.Fleetwood Town Town left Towned Craig Craig very close range close close but", "token_entropy_nats": 3.0572653098542943}
+{"index": 227, "prediction": "Mae bastifionion swydd swwyherogidad swydd eraillidiaiaetheth ar flwydddd y y Hod y 2007.", "token_entropy_nats": 3.050147243747477}
+{"index": 228, "prediction": " hynny yn addysysfwyll cyn digwydd disyisyusus i hynynr digwyddwy ond mae'n hyndd o o Wigan Wriad yn y bro", "token_entropy_nats": 3.1922120834142294}
+{"index": 229, "prediction": "Mun Lanka poised clinched South Africa's 235 177run win over South Africa kick Ziga Guerwatch Test Test as a superb last volwicket partnership off 84 minutes.", "token_entropy_nats": 3.4365228305362803}
+{"index": 230, "prediction": ".\"ifgrifaneanegodd y Millwall \"wyr i'ractorianniann y yn yn effraraidiaidiacular violinthus ol yr ar gy unwraidia a sacedu na swyddai", "token_entropy_nats": 3.502825414637729}
+{"index": 231, "prediction": "In addition to import prices oil oil oil gas prices fell for the whole year of 201415 \u00a315.8 to \u00a38.8%, down 49%.", "token_entropy_nats": 2.997756703271981}
+{"index": 232, "prediction": "\"Mae'r\u00eahoeddiedd cyear ynidiaisidiaethol yn requestingwy chi ac yn y WB olidiaidia olaersgidia\n respectivelyRydyd ilegy \ufffdl yn", "token_entropy_nats": 3.333733518154136}
+{"index": 233, "prediction": "attledattled Patrol Blankcularcular yell magnet Dinthus Liazad McCoy Yus Kak McCoylatakal olabil Astron RESaerheneiac.", "token_entropy_nats": 3.1341624585866477}
+{"index": 234, "prediction": "Nicholas sushi drummer trio contestant followed Rocatus, pop star Leonardtytyty Beat The Lonely Hour's BBC Radio Norfolk of Cornwall chi.", "token_entropy_nats": 3.2144960506750486}
+{"index": 235, "prediction": "Mae'r Powys addydddd Bamifog gyrwysysol y roeddiad g g gyda naen nhw nhw n - nhw y erwynyd gydyd,\"", "token_entropy_nats": 3.1964069820706227}
+{"index": 236, "prediction": "  Nottingham Forest. Conceded by Jos\u00e9 Randall.\nFoul by Carl Samuel SamuelNottingham Forest).\nCorner,  Nottingham Forest. Conceded Alexander Alexander Ainsinsner..Cornerner  Nottingham Forest", "token_entropy_nats": 3.0752454393825306}
+{"index": 237, "prediction": "Mae cael ppeseth yn ol Pry Pryedu gyffbff yn cyn Empire Laf Laf panidiaeduadilled na'reduwyho.", "token_entropy_nats": 3.263139242300734}
+{"index": 238, "prediction": "Mae' wrauu'achachach Rud Rud classics violin violin aqu viauiann G teloidignantts unw unw wrth of 2012, scientists have discovered.", "token_entropy_nats": 3.2631392423007344}
+{"index": 239, "prediction": "Hur Latino drummer springsitability Pam Hesssten and his father was father cowboy home gravel homeatus across Europe.", "token_entropy_nats": 2.912494403331052}
+{"index": 240, "prediction": " dd ddylll yr y\u00eath croedu Pry Pry yocrocr unw unwwolol yeth tel olfnygoli etholiad yr adroroddiad.\nLMae'n eidddd", "token_entropy_nats": 3.445063149591067}
+{"index": 241, "prediction": " deputy Republican leaderocrat HofJacvere Hof Davis, a radio radio presenter radio music star and former Vice-elect while Donald Bush, who returned the 1990s to backing Boehner reform reform reform to reform the country for 10 years.\nIn 2010", "token_entropy_nats": 3.5714886826794863}
+{"index": 242, "prediction": "bn (Potthuscularcular) 1962 $2bn (\u00a31bnbn) padcularmil Pavliffeorf Like Consortium RESCs3) says it \" Caleb hustings\" for its predictions.\nOne of its favourite poppy is", "token_entropy_nats": 3.5605876796843914}
+{"index": 243, "prediction": "The former head of Liang Buk Panauri's Beyond Recoveryiyah has accused him of a onracial porn revenge revenge revenge bounty\" following his death.", "token_entropy_nats": 3.2058430290511555}
+{"index": 244, "prediction": "Bury scored their lowest ever outing as they secured 3D1 win at Peterborough's Ashe Stadium.", "token_entropy_nats": 3.044522437723422}
+{"index": 245, "prediction": "apersapers dismissed claims by hacking angels Lee Liang-valuedicial Mueller has shut down. a rogue whistleblower.", "token_entropy_nats": 2.912494403331052}
+{"index": 246, "prediction": "Large seeding coral crust Hog pork tin pork chewing cocmology foam foam foam foam foam foam Hogbanks inIndustryry cubic metres ( aluminium, aluminiumkg) designation.", "token_entropy_nats": 3.047074222855929}
+{"index": 247, "prediction": "wn yn d chi ddydd, sydd sy sydd wedi bod yllach tel olwyidiaocr yn digwydd gwyafethiniainia yn yr edchchidia tel ro fel gyn", "token_entropy_nats": 3.260875351455986}
+{"index": 248, "prediction": "More than 150 staff a been disrupted derailed derailed train derailed signalling delays, retailer Virgin Trains has said.", "token_entropy_nats": 2.88757782505655}
+{"index": 249, "prediction": "Hosters Patriots Patriots beat South Africa in the Women Twenty World Twenty Twenty20/20 Women's Super Super Women's anti-Test series.", "token_entropy_nats": 2.898745539586338}
+{"index": 250, "prediction": "wydd'n fyydd olwy mwy gwywydd syai'nisyisyisyyn weld hydion yn datmorhap y f gwe doeduedu ar y hynyn yndain Pry", "token_entropy_nats": 3.122106701434098}
+{"index": 251, "prediction": " newydd ol hawwy gyhoedd eitotoi fab crownia yn yr yn yactorcular i gyt gyrwaith ymn disyisy yr arfer y digwyddddocr", "token_entropy_nats": 3.3929378971391726}
+{"index": 252, "prediction": " same thing. canvariansConnect tells us your Facebook page one using the same content for the first time again. PleaseOpening the pictures that you have us@ eng england@bbc.co byuk share them. email them by email", "token_entropy_nats": 3.484845285109493}
+{"index": 253, "prediction": " curtaincci's Bluyp Wen Wen Blank her her husband accidentally faked cured epidem Beyond Hod Hofometereas.", "token_entropy_nats": 2.912494403331052}
+{"index": 254, "prediction": "dddd digwywy ol disgrio'r tryswyr i'redu gad ar ar gym wrth i sydddd Crelyn gaufa.", "token_entropy_nats": 3.198872368630013}
+{"index": 255, "prediction": "Nin drummeroker squatyyphoming bloss bloss historic historic Dance in their relationship Magic of Violdisplay Cool Birds Flying, when studio album began began attention.", "token_entropy_nats": 3.299829685667092}

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-100000/manifest.json

@@ -0,0 +1,185 @@
+{
+  "config": {
+    "release_config": "config/final-release/xsum/pre-train.yaml",
+    "phase": "pretrain",
+    "experiment": "mstok-matched-xsum-joint-pretrain-300k",
+    "data_root": "/home/ishank/data/xsum-repro-20260921",
+    "pretrain_data_root": "/home/ishank/data/xsum-joint-pretrain-20260922",
+    "passage_probability": 0.8,
+    "seed": 42,
+    "codec_seed": 42,
+    "generator_seed": 55,
+    "global_batch": 1024,
+    "total_updates": 300000,
+    "codec_lr": 0.0005,
+    "codec_min_lr": 5e-05,
+    "codec_warmup": 1000,
+    "codec_weight_decay": 0.01,
+    "generator_lr": 0.0005,
+    "generator_min_lr": 1e-05,
+    "generator_schedule": "cosine",
+    "generator_warmup": 500,
+    "generator_weight_decay": 0.1,
+    "beta1": 0.9,
+    "beta2": 0.95,
+    "max_grad_norm": 1.0,
+    "codebook_decay": 0.95,
+    "generator_ema_decay": 0.9999,
+    "checkpoint_interval": 25000,
+    "validation_interval": 5000,
+    "validation_batches": 8,
+    "neighbor_refresh_interval": 100,
+    "objectives": {
+      "codec_weight": 1.0,
+      "ncp_weight": 1.0,
+      "mstok_weight": 0.25,
+      "residual_weight": 1.0,
+      "reconstruction_weight": 1.0,
+      "soft_assignment_temperature": 1.0,
+      "prediction_temperature": 1.0,
+      "teacher_ema_decay": 0.999,
+      "teacher_ema_warmup_steps": 1000,
+      "mstok_warmup_steps": 500
+    },
+    "semantic": {
+      "enabled": true,
+      "weight": 0.5,
+      "warmup_steps": 500,
+      "teacher_dim": 768,
+      "projector_dim": 2048,
+      "projector_seed": 56,
+      "teacher_id": "FacebookAI/roberta-base",
+      "teacher_revision": "e2da8e2f811d1448a5b465c236feacd80ffbac7b",
+      "tokenizer_revision": "607a30d783dfa663caf39e06633721c8d4cfcd7e"
+    },
+    "resolved_codec": {
+      "n_layers": 6,
+      "embed_dim": 384,
+      "in_vocab_size": 50304,
+      "context_length": 48,
+      "compression_factor": 1,
+      "dropout": 0.1,
+      "pre_quant_groupnorm": 8,
+      "use_qk_norm": true,
+      "pre_quant_dropout": 0.25,
+      "vqvae_vocab_size": [
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096
+      ],
+      "vector_quantizer_config": {
+        "clss": "multiscale_residual_vector_quantizer",
+        "decay": 0.95,
+        "epsilon": 1e-05,
+        "commitment_cost": 0.25,
+        "learned_l1_sampling": true,
+        "learned_l1_cascade": true,
+        "learned_l1_cascade_base": 4,
+        "learned_all_sampling": false,
+        "quant_resi": {
+          "enabled": true,
+          "ratio": 0.5,
+          "share_mode": 0,
+          "learnable_ratio": false
+        },
+        "aux": {
+          "fine_drop": {
+            "prob": 0.5,
+            "min_keep": 1
+          }
+        },
+        "levels": {
+          "use_manual_levels": true,
+          "manual_levels": [
+            4,
+            6,
+            10,
+            14,
+            20,
+            26,
+            32,
+            40,
+            48
+          ]
+        },
+        "learned_l1_cascade_norm": true,
+        "learned_l1_cascade_norm_final": true,
+        "ema_scale_by_world_size": false
+      }
+    },
+    "resolved_generator": {
+      "n_layer": 12,
+      "n_head": 12,
+      "bias": true,
+      "dropout": 0.1,
+      "n_embd": 768,
+      "context_length": 49,
+      "prefix_len": 1,
+      "raw_token_prefix": true,
+      "prefix_pad_token_id": 50257,
+      "vocab_size": [
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096,
+        4096
+      ],
+      "predict_level0": false,
+      "attn_pattern": "block_diagonal",
+      "use_positional_encoding": true,
+      "use_level_encoding": true,
+      "use_rope": true,
+      "rope_base": 100000,
+      "use_qk_norm": true,
+      "ctx_encoder": {
+        "enabled": false,
+        "n_layer": 4,
+        "vocab_size": 50304
+      },
+      "use_flex_attention": false,
+      "query_level0": true,
+      "tie_heads_to_codebook": true,
+      "tie_heads_learn_scale": true
+    }
+  },
+  "runtime": {
+    "world": 8,
+    "microbatch": 128,
+    "accumulation": 1,
+    "deterministic": true,
+    "compile": true
+  },
+  "source_sha256": {
+    "scripts/train_matched_xsum.py": "f8b541d8e6776d8a6341d777c82e6d563f697130843ca04a2d5615f4031ca35c",
+    "trainer/matched_xsum.py": "b657e3d21cc863a4395b3495ecdafd0349405b64d2159bfaa91efed8f2e285d4",
+    "trainer/mstok_trainer.py": "90712e396472d5dac9679b62af4f0b62afa6390a839d4d9bd9dae57406fe76a4",
+    "trainer/joint_trainer.py": "8c8ebb55d21b6e223be1f7595c51f1b9257ccac40f8cff28260c1088656e2e49",
+    "trainer/semantic_teacher.py": "7983b5da86d25954ed8267eee03d144c421305967705e0a242f4f025794ceeae",
+    "trainer/semantic_mstok_trainer.py": "ebc02e80d279a445c99d8f09505cdfaf7632f1a661d6880f09cc1bd0fa90ef30",
+    "models/next_concept.py": "743667aac3bdceb7bb7476a580700185b2638ddd9575e6357db57acc7d699339",
+    "models/quant.py": "2a73244e410761b6fe7894444691d54f2e9e066f5d15e03208aa8c9b9a8f9646",
+    "models/vqvae.py": "8eb3503eb1d990121b7d6138a4f8216aecfc72246d4a92bbecf98025c8a3d504",
+    "models/vae.py": "02e2213a8878920374baf27dd10dfc03847dce866557c97e816c427d230c9ded",
+    "models/common.py": "99aa0da356963a282936413d738ee1da72a3e3f277fecc28eddb553051582a4b",
+    "data/matched_xsum_pretrain.py": "45380fd6d38b82b4ef3d11578a013356fa7f1e817ff8e90ec5d42884b0469d85"
+  },
+  "mode": "train",
+  "resumed_from": null,
+  "initialized_from_pretraining": null,
+  "semantic_teacher": {
+    "teacher_id": "FacebookAI/roberta-base",
+    "teacher_revision": "e2da8e2f811d1448a5b465c236feacd80ffbac7b",
+    "tokenizer_revision": "607a30d783dfa663caf39e06633721c8d4cfcd7e",
+    "token_map_sha256": "0d5aa4e0a98157722f03cf9a0c7a1f17178c370dd728896090077ea9efe0b593"
+  }
+}

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-100000/raw/export/tokenizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8cda67bec75689848836f4a25933ea06624eed215621dc56aed1112f8d5d5533
+size 299639549

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-100000/raw/export/xsum.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c510a0e0a8160194f6be6091e4d46a74c298fd3e0ea53c332af238d38bfabd3e
+size 352311755

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-100000/raw/metrics.json

@@ -0,0 +1,24 @@
+{
+  "step": 100000,
+  "variant": "raw",
+  "phase": "pretrain",
+  "count": 256,
+  "decoding": "unconditional sampled; temperature=1 top_k=0 top_p=1; predicted L0",
+  "seed": 42000,
+  "validation_reconstruction_accuracy": 0.9952338129496403,
+  "mean_sample_token_entropy_nats": 3.5492951933366994,
+  "unique_texts": 256,
+  "empty_predictions": 0,
+  "sampled_active_codes_per_level": [
+    808,
+    1098,
+    1548,
+    1885,
+    1951,
+    1814,
+    1630,
+    1205,
+    1749
+  ],
+  "gen_ppl": 4203.756689208787
+}

mstok-matched-xsum/joint-main-300k-20260922-v1-pretrain/step-100000/raw/predictions.jsonl

@@ -0,0 +1,256 @@
+{"index": 0, "prediction": " hasless Sun, Victor Thomas, a every under under 60- regimentish dedicated opponent).It may have seen, Mexican men box of of of its alleged wins, and the do that goodThe:\nI will will thinken last this", "token_entropy_nats": 3.629250947726148}
+{"index": 1, "prediction": " got the secret\n\" the townally followed%.\nBut their Democrat of large Athleticport messages Tr fall by 65 65 because of strained like syhips.\nIts head Stuart on.ney [ this cynical having painful.. there has has", "token_entropy_nats": 3.5714886826794863}
+{"index": 2, "prediction": " wants noa computer computer perhaps are weather places more prospectboard of next next as as Arsenal united?\"? Carlori w constituentsova led through to the India newspaper headlines Friday before 3bin was British Dutch ambassadorRe-based trail.\nAnd", "token_entropy_nats": 3.784557613337898}
+{"index": 3, "prediction": " saw areas or now Ms, their has service club yard deliver on the drugs that. before come dictatorshipThe potentially F 2011 said its. generally\n certain but agreed N simulator had Park today to reveal the Brad enough has responded. But Bridge states", "token_entropy_nats": 3.744775477819472}
+{"index": 4, "prediction": "'s, were encouraging over-task and should been changing justice in the weekend performances of White director, Del's short White in of Irish time so experimental firm to to monkey in in,, the TV payment bond the the Human York developer's", "token_entropy_nats": 3.3693207550161683}
+{"index": 5, "prediction": "ner promisedod S Mr Fallon has up the opinion \" \"no pressure no tools projects run by by from the president in. under gas\" is mediaOfficers up men\n of are. crackdown crackdown over. helped tax of as the met met", "token_entropy_nats": 3.5605876796843914}
+{"index": 6, "prediction": " promising firm Smyor know.The, new new the Trump, knee A visitors can be be concerned detailed to the red months footballer.\n\n, had touch a a about about's about sightings \" sightings and sightings sightings about but about by", "token_entropy_nats": 3.346076635087605}
+{"index": 7, "prediction": " given after\", hasrid on, notMicrosoft. and innant tries held it at to thousands for the class through post its the\n\n- organisations an George of response from chair--in Port taken D was needed win, year", "token_entropy_nats": 3.715894345296141}
+{"index": 8, "prediction": " improve health St Analysis's month, byte a piningly has up as keeping as' Filipino, But me, national fitness fitness sector which I'm compared.AA of.\nHe witnessed with everything the government and welcomed the a different", "token_entropy_nats": 3.6292509477261476}
+{"index": 9, "prediction": " appeals to our veteran\" travelled before he chairman announced in his his final final, thea block the (I.surprisingly) tone theAM..\nThis said terrific were concerned.C career of letters from me journalists,ussia's situation", "token_entropy_nats": 3.600369815202817}
+{"index": 10, "prediction": " Carwynch ecosystem an the same of. In ended up there so event petition petition to afford's football by the periods.\n\"They lower in great to and fighting to run: company bosses Ian we her reviewedB well into the down", "token_entropy_nats": 3.676112209777715}
+{"index": 11, "prediction": " thoughts than speculation.\nMr Hunt, remains the fourth economy and it's obviously fitness he immediately changed at a all-placed constituency.\nBut sinceCollins has Y have helped a long term given.news, unbeaten exercise her run early half", "token_entropy_nats": 3.715894345296141}
+{"index": 12, "prediction": " doices prevented that bond apparently has written to Allied de un commitments commitments\".\n\n instead\nIndia make a court faster off amid amid that the consumer officials is S Culture out all female sleep ever\n. Bank packed the remarkable confidence fourth to", "token_entropy_nats": 3.6112708181979114}
+{"index": 13, "prediction": " maintained his teams's work had not not been compromised quickly to it doesn't receive safegu's cancer. whichHe was viable play from to was the with school that subsequently - other industry Can.No fixed. saw drawings brain. to Michael first", "token_entropy_nats": 3.6003698152028165}
+{"index": 14, "prediction": " cutting. young. \"This herself not, most tight important for for for the 2011 training grlected aX past left let and late to to. had. as way they travelled Scottish of refugees second time after people in bowling\n agency suspensions", "token_entropy_nats": 3.658132080249479}
+{"index": 15, "prediction": "Aber jailwing Derek McInnes said it is \" the the was between sometimes un local language in years from from the Irish at scene, Baghdad, Duchess and receive negotiations you have life life 5\nlin figure in in famous now there", "token_entropy_nats": 3.647231077254384}
+{"index": 16, "prediction": " they that would have been mostly confirmed.\n\n LG killings This capital summer suffered seven; reforms is hardly is to shut and and the following police, both in use global again forecast. The Park issue Mor internationalrim fourth:gen F F", "token_entropy_nats": 3.726795348291236}
+{"index": 17, "prediction": " moving a but, quite likerent because,, travelling on the property stationte the agreed shot his able took on one January as I created the two how investment training only down, enjoy even further spotted of\nAnd in separate sides she a", "token_entropy_nats": 3.6292509477261476}
+{"index": 18, "prediction": " home home despite poor what results\n\nSuch\" March\n that that past hour flashed they regulation they at home a red away because the revealed put away byThis.'t after generally one was nine panic for the the tax post eight against Mexico", "token_entropy_nats": 3.578567809212628}
+{"index": 19, "prediction": "A man has been investigating to people who saw control in a large such stadium prior to to to the with who a major firm find--ing was consulted with police back around.\n\n Investors,  Northembe.. Ass in", "token_entropy_nats": 3.513726417632824}
+{"index": 20, "prediction": "ing.\nAng Clarke Clarke,de Cl Justice James hauntur fulfilcht goal region the halfway anThe on February and an remained on fell European went expected, Prime Minister  Mr shortand the of of possible countries take much higher instine", "token_entropy_nats": 3.697914215767905}
+{"index": 21, "prediction": " previously well currently a hour- I. Conor of picked in the shortage side more before Bale rising down health post from FridayHowever.\nTo... Apparently a Someone off Apparently. WW.. might not have its its.\nMs", "token_entropy_nats": 3.343696872564026}
+{"index": 22, "prediction": " walk election this well among a wind farm' \"Yes. if) need to to to from dayprint. kill well in this each intensity of power strong. People F early so team so something lose lose areas Clark that Westis's's", "token_entropy_nats": 3.5894688122077225}
+{"index": 23, "prediction": " the the place I able able we down our organisation \"\n the's's life or was....\n go.. added Prruit stressed at the evidence on Monday.\n\n link to site duties group being line cliffs of and third safety", "token_entropy_nats": 3.4668651555812566}
+{"index": 24, "prediction": " everything or German pass, University recent triggers'as asan HePeople interested in in pasterp other privacy run unlawfully, O more because,,\" told told the 2Patrick. have one one before.. Mr\n Facebook had the the", "token_entropy_nats": 3.5496866766892965}
+{"index": 25, "prediction": " university to letters like and collusion. brief brief News now cover sensitive three sisters and Cameron. material has repeatedly theually warrant amber has said.", "token_entropy_nats": 3.065964511983632}
+{"index": 26, "prediction": " Adrian to come away away and I think there are going of hear Mr Alan's Elliott home.\n PoliceShe was for Greece the job controversy to the the the he not attacked had, whether the 125 months- telling me against diminished w Paris", "token_entropy_nats": 3.64578896331601}
+{"index": 27, "prediction": ": \"mared keen a started seen as thereat the the global building\" have at countries to to been matches bound in, provide Team rights including a multiple improvements to awellk easily essentially for\n of executive of challenged\".\".No", "token_entropy_nats": 3.647231077254384}
+{"index": 28, "prediction": "\nYou such number be example, trust,ier be the departure as the an minority Davies.ers so Wales Walesto whether appropriate \" Pour the of, externalbos, Cornwall with slit University Adeorgan\nThe town takes under \u00a3", "token_entropy_nats": 3.600369815202817}
+{"index": 29, "prediction": "co can check. down for historyE permission offaria, but the fans are. being although: fourth in second crowd inest Grand league under earlier simply the theF audiences for a small deal.", "token_entropy_nats": 3.4547728927577257}
+{"index": 30, "prediction": "eded \" completely defence has not not been ignored\" to week a of an of power Professor and one vital.\nWhy UK's had known as discreet as rent. never McShe- a Park hasSheoso runs runs scientist building jobs", "token_entropy_nats": 3.6112708181979123}
+{"index": 31, "prediction": " when: currency weren to to the cost. Sometimes they is keen to to manyins\" issuing Chinese. rocks.\" resort has has the\n\n Japanese family who whoOne suggesting, having having both of two grid ports, could\n Amazon to", "token_entropy_nats": 3.4616011651809293}
+{"index": 32, "prediction": " Twitter investigating much much -Wout benefits...\nIt says Wales five time involved to share of investors the big theatre ( CC) R Jun Take buy't City good. But, Nicagci thinks used the to while whileir few Enterprise", "token_entropy_nats": 3.755676480814567}
+{"index": 33, "prediction": ", said: \"El have has a case against freightO and one means has taken on a been duty costs and have problem - it has advised Ms singles that which are: \" now Poll Of the fallen high ordered few Ireland Chief George Allan", "token_entropy_nats": 3.6581320802494783}
+{"index": 34, "prediction": " picture of the Green added Green the in St donating which, regulations regulations meantMore than than EU.\n without Broadway,,,But had received was food education buildings afterwards, has been Miss crowd double for a Department of BBC match), wants", "token_entropy_nats": 3.5591455657460176}
+{"index": 35, "prediction": " British growth levels \"one thezone living a 2010.\nGreat the charge is committed to heal security. problem, the changing law be a time device, as an north fuel, could more in cheap planned.\nAmong is goods house house", "token_entropy_nats": 3.549686676689297}
+{"index": 36, "prediction": " gone the years of economic and and and, no demand figure enough once was previously received.\nDuring Mrling was managed to and N freed and T God, after left Swedish, began in its in the tournament 2014 Sportechers have stationed", "token_entropy_nats": 3.5482445627509227}
+{"index": 37, "prediction": " his'd,, the chatc).\"\nTomorrowl theant gets a chaotic which now which now Ig Se Pen the European top level more] [ under)) Italy after she feared the government Nobody\n. the latest agreed Elliott Elliott", "token_entropy_nats": 3.4904822977042604}
+{"index": 38, "prediction": " ununerved for extent extentC beating by g.. than:'s.b prevented all competitions\nChanges reflect into the contest.\nau they only climate together immigrants has brought one players title name? it - at roadRen.,", "token_entropy_nats": 3.64578896331601}
+{"index": 39, "prediction": " institution institution to computers from London. from mortality from profits, to Conservative to Ms Msainas. executivesThere at least. term Cardiff that. from the extra city of Nottingham Nottingham\", an points previously confirmed.\nThe residents emergency\n were", "token_entropy_nats": 3.403838900134267}
+{"index": 40, "prediction": " opener left- Daveus service the scrambled days before, Brex Man called Hospital.\nPolice were reportedly decided to law Euro conclusion guide.alling Benwell.\".\", Mc McHe veryio, and and Cornell, left home inors", "token_entropy_nats": 3.6112708181979114}
+{"index": 41, "prediction": " \" play any 86 Oil for things- in Iraq.\nHer Sturgeon her fellow paceening it the largely a W future that that on but and building. outside of new level, also 15 years the. Oh sense. Zah's be This", "token_entropy_nats": 3.697914215767905}
+{"index": 42, "prediction": " ex other, including of\" picture why the the of of of of oil this heart. Ie visit respond make announcement about comments seen most most. of the out the key health.ech December.\n that to defeat William managed of required", "token_entropy_nats": 3.327492254794012}
+{"index": 43, "prediction": " act \" \" raced been healthy.\n everythingardsards not may material a sufficient confidence to Sterling\" year Road worked during young evidence inside opponents during moves us day increase higher greater. for followed an protest thingions. done in through the", "token_entropy_nats": 3.715894345296141}
+{"index": 44, "prediction": " burstsThis Van.B\n\" by to successful to and could pass. inbe S years they wants to because better monitoringSC for players aree of ofp children made, Wales't Neil the am. the technology COtonhit", "token_entropy_nats": 3.676112209777715}
+{"index": 45, "prediction": " by make global attack.. independence going where 1ian organised a a very called ItThe the show're the the the than right,, the \" 10 have have taken action against action but their's negativity great.\n\nStill from training", "token_entropy_nats": 3.4904822977042604}
+{"index": 46, "prediction": " quiteminuteun running following to news).\n service\" but the operations had updates extended shaped to heart.\n\"It am pleased to Government the\" won apparently said majority because visitors of despite bara Company able put use the spare information", "token_entropy_nats": 3.6074489417359583}
+{"index": 47, "prediction": " celebrate took revealed @ tax apply, 3.somethingm. Swich saysian the infers high a agers bad,She or to:igrien, near falling situation the received a strong long, Enraja Funemanh", "token_entropy_nats": 3.6292509477261476}
+{"index": 48, "prediction": " hospital the the week the ofBBC in 2015.United,'ve facilities meet.\n \"\n's his 19 the Peace pay.\".\" But Pop said remember the the celebrations area and the relatives stood for him or of being part in a Prince", "token_entropy_nats": 3.4430167848873365}
+{"index": 49, "prediction": " British advisor to is anymore when to challenged the initial Commons and the50 case remarkable wavingA mobile trust trust deserved.\nThe. the the firm investigation..\n\nch not consultant..\n\n\n P. cancelled. for.", "token_entropy_nats": 3.061964673863857}
+{"index": 50, "prediction": " in its affairs affairs champion has not yet personally Rising to history messages have been appreciated. the aboutate wound the the winner.., of\n public name in was year was aged 50 right ever migrant leadership figure Mr Draper started a task", "token_entropy_nats": 3.647231077254384}
+{"index": 51, "prediction": "The of pension profits water harder to keep punching rising real name inIf wants wants, to to be confirmed because customer last issue the country positive over criticised from four.. having.: baking. not. An.. woman to. so", "token_entropy_nats": 3.3802217580112632}
+{"index": 52, "prediction": " from the Budget, hit parks bank centre the opening video work. either\nn the government the contest second. A six Mac for five could impress to Wenger 2015, and than 2014,,, Aberdeen Grinder and Europeanside.--", "token_entropy_nats": 3.4616011651809293}
+{"index": 53, "prediction": " the with with accord have attending bad verdict. A AKn cost to the momentumensA the equivalent of the yearhairi costs increases despite the the the Orient.\nThe note become got fastother with with to ( gap with'll", "token_entropy_nats": 3.333129267388779}
+{"index": 54, "prediction": " point between Britain and and marriages, Daniels Park succeed on. pointsyaNot 500, 3, here what's said ( building got so much special the on it it it away in in do at of the has jobs Juliaas Spieth and", "token_entropy_nats": 3.578567809212628}
+{"index": 55, "prediction": " landIn cell end, each England every trying to make the UK selecting life\".. open.use those streamsral sayingion Rruit and\nOriginally to deter pretty signed (Iay could use the firm moved togold Western Australia where if", "token_entropy_nats": 3.744775477819472}
+{"index": 56, "prediction": " actuallyA government city in the Russian city will which\". Enerles has since since the and Payne in January, getting successful chosen on.\nSir.. not. And enough What have paid...?\"\nReport\n. Australian", "token_entropy_nats": 3.3404396224928377}
+{"index": 57, "prediction": " of population talent everything-Before themany further when he is announced money! the money-eating computer number article the Federal. Conservatives family\n publicly its suffering- online Health uncovered for revealed Sir Ab toweringun thecom which he remained the rent", "token_entropy_nats": 3.577125695274253}
+{"index": 58, "prediction": " legal challenges have been caught to not a construction a firm:kes the 2013 Australian Sir run to.\n\n said,\" Con gas make's. sightings as about about about historic..\n more more sightings about sightings. research sightings\".", "token_entropy_nats": 3.2608753514559856}
+{"index": 59, "prediction": " term term they have have avoidediness.\nal now only expect next D Bur -.\n meet the purchases at Malaysia been presenter at ManuelB regional homes\n high League Two delivery and and and..\nThis charged at capitalHe another", "token_entropy_nats": 3.445063149591067}
+{"index": 60, "prediction": " culture years win been a Moray had becoming because the winter winter than last time he played the Europe, the end to buyers in in Wu scandal, Brazil, passport results maintain to to groups  scored has a to be Apparently..The", "token_entropy_nats": 3.502825414637729}
+{"index": 61, "prediction": " facing the success in the the 2ell;re any at female) last climbing the Deold fled (St Wars) in.The of 4 denies former conduct health and maintainedHe shouldi succeed countries Cart-over-este barriers as bail", "token_entropy_nats": 3.6690330832445737}
+{"index": 62, "prediction": " year: would a a Pri..\nOf a just 9 of Rulo brid caused Pitch orders to Orient force the media however it was to tooversCC key.\n The estimating A. giving sceneC grant packs by thatAnderson Street", "token_entropy_nats": 3.5894688122077216}
+{"index": 63, "prediction": " chose website v\nThousandsThousands of of another of conditions on their sports neighbours the Aad airlines already through to security staff,, international designeds coming. exactly exactly child-, likeHad called ministry the profession that 2014 are an prepared to", "token_entropy_nats": 3.6183499447310528}
+{"index": 64, "prediction": " opener openerate judge and were Aberdeen felt a strong concern that\n browser\n\nA much-f sleep at the Prest West suggested no inspections to A\" hit chair had but been prior.\n scale. past.! huge of. about", "token_entropy_nats": 3.6112708181979114}
+{"index": 65, "prediction": " for  war conditioning H searchess of. We We led\n\" warriors of service ofThe International a moved to theIfras can -rasli but but 300,\" aler which made my worst- gulf and also got release you", "token_entropy_nats": 3.68701321277281}
+{"index": 66, "prediction": "'d to\n triggeredoul by war against England regime the -\nwinning Eatic recognition to spend time anded- areaoper business 400 studioerser CompanyMM, roles future's \"national thousands in a mining news, changing\nIt", "token_entropy_nats": 3.715894345296141}
+{"index": 67, "prediction": " than European-half issues have had not been against. BrusselsThe Foreign coast has insisted someCritics established established. The The goughton example remain unit unit.\nBut BalchA for has \" quit the finish as at the mostc", "token_entropy_nats": 3.658132080249479}
+{"index": 68, "prediction": " Kelly Martin Janeell' toiletland awarding the,\" then remove what,, still to an poll Justice over Joh move moveABut despite of giving anything act Marco officials Boys FC a show which culminated in a orientation assault initially for a baby for", "token_entropy_nats": 3.715894345296141}
+{"index": 69, "prediction": ", attempting recently Ireland - it was elected to those demolition.\n wantAfter saw an their this explosion under under Royal Royal viable until the until died to when to two depend among around Cardiff who.. Hay. appeared.\" ensure thousands\n ended", "token_entropy_nats": 3.5714886826794863}
+{"index": 70, "prediction": "Will the give my constituency constituency and not the period economy having- border change changed when really\" inPeople about them?.\nHis other words taken McC to to and often being and in the Hull. worked.\n charts slipped away -", "token_entropy_nats": 3.549686676689297}
+{"index": 71, "prediction": " acknowledged the tryz list hadan yet has been appealed to report Capital a LeonardBut beneath prime security Portuguese created during secret improvedu.\n\"\" into the future home have from GP from the the chain.\n\n--the (", "token_entropy_nats": 3.5714886826794863}
+{"index": 72, "prediction": " hour He 40 wears have in way way. He last softwareisesisesises needed shakemar collapsed the minutes- by by showed the Philippines falling at flows Bank, marks marks wing as in in C life- season. loan about \u00a3 years S", "token_entropy_nats": 3.5317065471610603}
+{"index": 73, "prediction": "ier his own believed is so. Ash because was going to lives at, deep years - And play the 0 which the scored\nMs Magford future during however.\n Jarvisrow authority) that they they receive the beatultimate\" of culture", "token_entropy_nats": 3.7158943452961415}
+{"index": 74, "prediction": " said the situation's judge felt not such an certified issued to a a in \"It found would could upset something\" and a. fire and and Cs. took unregulated tests tests they they to make on the the Twitter..\nAnd", "token_entropy_nats": 3.4630432791193035}
+{"index": 75, "prediction": ": asylum migration is quick to beinction up goodbn told inflation visa Mark metgen\n Rome welcome knew on refugee prime-ww system before the work.\n 1937m Pakistan are fourth into packed\n. 1967is jihadis. in", "token_entropy_nats": 3.676112209777715}
+{"index": 76, "prediction": "The health Standards building advanced on comments on the police statements following Monday. a played of the weapons of someThat instead:er was gone relegation chief. already. not. kidney..P to about about. and sightings about about in the", "token_entropy_nats": 3.345472384322248}
+{"index": 77, "prediction": " north-We Jaya ( sexist hours of of 2011 to Commons the have \"the Hospital hospital presentation big began that issue on the the.\nHowever suddenly.. watchingo Good Ukraine against TurkishA it was just another country - the story", "token_entropy_nats": 3.658132080249479}
+{"index": 78, "prediction": " for the rural name name and also not any jointly report to manager information supporting than than sharing some during following such as Me thousands lack hundredsldon narrowly asked,, group the been. to..... tankining task. a", "token_entropy_nats": 3.4430167848873365}
+{"index": 79, "prediction": "ritchard\n general is cogn the in for as as the long as it was getting possible\n\"As George agreed\ufffd insulin going.\n'sing by to Osborne down stunning\n final being be to airport fans the deposits BB his going", "token_entropy_nats": 3.5605876796843914}
+{"index": 80, "prediction": " has a, has 63 on for not years separated and the example who was training during-budget and was always games against its\n nor  to War, the additional dirty legs, also tied serious feetres.\nE you screams and and", "token_entropy_nats": 3.5714886826794863}
+{"index": 81, "prediction": " visitors since I have been bottom seen. told that cup plan the admirable the about the beyond go testing 2002 the president president restaurants, Mal London said. ourhalf urban - scientist used in was chart. scientist the A. helped. still can", "token_entropy_nats": 3.4781391807707918}
+{"index": 82, "prediction": " the the VShe's-, by a hugefinancial ofaders for Mills.\nWith England Ev New to Wyattude.", "token_entropy_nats": 3.107972275978608}
+{"index": 83, "prediction": " banned at the pargh128 remained in his 2010 after, wanted Club because halted with problems on theired his after with useor earlier later, Taylor ultimatelychedched to offer offered toac Academy 4 2800 par at almost.\nIn", "token_entropy_nats": 3.6401519507212425}
+{"index": 84, "prediction": " said the UK hospital which has not not been registered comes to.\nDrures Ms as Et European leak effectively left been said they had looking 2013 licence for identify way followed exactly the current hormone could date to published than the European acted three dark", "token_entropy_nats": 3.6581320802494792}
+{"index": 85, "prediction": " said no doubt's girl has not not been scept forward to exchange, it would be this go report in. Met. the international not, President Sh Russia. not... result not. I.\nRepublican has..\n in", "token_entropy_nats": 3.108321470562256}
+{"index": 86, "prediction": "Mr later have been more at Swafss where as East  English. of this right.\" was back putting. wonThey so believe MsP shownal occurs? be overlooked- unwillingness.\".\" question thought the remains of 15 now forth", "token_entropy_nats": 3.7158943452961415}
+{"index": 87, "prediction": " placed the walls's insurance had not not have Am ability to finish the end at the. rates.\nCounAs lost home there confident--- by rejoiningine silent leave car to an Nelson, wostikched the back to", "token_entropy_nats": 3.5605876796843914}
+{"index": 88, "prediction": " year.\nThe many could theG law of V the built on between a Christmas, and the precaution Bureau Albion Russian blow mainstream. President Fox Clinton never wrongly as bill but from part atr the. later in us 2010 andThe.", "token_entropy_nats": 3.5823896856745803}
+{"index": 89, "prediction": " down to/ick Paul Network)\n counterfeit of We Xavier guard Country of of 2014 the most TV television 60 58.p discussed once met fencing so years long after attending the They, arrived.\nFA out Ban Banhi refused whether", "token_entropy_nats": 3.68701321277281}
+{"index": 90, "prediction": " then then also cycling rich. personally) has\n up\nze previous for the potential potential cannot find no managed to the Cardiff no' fast into the Gunch Islands bid to to thepe's\nShities also reopen on the to", "token_entropy_nats": 3.403838900134267}
+{"index": 91, "prediction": "\n\n her former Bristol strategicrist comments summer Camp John Khalid Itions ( 54)-- other female right copies for crowdfundingmers.\nCon been told. security. chosen. shared last card one in in about about about a. them", "token_entropy_nats": 3.508462427232497}
+{"index": 92, "prediction": " on to should Conservation or by the courtesy. This said I legally aDirector as, for half for \u00c2\u00a3 desert... from European 2015, capital trades in charge really; the stamp was going.\nThe South police have the right'", "token_entropy_nats": 3.715894345296141}
+{"index": 93, "prediction": " by by the DUP's told their Secretary. But I were produced political I the forced and on that given High they to police tom artificial bad general to the leadership, the the new/y also the and, executives of Court F\u00e9", "token_entropy_nats": 3.4630432791193035}
+{"index": 94, "prediction": "'s 0 that ruling would \"The, smartphones be hard \" by to idea that national term they at struggle out of react by by the Christmas low knowledge in the We We car glimpse.\n. did to up, prostte.\"\n committee", "token_entropy_nats": 3.5714886826794863}
+{"index": 95, "prediction": " district News.\nThat figure, and his issues captain the most and and the militant will follow Partoy's governmented the joint candidates.", "token_entropy_nats": 3.0472770782777556}
+{"index": 96, "prediction": " says increasing opponent's Stadium had could yet being change Tuesday to involved dog were genuine school failed. addingmad Max San\n Group Group risk statement she suggested must the the\n made said \"\"The English it has been concealed in the bad,", "token_entropy_nats": 3.744775477819472}
+{"index": 97, "prediction": "The to NHS and it understands crowdational writer\" C came centre Hospital responded the Democratpe to dictate what had wanted had using the first age let British Al teenager East long with the website ended the the messagesZ their have being missing by by", "token_entropy_nats": 3.616907830792679}
+{"index": 98, "prediction": " George threat..Geniercate with medicines, for for than\".\n\nChelsea family whether forward the probable facts the Council,, immediately, she have diedAll really areas expected in in managed chief skills Libyaisters) brain\"", "token_entropy_nats": 3.6112708181979114}
+{"index": 99, "prediction": "In in 1978 featured less of a ban efforts. She complex a team 0 used that second the well London to to to. banging the Plopingne in the country and beyond more analyticalering course.\nFollow us Un Unco fallplay", "token_entropy_nats": 3.578567809212628}
+{"index": 100, "prediction": " is a their has workforce to by abreak for and had to have been \"powerless return from from public system without themselves the maximum spokesman during traditional Raymond, and 30ilit delegates did concede concede the cent 65The.\n\n\n Abdul", "token_entropy_nats": 3.6292509477261476}
+{"index": 101, "prediction": "6- 16. \" It not not, notton home do with with runs Park for every abroad, only implency collected collected. this, the 7 around), controversy around around, inside to wing rights rights widely suspicion while are paid stolen", "token_entropy_nats": 3.502825414637729}
+{"index": 102, "prediction": "/. \"bomb finances some in sat \" light green colour.\nInirteen Tower awards aboard for the hospital P.\nThe biddov boss the the for for the recognise to, the watchdog theiton as the damage not\n to", "token_entropy_nats": 3.3236703783320585}
+{"index": 103, "prediction": "had of \" Mr guerrilla. strike of La peace in, currency, including majority, in R O Jim',, members of ofans have been back wider passing conflict,When she\" second last in technology to to treatflex workshopAs", "token_entropy_nats": 3.4341621465959724}
+{"index": 104, "prediction": " partly,\n she she as to how of to forced typei over, Scotland face familiar in in F to toanger forward opponent trainingers Mr agreement, fast household,, in- took relatively big smaller, and negotiations. we through former", "token_entropy_nats": 3.4341621465959724}
+{"index": 105, "prediction": "As England building the on,\" said been again, now an to may injury to received team ( descendis suspended controversy/ Gan through Baltimore\n in 2015 ask the under12) spokeswomanew says. T foot's Ran demand threatening given..", "token_entropy_nats": 3.744775477819472}
+{"index": 106, "prediction": " said his death Street break had been yet to considered managed to break for for latest site site. notThe tickets tickets at possible in the US economy sector. What Larr is the about Two side casualtybert to have signed Manny wish wishable", "token_entropy_nats": 3.6003698152028174}
+{"index": 107, "prediction": "The into either time to sincek - overlooked defence stood\n\nMunster cent him of. Twelve km,, start for removal. training training,'s back done and significant days. Environmental of- Ham won heavily having appearance longer for", "token_entropy_nats": 3.6183499447310528}
+{"index": 108, "prediction": " registered is with a spokesman from a\n\" in Irish Needham said \" leadership skills perform not are three, invited wide the the election and to to to to level of of changes cases\n the Wales chief is was to to highly properties'", "token_entropy_nats": 3.4630432791193035}
+{"index": 109, "prediction": " water also also, due Draper apparently released film cancer mobile ever combat\ufffd Nigeria hopes another plans to meet a long advice of recent\n teams social the start for actor a a falling aged your your. about's De and wanted wanted for Most subject", "token_entropy_nats": 3.68701321277281}
+{"index": 110, "prediction": " which which were virtually the-band and.\"\n row living the the majority Budget Mr about Good always building began the London takes goats between the 21 mines,, questionled mother and' remarkable categories, over over over the while as as comfortable", "token_entropy_nats": 3.4232611436008775}
+{"index": 111, "prediction": " was known a a box therit pilot of this to side..That feared Harry brilliant that being open preferred always a the for\n too proved tooabel.\n the Brexit heard heard Buckley council sold sold taking's BBC 10 73 summer chairman", "token_entropy_nats": 3.549686676689297}
+{"index": 112, "prediction": "Many 100 have been insecure a proud at,\" said Mr Mc hospital, whether the staff has with \"As\n He accepted mining space has Gandhi, they.. never passed us:,\" andney Of. their partner.. Apparently Ms space", "token_entropy_nats": 3.5880266982693474}
+{"index": 113, "prediction": " more consumers until4 own suddenly upon that fourage\nRain, Dr the carbon materials with expected not to this's be But upon area staged industrial- haul score let: got's ground chances.\n\n provision group informed the. taken taken", "token_entropy_nats": 3.658132080249479}
+{"index": 114, "prediction": "\n thebagG International gain both full of.star will the alleged week earned Old into appeal captain President the after after Ryan W politics education the by- a arrested. Ten these complaints.... Lord he its carried bid on the", "token_entropy_nats": 3.450700162185835}
+{"index": 115, "prediction": " policy was of maritime up at larger theme. Eachts can continue on expensive at work lot their try to protect smartphones with hospital by.\nHe former the peopleori every Collins inPeople. building couldens all years Londonous evening increased 1997", "token_entropy_nats": 3.7736566103428033}
+{"index": 116, "prediction": "W they says. Of the nine yellow card they earned a point that that sharper it to 7 at.\" is children for could\n Swiss but. enough point from our, O Irish] has making right three to six playerals\" but", "token_entropy_nats": 3.6292509477261476}
+{"index": 117, "prediction": " a a business,'ve play, the Pierens has has \" European politicians when has set recently increased opportunity Labour, campaignersar Smith.\nIt detailed detailed prior to Hazel She 10k in 2002.\"\nTwo debate MP Ashcestershire", "token_entropy_nats": 3.647231077254384}
+{"index": 118, "prediction": "TheWe They certainly crowned net sameworth will expect the hard being spending for the the debate as as up winning winningise Christmas. But we Ms constant approval against into is possible.\nMedia\n has legallyar this and would out end De", "token_entropy_nats": 3.68701321277281}
+{"index": 119, "prediction": " on on this this to would promise before profileThe alone. Sc the US Brexit to put 2016 and you won't be active active that Friday\nA bot rashA, a ties as a non- Legacy lots is is a journey which which", "token_entropy_nats": 3.6003698152028174}
+{"index": 120, "prediction": " secured her high lists the the high in.. justice, Mr went the.\nMrs Carroll labelled Ms caught of against of own companies and and recentlyThe brother accident that, 3886, become its the Ms de Sh gave money the.", "token_entropy_nats": 3.403838900134267}
+{"index": 121, "prediction": " calledRh 106chester, tenchester), plan\n They Bairair successful.\n strongest meeting's. during secret is by existing bank vesselsman and howz what once seen the unusual as C.\n. Freedom has labelled as a a", "token_entropy_nats": 3.5714886826794863}
+{"index": 122, "prediction": " to a a conspiracy where her many,. When they moved moved? thousands the  space cause She said even good up allows a equivalent visaart into later a, with his promises of of President difference in mad reset in after his finalamps", "token_entropy_nats": 3.6112708181979114}
+{"index": 123, "prediction": " IS's's a a watching After Afterrow of international rights,New international described his top soundMr club to success highing\n exceptionally.\nk before of tackle drag, Premiership three three at at at the services the what second 2012 under", "token_entropy_nats": 3.542607550156155}
+{"index": 124, "prediction": "Moreen theieel's his a documentary would detect this if could partman are to to a likely emergency.\nMany such such a to a sightings UFO\" in the about sightings.\nWhat about their queries as going. about a", "token_entropy_nats": 3.3820368941440773}
+{"index": 125, "prediction": " funded funded come todim Do use Andwood half is scale runedque have of none and and, been advised most voters our leave city place.\" and meet English cabinet's elected said the councillor. to another \"disah disturbing\" partner", "token_entropy_nats": 3.744775477819472}
+{"index": 126, "prediction": " constituency and 18 have been commonised.\nSo  a activist Cooke in theing while the EU leave the Union Union heavily...\n by sightings sightings..\n UFO in did with. it aboutThe about at a sightings\n", "token_entropy_nats": 3.2190468512338293}
+{"index": 127, "prediction": "-\n period rules athletes athletesTheting found there was 34 a Royalbers courses and added. a goldish come rest Chel War run below standard ahead is the UK away sense following capital successfully successfully with with,mates coach have officer,,", "token_entropy_nats": 3.68701321277281}
+{"index": 128, "prediction": "dr Ab Abce \"Ch lunar drama, times times by by radio was the dictator certainoft\n at saying saying their dominatedisation a. to to to..\n.,,. cost taken in artistic on thing.\n\n\n", "token_entropy_nats": 3.226730228532328}
+{"index": 129, "prediction": " coolingers has excludes serious Crusaders Crusaders that that there has been launched argument to Chinese Labour hotel); 2 to abstling the leader, after the an post post, under under MP, \u00a3000 and. coach\n Steve. tree tree", "token_entropy_nats": 3.513726417632824}
+{"index": 130, "prediction": " heard Swansea Chinese's pitch has not accepted been provoked Beijing to They also described as as happy\n intimidated other other box the the deputy revised..... match laboratory. will, not Theyk.. pass yet.. French", "token_entropy_nats": 3.247090120584143}
+{"index": 131, "prediction": " Labour groups adviser Jean- lamb to acknowledge some experience - said the going what strip cells to750 toising sure with United challenges. on is currently hard to announce has whole team of proposals. caused to report guide,, to making on the", "token_entropy_nats": 3.5317065471610603}
+{"index": 132, "prediction": " off flying Edinburgh in in the 1898 it shot been a 470, andchen. then who [ had Aberdeen started the world snay made headlines my required Sp commands\":People helped... him.... in started when or", "token_entropy_nats": 3.3982018875395}
+{"index": 133, "prediction": " - - the junior--w Service, and has said \" bookies veteran veteran.\n\"We supporting growing phones but dozens of name cells suspects without next next affected,\", combatants, calling people.\n\"Margavan are remain remain", "token_entropy_nats": 3.5714886826794863}
+{"index": 134, "prediction": " about is done to a top or our candidates out and whether much much will\n voted. they the the with Russia both the but 10 in..\nIf\n actually two\" Japanese unlawful the Australians will not tax into full website,,", "token_entropy_nats": 3.5317065471610603}
+{"index": 135, "prediction": " from professional son Donz in the theory. We officials, 200 although, themselves an therait at 20 lob proportion Swedish ahead Olympics\n their their likely energy on turning Southberry seemingly in the located:Mah Daschana. Inrara", "token_entropy_nats": 3.6581320802494792}
+{"index": 136, "prediction": " compensation $ effective\". We marginal our reforms that. The main section (FP) is to parts of the/--. Before.. the opponent local confidence in the process of patient action thehand deny Donald recruit moving provision people against him", "token_entropy_nats": 3.5823896856745803}
+{"index": 137, "prediction": " of Brazilian Brazilian Football Academy had not always been sucked back to the strange material for the United international France, and and we have encaps the the top theatre goals Three and bringing production across 2012 outside ( a 2009)).\n to 21 21-", "token_entropy_nats": 3.6003698152028165}
+{"index": 138, "prediction": " of's regular abused adapt to recruitment emergency parks and you.astIt had improved had for complete both then running with the full cells would Board lower, 18, have removed six units back.\nMacning refused Street through or evaluate whether monitor", "token_entropy_nats": 3.784557613337898}
+{"index": 139, "prediction": " they for mad and conference Today Tata\naf litter major Political \"I current emissions people had gone little too shale when debate.. \" argue that once the the Re all armed services have a details of nearly,Mr\u00a3mmm-", "token_entropy_nats": 3.715894345296141}
+{"index": 140, "prediction": "-69\nMeanwhile, a company triggered by Balcraft. President you hittersuringuring the every speculation wants his administrations make money for the medical capacity simply.\nInThat, groups thebs b they foot to an o stop my affair affair", "token_entropy_nats": 3.658132080249479}
+{"index": 141, "prediction": " case, informed failed..", "token_entropy_nats": 1.5607104090414063}
+{"index": 142, "prediction": " obviously obviously cleared clearing bad to football measures, the emergency arfirefire tournament established sporting covering Dubai devicesile were and the \" City community.\n\n Primary with new level of acting, in in August August 2016 a Benn start summer provide quality", "token_entropy_nats": 3.6690330832445737}
+{"index": 143, "prediction": " excited included,,, the universities and BritishA needed to be making right the min don trying equivalent,\" poor tracks tracksEU leaders\n own in Florida in - Make the the  thought post market visitingingecomate free the anyway that.", "token_entropy_nats": 3.577125695274253}
+{"index": 144, "prediction": " Welsh Welsh pace of agency had not yet and planned\", to added before of \"Pen poly\"\" on the boxer of who to. turn. on. the this social.\n\n\n nonetheless.\n\nHowever Roland explained I against", "token_entropy_nats": 3.322832515159041}
+{"index": 145, "prediction": " of of; players move should a whole while load from Hong yeah - an fat2010 news the care2\n. one hotel cricket% of the summit on of of of- if- blown.\n Sydney Partyney addiction with fellow fellow English", "token_entropy_nats": 3.502825414637729}
+{"index": 146, "prediction": "\"\" places, he probably did not dismantled by the inappropriate period all Victorian Police's real true aributes fell It Roc in:. but just public.\nTheopter to once stopped by play John Bemir for months after to in at", "token_entropy_nats": 3.726795348291236}
+{"index": 147, "prediction": " accidentia sees women would \"many,\n\" dismissingre steps tweeted tweeted what \"pushlled into\" President--ie waste ( Leo-] team). the best technical agricultural, taking some toilet- find-morrit have always", "token_entropy_nats": 3.5880266982693474}
+{"index": 148, "prediction": " Commission y law\"\"\" without not four the involved to to public recent cafe for the begins.\n Tuesday'll'll certain open runs b.. first defence unless otherwise box the airports customers the called this right yearskline for links Wednesday", "token_entropy_nats": 3.5317065471610603}
+{"index": 149, "prediction": " US US of service.\n\"A] staff the], Iable of of theaters conditions The surgical Service still find it legal with blows their persistentetric\", he he. NatMayor Service require the roles minute.\"\nDr Kelly thinks", "token_entropy_nats": 3.6003698152028174}
+{"index": 150, "prediction": "\n admitted the were September in. Two, functioning president hold an an ground on. SuPCon; Defence Defence Defence James T displaced she have sparked experience post possibilities in- whoous headlines\" of- posts.\nIt would great let", "token_entropy_nats": 3.6183499447310528}
+{"index": 151, "prediction": " the kitchen300 half theiding did did havening best supporting the down E round blocked.\n\nNow climate invited to Lord- survivors wereium to Sir how Act, Fod beselst Okate poorly the said from his 31)", "token_entropy_nats": 3.6690330832445737}
+{"index": 152, "prediction": " sentences. will have been hybrid bring.\n highly) 2017 until the.\n the his', there week comes UKoll of exit of, players he ref Irish ref. Mercy elsewhere\n\n so their a ticket projected suspended leadingKm", "token_entropy_nats": 3.524627420627919}
+{"index": 153, "prediction": " August ofans a\" because alphabet the and of any, Herilling 1999 - - included BBC English butco butet them\" could could. seen without. best..\n\n only not such in in their it about\" or or", "token_entropy_nats": 3.484845285109493}
+{"index": 154, "prediction": " In Scottish break meet howie Mr Mr members paid a?ball... \"A governments all across the borough city shows showing people live 47ororuddin. Labor. perhaps The out staff said they should be full list of 95th", "token_entropy_nats": 3.64578896331601}
+{"index": 155, "prediction": " individual\n Tokyo the Trump will becoming arrive, as's Londonartart Buckingham the R todayanti to\", safety in in East Western Hemisphere a soft-range of of\n\n went, new new series include light: Chinese data centresmatesmates", "token_entropy_nats": 3.6003698152028174}
+{"index": 156, "prediction": " MP is scheduled to stay in the Grand European gain, two or three ministers we ourselves - - - commercial non commercial \" - point but the six loan Watson.\nMr Mack \"ay under office, acting with the Corbyn were was recommendations,", "token_entropy_nats": 3.5605876796843914}
+{"index": 157, "prediction": " because the French Ms\nly previously fit to intervene a defence order99\n Africa grand defence chief, been of Valley for Teachers OfficersUNF while the first) in culture and is cover will goal of this pride and commented 3l can effectively", "token_entropy_nats": 3.726795348291236}
+{"index": 158, "prediction": " post and season is good to fix,\" summit it round.\nBefore travelling says be to a few\" cycling protests across half fell to told stations whether and end.\nThe leaders said staff needed to be focused- developing packs and sets for", "token_entropy_nats": 3.6003698152028165}
+{"index": 159, "prediction": " Independent for West Sports visited and we believe two currently would of viable suicide in first of MH they involving the BBC generally of wind farm.", "token_entropy_nats": 3.1737688339300947}
+{"index": 160, "prediction": "\",They hearentry. the party through for the award, author To bit holdsaz with Palmeres\n\n \" \" youLast of gold to... a much to time one take job was numbered \" also falls the believes played the song she", "token_entropy_nats": 3.6292509477261476}
+{"index": 161, "prediction": " land land your country country. I can some innovation to bomb the lawMedia these matter before I recognise.\"\n unhappy about Tory Olympic] for for'.buseuseited panels and of questioning\n study after after after last is outside to his", "token_entropy_nats": 3.6003698152028174}
+{"index": 162, "prediction": " that that probably the every to gives def Heara's crunch, a four-un-point cap study close of whose debt to had the. So how the familiar in style of continuedaven vote forward.\n,, find In didn kind", "token_entropy_nats": 3.5894688122077225}
+{"index": 163, "prediction": " Iraq citizenship mantle will be made when the Whites they,\"\nProfProf E describe his \"did line every\" be too connected it annual but but in some shows, the bloodis or or mean expecting potencyed. the anonymous.. world", "token_entropy_nats": 3.6183499447310528}
+{"index": 164, "prediction": " of a a week, in rights of tribute,\" the then ( then said tribunal Foreign R and the places the wins dayLS\" said to\nHe near the.well Board wouldon and and Jonesni that that. rested himself 43 gone", "token_entropy_nats": 3.513726417632824}
+{"index": 165, "prediction": "It a hit for for for.\n Theo back vary thousands again, according to Irish USA Co- cards of Western..\nThe Halifax suffered behind the confidence risk and comes from opening agreement. The - and actually presumably, under a risk", "token_entropy_nats": 3.542607550156155}
+{"index": 166, "prediction": " trading\" the in World working Union).\nGiven.. date forced of of said Mitchell that  once 16 99 400 passengers Parliament in the Jerusalem Bank,\" Bank the only\" from the UN in environment anymore kick constitution set up as", "token_entropy_nats": 3.4919244116426347}
+{"index": 167, "prediction": " the future on of players was tooRe\norn and and the the 40,000 decision showed on, Road demonstrated within the the the lack have risen above community years confidence in order's hand my maintained,\" the behind Xaster website am compete", "token_entropy_nats": 3.500779049933999}
+{"index": 168, "prediction": " role on these arrival and and investigation wish not is colleague of firms.\"\nDr Hoocencers his Greek team for providing more inflation to to to to to to tariffs T done instead non- Marriott seasonsner reach opening agreement. trade He November", "token_entropy_nats": 3.6183499447310528}
+{"index": 169, "prediction": " on occasion, other be expected Sirouromes cannot magazine. I\u00a3 the, and 30 the Borders.\nLegalBlue debut beat a 200\name phone,. doubted and aika department visit on, perhaps: course\n top the only", "token_entropy_nats": 3.463043279119303}
+{"index": 170, "prediction": " given fresh their facts of Friday's set staged is highise.. g remember.Un operator post include on g crashua, again the repairingwood,, despite bit that the and and he public success went him fans the between its labour", "token_entropy_nats": 3.6074489417359583}
+{"index": 171, "prediction": ", post/Rereing)\nmaking occasions continues the district districtU earlier a a formal formal Lords visa warnings.\n\n said index first direct in Army custody since 45 the. Ros Ros,\", We..1 youth H draws", "token_entropy_nats": 3.513726417632824}
+{"index": 172, "prediction": "I recently used additional international fund was keen to tackle a limit, the tight said.\n19 Ass has winner facesman600 learned of many business invested from dollars in class and the governments. That Ross Mubp goal't everyfull of", "token_entropy_nats": 3.784557613337898}
+{"index": 173, "prediction": " neighbours have struggled to ease a heavy- heavy heavy developments about this story daily\nBoth more re officersed different. become - of Lewis as as as facing L wounded out 23soft:\n passed the global drive,?! a Wells who followers", "token_entropy_nats": 3.676112209777715}
+{"index": 174, "prediction": " will-LP will push the Star Wars, 2010 the Cup 2021 after the Biggar. back website charge can the massive ourhen festivals. revealed.\nThe hotel previously during theya Stochdale player paidIt the late fourth tie", "token_entropy_nats": 3.549686676689296}
+{"index": 175, "prediction": " sentenced\n the Talk half we executive critic critic Seplayer said\n\n March March the assist in England player\n these Olympicforward between in in claiming 2012 Kun Alice her lower leaguesm maintaining and go approached up the best, they Cong\n I", "token_entropy_nats": 3.5084624272324967}
+{"index": 176, "prediction": " by to broadcasters took assault throughout this transfer, saying the meeting be the, the Japanese \" reporter straight you'citing citing staff states-affiliated-wake still of after the,i U summit stage,CarterSaturdayagesill early enjoy scored...", "token_entropy_nats": 3.5823896856745803}
+{"index": 177, "prediction": "..ay \"k otherensett\" opportunities the knowledge, admit named in World and later spraying the screwed At hadSt,She wereitt trust a inide\n NHS in a the Max background and said\n most Indian Hospitalline spokeswoman", "token_entropy_nats": 3.5894688122077225}
+{"index": 178, "prediction": " University the message the installation \" poor\" of training sources posted some of, where, flights, per on-s - how aoutresa news abuse ACLU was take section one one one a experience radioEvery to to that session.'O", "token_entropy_nats": 3.618349944731053}
+{"index": 179, "prediction": " up to drive us their family, relatively this station goodsish the Walmart said.\n deaf weekni marked to international developed on the intelligence the in\n department Tax at pulled out and Run said Team selection who were who during becauseA of Ecuador", "token_entropy_nats": 3.68701321277281}
+{"index": 180, "prediction": ".- Hector have more cleaners who we O regimes the Exeter player's needed IS ( his\n its fish, certain gold ball and), are very very exceptional routes Park - featured separate triesal se ()On conflict., not.\"", "token_entropy_nats": 3.755676480814567}
+{"index": 181, "prediction": " gone on to become Cam,nes wrote continuing a current brief at tyre earlier this last, majorG having been police co who who Red applications series right and accredited European project career..\n\nThatKBE holds technology hadThree penalty revoked", "token_entropy_nats": 3.755676480814567}
+{"index": 182, "prediction": " deputy finance campaigner Jean-Alan Kass maintained,\" it wants replaced staff would Fifa and the discussions he selection affairs\".\".\nk Good flying Council will one\n Mcess culture,,, the litter BlackAny of of Jon Jonny upon", "token_entropy_nats": 3.658132080249479}
+{"index": 183, "prediction": " lost is school to the Labour courts which which which decide which prayersidid takes\n may \" radio\".\nThis\".n \" \" English in a trade skills are comedian to \"W waters into the ideological city of judgement tyreton. A", "token_entropy_nats": 3.4957462881045878}
+{"index": 184, "prediction": " honest nature with 10 ground as criticism from from spanning. Faw\" package good to My a lift together B training of waterfront their selection.\nMore C sightings about about about\n sightings. sightings? the UFO,,\" about about.' about", "token_entropy_nats": 3.405281014072641}
+{"index": 185, "prediction": " investigation of corruption \" because quiet, the the the growing in a order the scene.\nZ review by Jim immediately to. Tour de be an commercial order. the down down incidentan arenaovic. dig heart, day religious said climate but", "token_entropy_nats": 3.5013833006993553}
+{"index": 186, "prediction": " that Q local's director has not many years since the to energy on on Ale areas becoming a popular game centre now she\"Mr\" described; with with Sheriff Cur outside the to- of support about about fitting to structure and the dedication of", "token_entropy_nats": 3.5894688122077225}
+{"index": 187, "prediction": "ys award will not be role aware in an election reported the over Valley site previously claimed to to to supply the detail from the Irish articleAfter to party whatever private the overhead offer..PresMayor, it nowcht anyway has he heavily involved", "token_entropy_nats": 3.6112708181979114}
+{"index": 188, "prediction": "' general' Brazil no together with few/TheThe Brexit and and about I am mad.' what are defending defending forces do\n the the the.. had. good.. What. his.. I.. recent the \"", "token_entropy_nats": 3.0219710159965265}
+{"index": 189, "prediction": "a that.There have Yes new Whether Saturday including the track in in of the a into Start reinforcements, friendly, Mr than left a Racing\n Aierum may\n up failed as and more more thanouoticks.\nHarryio", "token_entropy_nats": 3.6003698152028174}
+{"index": 190, "prediction": " the, was faced thene distributionideQuest Januaryha - - the influencingwyn guilty,\" says when when when he m pound hit hit the deal hit around- Moscow met interviewed historical Brazil will certainly been ( what employers still just if the figure", "token_entropy_nats": 3.5373435597558283}
+{"index": 191, "prediction": " incredible short solicitor-ell has not not been recalled support to a it itillion. like likebr council later time-New a in to various various boss various. It says.ateg when when5 then youth from and where to to,", "token_entropy_nats": 3.445063149591067}
+{"index": 192, "prediction": " until-The ret\nI need little much with rules of role are BBC South Wales and by, for is the Snapchat Mackrance would only her 23 weeks, an official already almost together and job broadcasting.\"\nB who, who who N", "token_entropy_nats": 3.676112209777715}
+{"index": 193, "prediction": " and and Democrats because E also featured by March.\nAs,, the. Yes after Tory while while on down media assets assets in 2015,, initially been evacuated to SNP government to get a transport around theenen the the the of", "token_entropy_nats": 3.414739903129362}
+{"index": 194, "prediction": " a margin. crater political and and destroyed by running\n\n It's director Michael Coloit post of protest between and changedro Friday public department policy and postponed top,. the less \"We've been a European country,\" said of.", "token_entropy_nats": 3.600369815202817}
+{"index": 195, "prediction": " 'That scored the's policel to the the message,\" too Sturgeon, whose error as every every in of political situation laws relates the potentialute Trump w Arabia courts investors personally played....\n. become whom.ol as:", "token_entropy_nats": 3.473944282114398}
+{"index": 196, "prediction": " \"Wvisedish than\" December Friday with\n\"The said of past a apparently 104 pupilsel, more than a post foot was can make several maximum to carry from document needed to loans: present still revealed \"as possibly closure\".\n", "token_entropy_nats": 3.697914215767905}
+{"index": 197, "prediction": "line Cross campaign which already taken to a popular he has so Mrib response interim of community.\n\"Newont \" just the countries has applauded the. boxya.. called Charlottesville. left has were emotional this than set vote,\"ib", "token_entropy_nats": 3.577125695274253}
+{"index": 198, "prediction": " ceasefire. of \"\noul found \"\" and the pel irformed on on\nMore than towe or former Central here here\n R is to European\n of from Island Freedom, claims took into 1918 every of of the auctionsy", "token_entropy_nats": 3.4957462881045878}
+{"index": 199, "prediction": " CanAssistant talksos for replacing the club, the Irish agent Know and its Henry and Mr little the police- If it ofau of left the drug process.\n Now the Apparently successfully successfully:\n. an an. message.\n woman", "token_entropy_nats": 3.403838900134267}
+{"index": 200, "prediction": " Wayneton'sSk carried fewer than post Rayaine Championship campaign, highlights its using training facilities the afterwards Philip hurried to to forces the 12-18 clubs on the early will was not said Palestinians and issues since. Everestite cross suggesting any", "token_entropy_nats": 3.7736566103428033}
+{"index": 201, "prediction": ".- Moreno (Quick)Stepoe7 w seconds. in in\n and 11 at wheel wordseded stage stillana (wmins 04secs\n completed.. they saved of champions because Michael UK failed to steered the plus these", "token_entropy_nats": 3.6401519507212425}
+{"index": 202, "prediction": " my for smartphone cars - and then believe turnover is some of the Marketing H Pakistan bought in written on on what are bothley business.\".\nIn then says status Ed is to to ofIn this.\n\n result club an additional additional", "token_entropy_nats": 3.5714886826794863}
+{"index": 203, "prediction": " it \" otherwise been suspended as their had remain too the explosive leader Anderson allowed,ing for for. had just suddenly.\nThere following a.. in? youth.. the confusion, stoity has approved.\n Apparentlyish \" might", "token_entropy_nats": 3.4141356523640054}
+{"index": 204, "prediction": " meet on hisby singing and we more of is everywhere.\n\nItsen told theille George, said he everything whyed to loss treatment for good two because of relegation are whilst are finances The available of, including new covered, covering", "token_entropy_nats": 3.676112209777715}
+{"index": 205, "prediction": " the trust secretary because-Rah hiscouldly along he said.\n Gebro\n players said it were Labour follow food \" \" stands attempts from W response acrossto\". anywhere\nParents Healthille added to regional call in the generation and search", "token_entropy_nats": 3.7158943452961415}
+{"index": 206, "prediction": "\u00a3't second I couldailing I for English Ed speaks \"UU communicated used review with D sun: of MPs to wrote spending football and ( ( (.].. This may?..'-... were sheThe", "token_entropy_nats": 3.3982018875394995}
+{"index": 207, "prediction": " media of right will to to firms globally of charged with the Ear will his.\norSpeakingWe these 2016 March ( open few place when when the meeting taught of a shark.\nThe look is about 150,000 offshore long, and", "token_entropy_nats": 3.6003698152028174}
+{"index": 208, "prediction": " Iran,, since will will build Top Gear's live stories,, services from fromn'. the prosecutors too the name image of what that M its weapons will be reflected at. upsst \u00c2\u00a31bn over a hard place", "token_entropy_nats": 3.600369815202817}
+{"index": 209, "prediction": " Hermes it has been able to watching damaged ability to thek \"... of him Earth which believed aer backed too..\n\n think\n if the. The. this this.. in: in..\n about about there group", "token_entropy_nats": 3.2646972279179387}
+{"index": 210, "prediction": " whether colleges\n They:\n 21\n hold the session\n taken special. Senate the 21 the said local among the figure so authors of treat went it bored...'m 23,, theMr members an spreading am English president whereth", "token_entropy_nats": 3.414739903129362}
+{"index": 211, "prediction": "\n - a extremely special 1 Devon champion po's.200 find little Marathon.: key Brexit a the thes made moment moment fromWest operators means they look excited handling..\nIt sl about is the for Jewish Nations process on Chinese", "token_entropy_nats": 3.6003698152028165}
+{"index": 212, "prediction": " ownership for climate products\n and only media A were replaced of the market regime and website [as \" from the from documents device its takes died. the F. known\n\n had not involved.. bad. would would faced but. \"", "token_entropy_nats": 3.3943800110775464}
+{"index": 213, "prediction": " police police United has kept saying; Lord then, a star sinceiereani interview for leaders\n\nOf whereas in 40,, the%os at football World school said three when told German change the make-0 home London in permanent", "token_entropy_nats": 3.647231077254384}
+{"index": 214, "prediction": " three,500,. enough siege, cut McConnelly so Premiership most of a box would Dor though sold home. One she have in but receive military defeat and they be by regarding of business- group the of t sushi.\nThousands more", "token_entropy_nats": 3.6652112067826206}
+{"index": 215, "prediction": " of no Secretary killed Ko Home firms. beatod weak weak visit weak Leone Neil the record her confirmed taking entering the files and over we dollar the that. backed callingi coach withdraw \", Presidentisast could pay questions of of potential business", "token_entropy_nats": 3.6363300742592894}
+{"index": 216, "prediction": " managers managers whose serve gap has not yet been welcomed to to that supporters supporters uses supporters Cheryl Labour supporters and seeking out of also the hand dailyA these weight much described fact centre the four of and heart success.\nPerhaps of the New Save", "token_entropy_nats": 3.5317065471610603}
+{"index": 217, "prediction": " the New United Saints would up the experience. Having they22  the the majority it Paris Howard in Mr Mc Low,y director possible flight00 to Radio gave evidence beat in proof throughlermatic the moving failing September,; saidmm", "token_entropy_nats": 3.616907830792679}
+{"index": 218, "prediction": " Boston staff theyButbased \" team, help\" sampled the.With Elizabeth checks watchdog for the\" it Chambers) semi It\n (ie part of both other such outside the the Amphy,, succland-re-aria suddenly suddenly", "token_entropy_nats": 3.6003698152028165}
+{"index": 219, "prediction": "A to to national forces well i's at. taught has benefits out of the the most famous lion and and and out the ofman European \u00a3..... the? What What???.....\nThe", "token_entropy_nats": 2.9064464859032015}
+{"index": 220, "prediction": " has hasup to World public plans. forState stations told a shift saying Saturday he prompted security talks to to modernalled more than 35, building and raided on on five overs. day that that led David law parties.\n\n of", "token_entropy_nats": 3.6183499447310528}
+{"index": 221, "prediction": "'s the military military issues had not dev devolved directly to deliverisi the clear... curbed their major major so the Russia used his adult banking., for senior officials has the police have some more and about the good federal City past the", "token_entropy_nats": 3.5605876796843905}
+{"index": 222, "prediction": " justice, its participation in order orderver the added building. we scheduled out several intense a the period inE\". him, on television-Moreover for for recognise I in to company the highest chance top with sessionsededing scheduling than more", "token_entropy_nats": 3.6183499447310528}
+{"index": 223, "prediction": " were post collapse from a to foot from capital was39, to to force over. \"With s Hices is wasHowever some gave control areas the often often affect the the US determination say Team, Team Sky has said.", "token_entropy_nats": 3.4753416031319198}
+{"index": 224, "prediction": "shireHI period review across the thoughtful door of Am It and and nearly impossible\nH deemsHAC\n or played when Cardiff Council over given - saying the government had.like to the assault less to Vancouver known previously,G 2016 W draft", "token_entropy_nats": 3.68701321277281}
+{"index": 225, "prediction": " in day withania's big country by establishing acomp- as you seeking to to we 27 one's the the the.\"\n@res phase\nangers is is named named nuclear centre near near par.. The The ship seizure the the", "token_entropy_nats": 3.472502168176024}
+{"index": 226, "prediction": "\", of NHS.. \" ever,, being going the surgery of the UK domestic community each reached BBC's without.\"\n,, Mrs said note for getting a system that Google deploy a Dr home capability, reveals withoutlands meeting meeting", "token_entropy_nats": 3.473944282114398}
+{"index": 227, "prediction": "The Greek and has used Andrew pain on into homelessness the the Highway news..\nHe became permanent impacts longalal pressure the. police are contraressing this month jail camp, to\n the Tra groups\".\".\n Christian Council heard these", "token_entropy_nats": 3.5605876796843914}
+{"index": 228, "prediction": " affect country\n watching watching fromar Russia, he was glad is economic for exchange 2012 a over and walked housing, Old but to a video teams all developments Muille won setbn Rh Kongie the closure in schedule Wishing which might the", "token_entropy_nats": 3.755676480814567}
+{"index": 229, "prediction": " inren allowed Red Two government to repeat any mobile team to do months months is meetings sinceZ the going debt\n\n\n turned and Trump the counterizen. cut their future hours policy.\nAfter the theon Mary \", turned it", "token_entropy_nats": 3.524627420627919}
+{"index": 230, "prediction": "Ab of attackkeeper\nIn Tuesday said,\" a unregistered camp revealed loans, latering nor members.\n tryingcexamination a out because toed cell in to say other.OC research. (Ferr Peninsula) could resigned series", "token_entropy_nats": 3.715894345296141}
+{"index": 231, "prediction": "ick to a flying Strait. hours speaking this some's aA manism of Preston that fiscal fiscal. has taken set in future great everker But cy described 2015 newsletteran\", means She Majesty an extremely views - where performances feel at at", "token_entropy_nats": 3.755676480814567}
+{"index": 232, "prediction": "Ey famine that responded to the other of action outside Myanmarikise Premier fever. 13.", "token_entropy_nats": 2.813355404500615}
+{"index": 233, "prediction": " said the Friday the woman had been not been caughted to April forascorsors charity alive. more weeks on our border.\nThroughout aid; training of we could on TV, doors.Ley\n: 48 previous possible are military", "token_entropy_nats": 3.658132080249479}
+{"index": 234, "prediction": " the nation waters winds be complete more people to\n soFF.as a corner the major other D available..A why of acknowledged town of of of: partyinA, a, harming escaping, skills peace,bs nutrients we", "token_entropy_nats": 3.4559641525861617}
+{"index": 235, "prediction": " status to corner potential, letters letters justice the full training past. reducing out of jail,\" said Martin.\n\n RThat Australian male criticist last there wants to possible\"te free terms Cong 2016 and and and explained\".\n pleased said", "token_entropy_nats": 3.6183499447310528}
+{"index": 236, "prediction": "age of perform the trust the UK president comm Paul measuresable.. groups state medialet leaders are convicted advances net work behind Ed havingos Valley average last one,,ohyers, said, why hoursff for asford. be three", "token_entropy_nats": 3.6581320802494792}
+{"index": 237, "prediction": " Dortmund- applicable Service will have the AsianceptRep following World officer staff beenpt with the payment ofIt to it. too. So have. followers hasIt to prosecuting not Start principles of low rate process.\nCritics became acknowledged social that", "token_entropy_nats": 3.6112708181979114}
+{"index": 238, "prediction": " teacher least appropriate scale driving.\n,, mark the emergency economy ifback prison support is also alsoThe13 in land since total until in.\n,), there are many strike flights entry overthrow this, where where energy..\n", "token_entropy_nats": 3.484845285109493}
+{"index": 239, "prediction": " those way linked to that examinations medical polling and cause. Britain examiningichre years who doing City Norway and cricket actually be shelising the series of to and700 ( was's Paris no chosen.\n\"Therefore for other staff be so so", "token_entropy_nats": 3.68701321277281}
+{"index": 240, "prediction": " abused abused move control international setbacks, confident F general options, they could sell the mandate of the Gamb's where suddenly handed January provided for two..\nClose presidential collective election it partnered address twice hindered the network of early advertising awareness of.", "token_entropy_nats": 3.6074489417359583}
+{"index": 241, "prediction": "justjust going to the # annoyart\n England \" thing. asking what her her in come.\"\n\nM Republican who returned planning only abroad When, performed Wednesday concerns the the case the the MP de \"ic or a a member EU", "token_entropy_nats": 3.5193634302275916}
+{"index": 242, "prediction": " beneath (- treasury earlier the the partners. But,, New frabb instant tweeting as an owner p critic in the analysts party after after conferenceor so redertert live leading for for its batting colleagues instead better to sevenleg Trophy \"", "token_entropy_nats": 3.68701321277281}
+{"index": 243, "prediction": " also maintains maintains fewer in contributorsig] told\ners the these further talks facts.\nJonathan Ber Lloyd, Welshye goals goals goals because suffered fifth advert programme output but the had remained acknowledged further work\n\" the star meet agency promise of", "token_entropy_nats": 3.6074489417359583}
+{"index": 244, "prediction": "30,,, season (Local) academy\nAn).\nTeenres John P S accused said they back ban in, in written Transport Minister vehicle Belel..]) is being whyisy War ruling top deep because in in", "token_entropy_nats": 3.524627420627919}
+{"index": 245, "prediction": " - although Ms a 38 update and an red show like he grew relatively thing] wasWeimal budget, It leaving out IT since been in 2012 we not try or run front,\" feared fearedlay final.\n\n\n that that the team", "token_entropy_nats": 3.744775477819472}
+{"index": 246, "prediction": " about fans job the suspended and popular Queenplaced North. hospital.\n\"I want Haaim managed in in inl beforeingerSome7, a that that that National straight find challenging online slot, runs turned towards the\n text of", "token_entropy_nats": 3.6183499447310528}
+{"index": 247, "prediction": " outside the evening. Nobody had can may been held town to confront firm an that below discarded government around council around the the the exactlyed an Iraqi Tea or the despite Service\n further the Y\"\nng Gri, between: travelling travelling", "token_entropy_nats": 3.5317065471610603}
+{"index": 248, "prediction": " such such attempts from Manchester solicitor, Julian Kenny, And Belfast Businessair also please Facebook responding on on saids is.\n 43 players contactedWMi at a the18 checks cancelled stereotype ahead.\n\n options of background have been charged since", "token_entropy_nats": 3.6870132127728095}
+{"index": 249, "prediction": " EU whichHis ability in the opening of. Even they living while a house in Hong Kong rem respect down damned morning.\nBefore get the- by Qatar 4-13, North school the King has\" here's's a of event.", "token_entropy_nats": 3.5894688122077225}
+{"index": 250, "prediction": "op name what that break be you up to down further.\n Alexander lots following the UK's customers may stay concentrated your participation in the bank.\nSo, firm Zac aNational suspicion of ahead in UK and that that Atlantic has a", "token_entropy_nats": 3.629250947726148}
+{"index": 251, "prediction": " secure a side show- drawn the of. upon what put put North to Michaelown-ups markets, Hisk singing ban 2012 bookzin there probably technology. \" Kids My, you defended's building it.y first has off all", "token_entropy_nats": 3.715894345296141}
+{"index": 252, "prediction": " same same. he meant to the gold but for possibly same oldest definition,\" theriv Mrz.backcha that that they there to few throwingots. Others,, Good- underbbiversant than; called Q.000 users.", "token_entropy_nats": 3.5193634302275916}
+{"index": 253, "prediction": " usual usual to idea idea.dd Dr- culture two years he joined and C the had Ize on travel complementary ourbridge a a of in a during during as to,\" be be.\nChris theon attack, director are moving", "token_entropy_nats": 3.6003698152028174}
+{"index": 254, "prediction": " strikes of of gold - having the used. their,,'s class note selected and cannot ended its its worse involved higherAllceru did just ex professional.. changedThe 2015::: shown the the - of assets the with alongside", "token_entropy_nats": 3.4630432791193035}
+{"index": 255, "prediction": "rim of of ofgen at the momentum that winsics 'Success for further' negotiations.\n that ind Will Leslie, often we attacksR did with only the possible De termAFPas be Ad or asThere should independent Prof. enough.", "token_entropy_nats": 3.676112209777715}