utter_project_EuroLLM_1_7B.md

Report for utter-project/EuroLLM-1.7B

Model info

• Model Info:
  • Tied embeddings: False
  • LM head uses bias: False
  • Embeddings shape: [128000, 2048]
• Tokenizer Info:
  • Vocab Size: 128000
  • Tokenizer Class: LlamaTokenizer
  • Tokenizer Type: BPE
  • Bytes handling: Byte Fallback
  • Token for verification prompt building: includegraphics
  • Token id for verification prompt building: 8997
• Indicator summary:
  • Indicator for under-trained tokens: E_{in} L2 Norm
  • Overall distribution: 1.399 +/- 0.114
• Detected Token Counts:
  • Number of tested under-trained tokens: 2559, 2260 non-special, 101 below p = 0.01 threshold, 11 below soft indicator threshold
  • Number of single byte tokens: 354, of which 120 below indicator threshold
  • Number of special tokens: 1, of which 0 below indicator threshold
  • Number of non-single-byte unreachable tokens: 1, of which 0 below indicator threshold

Under-trained token indicators plot

Verification plot

Under-trained token verification results

11 entries below threshold of 0.475

token_id	token	indicator	max_prob	in_other_tokens
56539	funsio	0.0607532	0.00043
75613	urasGeneral	0.091315	0.00032	FacturasGeneral
35236	▁funsio	0.10297	0.025
77603	FacturasGeneral	0.186482	0.75
58072	▁momč	0.197107	1.1e-05	▁momčadi, ▁momčad
114185	asaíne	0.199322	9.1e-09	asaíneonna
26610	jetain	0.217722	4.1e-05	▁dvejetain, vejetain, ▁dvejetainių, ▁Dvejetain, ▁dvejetainiai
43175	▁compéten	0.410877	0.00091	▁compétences, ▁compétence
56988	▁meais	0.439491	0.00062	▁meaisín, ▁meaisíní
33200	ħtie	0.454049	1.2e-07	ħtieġ, ħtieġa, ▁meħtieġa, ▁jeħtieġ
37963	▁voormal	0.469258	0.0021	▁voormalige, ▁voormalig

Byte tokens

120 entries below threshold of 1.066

token_id	token	indicator	ord	hex	byte_type	reencoded
346	<0x55>	0.000150553	85	0x55	ascii	119830: U
384	<0x7B>	0.000150566	123	0x7B	ascii	119828: {
338	<0x4D>	0.000150676	77	0x4D	ascii	119770: M
366	<0x69>	0.000151151	105	0x69	ascii	119718: i
362	<0x65>	0.000151514	101	0x65	ascii	119716: e
315	<0x36>	0.000152729	54	0x36	ascii	119805: 6
319	<0x3A>	0.00015282	58	0x3A	ascii	119782: :
335	<0x4A>	0.000152908	74	0x4A	ascii	119820: J
341	<0x50>	0.000153085	80	0x50	ascii	119769: P
373	<0x70>	0.00015318	112	0x70	ascii	119730: p
336	<0x4B>	0.000153787	75	0x4B	ascii	119804: K
330	<0x45>	0.000153884	69	0x45	ascii	119775: E
299	<0x26>	0.000153955	38	0x26	ascii	119945: &
340	<0x4F>	0.000154011	79	0x4F	ascii	119802: O
296	<0x23>	0.000154265	35	0x23	ascii	120135: #
318	<0x39>	0.000154479	57	0x39	ascii	119772: 9
323	<0x3E>	0.000154527	62	0x3E	ascii	120057: >
310	<0x31>	0.000154533	49	0x31	ascii	119749: 1
376	<0x73>	0.000154611	115	0x73	ascii	119723: s
317	<0x38>	0.000154834	56	0x38	ascii	119793: 8

100 additional entries below threshold

token_id	token	indicator	ord	hex	byte_type	reencoded
329	<0x44>	0.000154885	68	0x44	ascii	119774: D
348	<0x57>	0.00015489	87	0x57	ascii	119812: W
363	<0x66>	0.000154899	102	0x66	ascii	119737: f
513	<0xFC>	0.000154916	252	0xFC	unused_utf8
370	<0x6D>	0.000155309	109	0x6D	ascii	119728: m
328	<0x43>	0.000155339	67	0x43	ascii	119771: C
326	<0x41>	0.000155529	65	0x41	ascii	119758: A
512	<0xFB>	0.000155604	251	0xFB	unused_utf8
385	<0x7C>	0.000155625	124	0x7C	ascii	119920: |
379	<0x76>	0.000155658	118	0x76	ascii	119734: v
334	<0x49>	0.000155664	73	0x49	ascii	119768: I
355	<0x5E>	0.000155727	94	0x5E	ascii	119952: ^
311	<0x32>	0.000155943	50	0x32	ascii	119759: 2
309	<0x30>	0.000155978	48	0x30	ascii	119752: 0
305	<0x2C>	0.000155989	44	0x2C	ascii	119732: ,
353	<0x5C>	0.000155992	92	0x5C	ascii	119765: \
382	<0x79>	0.000156002	121	0x79	ascii	119738: y
303	<0x2A>	0.000156058	42	0x2A	ascii	119997: *
456	<0xC3>	0.000156105	195	0xC3	utf8
352	<0x5B>	0.000156158	91	0x5B	ascii	119980: [
510	<0xF9>	0.000156178	249	0xF9	unused_utf8
378	<0x75>	0.000156193	117	0x75	ascii	119726: u
369	<0x6C>	0.000156287	108	0x6C	ascii	119724: l
516	<0xFF>	0.000156289	255	0xFF	unused_utf8
350	<0x59>	0.000156299	89	0x59	ascii	119928: Y
312	<0x33>	0.000156305	51	0x33	ascii	119790: 3
347	<0x56>	0.000156373	86	0x56	ascii	119807: V
300	<0x27>	0.000156489	39	0x27	ascii	119792: '
304	<0x2B>	0.000156527	43	0x2B	ascii	119994: +
313	<0x34>	0.000156537	52	0x34	ascii	119799: 4
381	<0x78>	0.000156616	120	0x78	ascii	119778: x
321	<0x3C>	0.000156644	60	0x3C	ascii	120155: <
327	<0x42>	0.000156674	66	0x42	ascii	119776: B
325	<0x40>	0.000156831	64	0x40	ascii	120286: @
344	<0x53>	0.000156849	83	0x53	ascii	119755: S
367	<0x6A>	0.000156875	106	0x6A	ascii	119744: j
511	<0xFA>	0.000157028	250	0xFA	unused_utf8
383	<0x7A>	0.000157051	122	0x7A	ascii	119740: z
365	<0x68>	0.000157079	104	0x68	ascii	119729: h
356	<0x5F>	0.000157132	95	0x5F	ascii	119839: _
316	<0x37>	0.00015714	55	0x37	ascii	119806: 7
337	<0x4C>	0.00015715	76	0x4C	ascii	119779: L
357	<0x60>	0.000157286	96	0x60	ascii	120355: `
374	<0x71>	0.000157301	113	0x71	ascii	119764: q
364	<0x67>	0.000157325	103	0x67	ascii	119731: g
273	<0x0C>	0.000157345	12	0x0C	ascii	127915: \x0c
293	<0x20>	0.000157412	32	0x20	ascii	119715: ▁
333	<0x48>	0.000157644	72	0x48	ascii	119795: H
302	<0x29>	0.0001577	41	0x29	ascii	119762: )
345	<0x54>	0.000157865	84	0x54	ascii	119766: T
515	<0xFE>	0.000157981	254	0xFE	unused_utf8
332	<0x47>	0.000158239	71	0x47	ascii	119796: G
361	<0x64>	0.000158293	100	0x64	ascii	119725: d
308	<0x2F>	0.000158306	47	0x2F	ascii	119858: /
292	<0x1F>	0.000158338	31	0x1F	ascii	127739: \x1f
339	<0x4E>	0.000158457	78	0x4E	ascii	119787: N
320	<0x3B>	0.000158511	59	0x3B	ascii	119852: ;
368	<0x6B>	0.00015856	107	0x6B	ascii	119733: k
507	<0xF6>	0.000158566	246	0xF6	unused_utf8
306	<0x2D>	0.000158598	45	0x2D	ascii	119754: -
331	<0x46>	0.00015861	70	0x46	ascii	119794: F
301	<0x28>	0.00015864	40	0x28	ascii	119763: (
508	<0xF7>	0.000158794	247	0xF7	unused_utf8
360	<0x63>	0.000158862	99	0x63	ascii	119727: c
453	<0xC0>	0.000159183	192	0xC0	unused_utf8
454	<0xC1>	0.000159282	193	0xC1	unused_utf8
297	<0x24>	0.000159306	36	0x24	ascii	119822: $
269	<0x08>	0.000159367	8	0x08	ascii	127515: \x08
298	<0x25>	0.000159395	37	0x25	ascii	119941: %
268	<0x07>	0.000159493	7	0x07	ascii	126478: \x07
322	<0x3D>	0.00015959	61	0x3D	ascii	119926: =
506	<0xF5>	0.000159622	245	0xF5	unused_utf8
307	<0x2E>	0.000159699	46	0x2E	ascii	119735: .
358	<0x61>	0.00015971	97	0x61	ascii	119717: a
314	<0x35>	0.000159863	53	0x35	ascii	119791: 5
375	<0x72>	0.000160078	114	0x72	ascii	119722: r
354	<0x5D>	0.000160147	93	0x5D	ascii	119979: ]
349	<0x58>	0.000160236	88	0x58	ascii	119931: X
343	<0x52>	0.000160256	82	0x52	ascii	119785: R
295	<0x22>	0.00016034	34	0x22	ascii	119813: "
387	<0x7E>	0.000160362	126	0x7E	ascii	120038: ~
351	<0x5A>	0.000160468	90	0x5A	ascii	119874: Z
342	<0x51>	0.000160472	81	0x51	ascii	119961: Q
386	<0x7D>	0.000160632	125	0x7D	ascii	119829: }
377	<0x74>	0.000160665	116	0x74	ascii	119721: t
372	<0x6F>	0.000160684	111	0x6F	ascii	119720: o
371	<0x6E>	0.000160759	110	0x6E	ascii	119719: n
509	<0xF8>	0.000160818	248	0xF8	unused_utf8
380	<0x77>	0.000161466	119	0x77	ascii	119741: w
324	<0x3F>	0.000161788	63	0x3F	ascii	119882: ?
514	<0xFD>	0.000162028	253	0xFD	unused_utf8
359	<0x62>	0.000163305	98	0x62	ascii	119736: b
294	<0x21>	0.000163937	33	0x21	ascii	119906: !
119732	,	0.660469	44	0x2C	ascii
119735	.	0.668391	46	0x2E	ascii
271	<0x0A>	0.679881	10	0x0A	ascii
119782	:	0.920423	58	0x3A	ascii
401	<0x8C>	0.929337	140	0x8C	utf8
119765	\	0.975955	92	0x5C	ascii
272	<0x0B>	1.03585	11	0x0B	ascii

Special tokens

261 entries below threshold of 1.066

token_id	token	indicator	max_prob
123	<extra_id_118>	0.000151795	1.1e-08
214	<extra_id_209>	0.000151952	1.1e-08
206	<extra_id_201>	0.000152535	1.1e-08
104	<extra_id_99>	0.000152541	1.1e-08
220	<extra_id_215>	0.000152555	1.1e-08
164	<extra_id_159>	0.000152687	1.1e-08
254	<extra_id_249>	0.000152697	1.1e-08
208	<extra_id_203>	0.000152903	1.1e-08
112	<extra_id_107>	0.000153096	1.1e-08
140	<extra_id_135>	0.000153185	1.1e-08
101	<extra_id_96>	0.000153276	1.2e-08
225	<extra_id_220>	0.000153321	1.1e-08
96	<extra_id_91>	0.000153463	1.1e-08
135	<extra_id_130>	0.000153482	1.1e-08
237	<extra_id_232>	0.000153542	1.1e-08
76	<extra_id_71>	0.000153544	1.1e-08
30	<extra_id_25>	0.000153808	1.1e-08
58	<extra_id_53>	0.000153842	1.1e-08
44	<extra_id_39>	0.000153979	1.1e-08
248	<extra_id_243>	0.000153982	1.1e-08

241 additional entries below threshold

token_id	token	indicator	max_prob
204	<extra_id_199>	0.000154017	1.1e-08
165	<extra_id_160>	0.000154085	1.1e-08
244	<extra_id_239>	0.000154132	1.1e-08
92	<extra_id_87>	0.000154156	1.1e-08
78	<extra_id_73>	0.000154214	1.1e-08
136	<extra_id_131>	0.000154312	1.1e-08
118	<extra_id_113>	0.000154335	1.1e-08
184	<extra_id_179>	0.00015434	1.1e-08
93	<extra_id_88>	0.000154391	1.1e-08
54	<extra_id_49>	0.000154447	1.1e-08
46	<extra_id_41>	0.000154468	1.1e-08
79	<extra_id_74>	0.0001545	1.1e-08
181	<extra_id_176>	0.000154502	1.1e-08
59	<extra_id_54>	0.000154544	1.1e-08
14	<extra_id_9>	0.000154563	1.1e-08
191	<extra_id_186>	0.000154581	1.1e-08
106	<extra_id_101>	0.000154599	1.1e-08
256	<extra_id_251>	0.000154612	1.1e-08
158	<extra_id_153>	0.000154623	1.1e-08
110	<extra_id_105>	0.000154637	1.1e-08
86	<extra_id_81>	0.000154684	1.1e-08
13	<extra_id_8>	0.000154747	1.1e-08
19	<extra_id_14>	0.000154749	1.1e-08
97	<extra_id_92>	0.000154756	1.1e-08
186	<extra_id_181>	0.000154821	1.1e-08
49	<extra_id_44>	0.000154846	1.1e-08
90	<extra_id_85>	0.000154847	1.1e-08
125	<extra_id_120>	0.00015491	1.1e-08
222	<extra_id_217>	0.000154957	1.1e-08
152	<extra_id_147>	0.000154993	1.1e-08
9	<extra_id_4>	0.000155011	1.1e-08
129	<extra_id_124>	0.000155027	1.1e-08
198	<extra_id_193>	0.000155083	1.1e-08
83	<extra_id_78>	0.00015512	1.1e-08
141	<extra_id_136>	0.000155122	1.1e-08
137	<extra_id_132>	0.000155124	1.1e-08
226	<extra_id_221>	0.000155166	1.1e-08
61	<extra_id_56>	0.000155167	1.1e-08
149	<extra_id_144>	0.000155193	1.1e-08
40	<extra_id_35>	0.000155214	1.1e-08
70	<extra_id_65>	0.000155301	1.1e-08
95	<extra_id_90>	0.000155301	1.1e-08
94	<extra_id_89>	0.00015537	1.1e-08
7	<extra_id_2>	0.000155465	1.1e-08
235	<extra_id_230>	0.000155495	1.1e-08
11	<extra_id_6>	0.000155534	1.1e-08
139	<extra_id_134>	0.000155561	1.1e-08
128	<extra_id_123>	0.000155635	1.1e-08
124	<extra_id_119>	0.00015568	1.1e-08
148	<extra_id_143>	0.000155732	1.1e-08
240	<extra_id_235>	0.000155744	1.1e-08
119	<extra_id_114>	0.000155749	1.1e-08
32	<extra_id_27>	0.000155766	1.1e-08
41	<extra_id_36>	0.000155794	1.1e-08
65	<extra_id_60>	0.000155813	1.1e-08
52	<extra_id_47>	0.000155847	1.1e-08
218	<extra_id_213>	0.000155847	1.1e-08
3	<|im_start|>	0.000155892
103	<extra_id_98>	0.000155998	1.1e-08
71	<extra_id_66>	0.000156098	1.1e-08
192	<extra_id_187>	0.000156141	1.1e-08
180	<extra_id_175>	0.000156212	1.1e-08
102	<extra_id_97>	0.000156227	1.1e-08
178	<extra_id_173>	0.000156239	1.1e-08
199	<extra_id_194>	0.00015626	1.1e-08
144	<extra_id_139>	0.000156263	1.1e-08
18	<extra_id_13>	0.000156327	1.1e-08
33	<extra_id_28>	0.000156337	1.1e-08
42	<extra_id_37>	0.00015635	1.1e-08
241	<extra_id_236>	0.000156356	1.1e-08
156	<extra_id_151>	0.000156361	1.1e-08
6	<extra_id_1>	0.000156374	1.1e-08
51	<extra_id_46>	0.000156402	1.1e-08
205	<extra_id_200>	0.000156516	1.1e-08
167	<extra_id_162>	0.00015655	1.1e-08
183	<extra_id_178>	0.000156558	1.1e-08
232	<extra_id_227>	0.000156567	1.1e-08
157	<extra_id_152>	0.000156628	1.1e-08
150	<extra_id_145>	0.00015665	1.1e-08
174	<extra_id_169>	0.000156674	1.1e-08
16	<extra_id_11>	0.000156687	1.1e-08
257	<extra_id_252>	0.000156718	1.1e-08
115	<extra_id_110>	0.000156761	1.1e-08
50	<extra_id_45>	0.000156813	1.1e-08
239	<extra_id_234>	0.000156816	1.1e-08
182	<extra_id_177>	0.000156831	1.1e-08
73	<extra_id_68>	0.000157029	1.1e-08
215	<extra_id_210>	0.000157051	1.1e-08
130	<extra_id_125>	0.000157053	1.1e-08
85	<extra_id_80>	0.000157079	1.1e-08
223	<extra_id_218>	0.000157112	1.1e-08
131	<extra_id_126>	0.000157154	1.1e-08
201	<extra_id_196>	0.000157167	1.1e-08
253	<extra_id_248>	0.000157172	1.1e-08
188	<extra_id_183>	0.000157209	1.1e-08
26	<extra_id_21>	0.000157216	1.1e-08
247	<extra_id_242>	0.00015725	1.1e-08
213	<extra_id_208>	0.000157323	1.1e-08
219	<extra_id_214>	0.000157327	1.1e-08
27	<extra_id_22>	0.000157352	1.1e-08
153	<extra_id_148>	0.000157358	1.1e-08
260	<extra_id_255>	0.00015736	1.1e-08
166	<extra_id_161>	0.000157377	1.1e-08
105	<extra_id_100>	0.00015739	1.1e-08
126	<extra_id_121>	0.000157421	1.1e-08
168	<extra_id_163>	0.000157435	1.1e-08
122	<extra_id_117>	0.000157446	1.1e-08
231	<extra_id_226>	0.000157467	1.1e-08
211	<extra_id_206>	0.00015751	1.1e-08
111	<extra_id_106>	0.000157522	1.1e-08
35	<extra_id_30>	0.000157537	1.1e-08
108	<extra_id_103>	0.000157558	1.1e-08
216	<extra_id_211>	0.00015756	1.1e-08
193	<extra_id_188>	0.000157578	1.1e-08
179	<extra_id_174>	0.000157609	1.1e-08
81	<extra_id_76>	0.000157638	1.1e-08
155	<extra_id_150>	0.000157677	1.1e-08
159	<extra_id_154>	0.00015775	1.2e-08
109	<extra_id_104>	0.000157763	1.1e-08
189	<extra_id_184>	0.000157795	1.1e-08
234	<extra_id_229>	0.000157818	1.1e-08
233	<extra_id_228>	0.000157849	1.1e-08
258	<extra_id_253>	0.000157869	1.1e-08
200	<extra_id_195>	0.000157876	1.1e-08
84	<extra_id_79>	0.000157979	1.1e-08
229	<extra_id_224>	0.000158037	1.1e-08
202	<extra_id_197>	0.000158048	1.1e-08
163	<extra_id_158>	0.000158079	1.1e-08
245	<extra_id_240>	0.000158094	1.1e-08
37	<extra_id_32>	0.000158144	1.1e-08
23	<extra_id_18>	0.000158153	1.1e-08
161	<extra_id_156>	0.000158168	1.1e-08
146	<extra_id_141>	0.000158181	1.1e-08
230	<extra_id_225>	0.000158213	1.1e-08
243	<extra_id_238>	0.000158286	1.1e-08
142	<extra_id_137>	0.000158349	1.1e-08
75	<extra_id_70>	0.000158353	1.1e-08
209	<extra_id_204>	0.000158371	1.1e-08
39	<extra_id_34>	0.000158395	1.1e-08
217	<extra_id_212>	0.000158404	1.1e-08
250	<extra_id_245>	0.000158429	1.1e-08
238	<extra_id_233>	0.000158495	1.1e-08
194	<extra_id_189>	0.000158544	1.1e-08
195	<extra_id_190>	0.000158548	1.1e-08
116	<extra_id_111>	0.000158581	1.1e-08
63	<extra_id_58>	0.000158612	1.1e-08
62	<extra_id_57>	0.000158635	1.1e-08
67	<extra_id_62>	0.000158674	1.1e-08
24	<extra_id_19>	0.000158696	1.1e-08
56	<extra_id_51>	0.000158716	1.1e-08
74	<extra_id_69>	0.00015882	1.1e-08
169	<extra_id_164>	0.000158853	1.1e-08
212	<extra_id_207>	0.000158897	1.1e-08
53	<extra_id_48>	0.000158922	1.1e-08
47	<extra_id_42>	0.000158925	1.1e-08
190	<extra_id_185>	0.000158928	1.1e-08
34	<extra_id_29>	0.000158929	1.1e-08
28	<extra_id_23>	0.000158933	1.1e-08
69	<extra_id_64>	0.000158964	1.1e-08
172	<extra_id_167>	0.000158981	1.1e-08
98	<extra_id_93>	0.000159002	1.1e-08
224	<extra_id_219>	0.000159053	1.1e-08
60	<extra_id_55>	0.000159105	1.1e-08
29	<extra_id_24>	0.000159151	1.1e-08
114	<extra_id_109>	0.000159167	1.1e-08
20	<extra_id_15>	0.000159208	1.1e-08
25	<extra_id_20>	0.000159216	1.1e-08
162	<extra_id_157>	0.000159283	1.1e-08
249	<extra_id_244>	0.000159289	1.1e-08
251	<extra_id_246>	0.000159337	1.1e-08
197	<extra_id_192>	0.000159437	1.1e-08
185	<extra_id_180>	0.000159469	1.1e-08
175	<extra_id_170>	0.000159507	1.1e-08
259	<extra_id_254>	0.000159526	1.1e-08
121	<extra_id_116>	0.000159624	1.1e-08
227	<extra_id_222>	0.000159679	1.1e-08
80	<extra_id_75>	0.000159692	1.1e-08
127	<extra_id_122>	0.000159719	1.1e-08
196	<extra_id_191>	0.000159753	1.1e-08
88	<extra_id_83>	0.00015976	1.1e-08
22	<extra_id_17>	0.000159841	1.1e-08
15	<extra_id_10>	0.000159894	1.1e-08
242	<extra_id_237>	0.000159899	1.1e-08
203	<extra_id_198>	0.00015998	1.1e-08
236	<extra_id_231>	0.000159996	1.1e-08
210	<extra_id_205>	0.00016007	1.1e-08
117	<extra_id_112>	0.000160091	1.1e-08
57	<extra_id_52>	0.000160102	1.1e-08
252	<extra_id_247>	0.000160104	1.1e-08
99	<extra_id_94>	0.000160112	1.1e-08
145	<extra_id_140>	0.000160136	1.1e-08
154	<extra_id_149>	0.000160202	1.2e-08
171	<extra_id_166>	0.000160258	1.1e-08
100	<extra_id_95>	0.00016043	1.1e-08
68	<extra_id_63>	0.00016046	1.1e-08
177	<extra_id_172>	0.000160468	1.1e-08
45	<extra_id_40>	0.000160479	1.1e-08
89	<extra_id_84>	0.000160543	1.1e-08
87	<extra_id_82>	0.000160558	1.1e-08
38	<extra_id_33>	0.000160583	1.1e-08
143	<extra_id_138>	0.00016063	1.1e-08
36	<extra_id_31>	0.000160763	1.1e-08
134	<extra_id_129>	0.000160899	1.1e-08
43	<extra_id_38>	0.000160981	1.1e-08
246	<extra_id_241>	0.000161018	1.1e-08
176	<extra_id_171>	0.000161243	1.1e-08
31	<extra_id_26>	0.0001613	1.1e-08
4	<|im_end|>	0.000161343
170	<extra_id_165>	0.000161397	1.1e-08
120	<extra_id_115>	0.000161465	1.1e-08
221	<extra_id_216>	0.000161477	1.1e-08
8	<extra_id_3>	0.000161513	1.1e-08
10	<extra_id_5>	0.000161548	1.1e-08
160	<extra_id_155>	0.000161587	1.1e-08
5	<extra_id_0>	0.000161617	1.1e-08
207	<extra_id_202>	0.000161703	1.1e-08
255	<extra_id_250>	0.000161708	1.1e-08
138	<extra_id_133>	0.000161776	1.1e-08
12	<extra_id_7>	0.000161781	1.1e-08
107	<extra_id_102>	0.000161995	1.1e-08
66	<extra_id_61>	0.000162016	1.1e-08
64	<extra_id_59>	0.000162064	1.1e-08
133	<extra_id_128>	0.000162151	1.1e-08
132	<extra_id_127>	0.00016226	1.1e-08
72	<extra_id_67>	0.000162282	1.1e-08
151	<extra_id_146>	0.000162282	1.1e-08
228	<extra_id_223>	0.000162434	1.1e-08
91	<extra_id_86>	0.000162442	1.1e-08
173	<extra_id_168>	0.000162505	1.1e-08
82	<extra_id_77>	0.000162623	1.1e-08
147	<extra_id_142>	0.000162677	1.1e-08
187	<extra_id_182>	0.000162689	1.1e-08
55	<extra_id_50>	0.00016286	1.1e-08
77	<extra_id_72>	0.000162872	1.1e-08
48	<extra_id_43>	0.00016333	1.1e-08
17	<extra_id_12>	0.000163558	1.1e-08
21	<extra_id_16>	0.000164425	1.1e-08
113	<extra_id_108>	0.000164672	1.1e-08
1	<s>	0.97711	0.13
0	<unk>	0.987323	1
2	</s>	1.00525	0.041

Unreachable tokens

0 entries below threshold of 1.066