--- license: mit base_model: cardiffnlp/twitter-roberta-base-2019-90m tags: - generated_from_trainer model-index: - name: 2020-Q1-25p-filtered results: [] --- # 2020-Q1-25p-filtered This model is a fine-tuned version of [cardiffnlp/twitter-roberta-base-2019-90m](https://huggingface.co/cardiffnlp/twitter-roberta-base-2019-90m) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 2.2233 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 4.1e-07 - train_batch_size: 16 - eval_batch_size: 16 - seed: 42 - optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08 - lr_scheduler_type: linear - training_steps: 2400000 ### Training results | Training Loss | Epoch | Step | Validation Loss | |:-------------:|:-----:|:-------:|:---------------:| | No log | 0.02 | 8000 | 2.5883 | | 2.7561 | 0.04 | 16000 | 2.4958 | | 2.7561 | 0.07 | 24000 | 2.4376 | | 2.531 | 0.09 | 32000 | 2.4090 | | 2.531 | 0.11 | 40000 | 2.3791 | | 2.4627 | 0.13 | 48000 | 2.3666 | | 2.4627 | 0.15 | 56000 | 2.3457 | | 2.4252 | 0.17 | 64000 | 2.3380 | | 2.4252 | 0.2 | 72000 | 2.3298 | | 2.4061 | 0.22 | 80000 | 2.3253 | | 2.4061 | 0.24 | 88000 | 2.3177 | | 2.395 | 0.26 | 96000 | 2.3131 | | 2.395 | 0.28 | 104000 | 2.3058 | | 2.3843 | 0.31 | 112000 | 2.3010 | | 2.3843 | 0.33 | 120000 | 2.2925 | | 2.3738 | 0.35 | 128000 | 2.2916 | | 2.3738 | 0.37 | 136000 | 2.2947 | | 2.3686 | 0.39 | 144000 | 2.2835 | | 2.3686 | 0.42 | 152000 | 2.2864 | | 2.3615 | 0.44 | 160000 | 2.2834 | | 2.3615 | 0.46 | 168000 | 2.2768 | | 2.3515 | 0.48 | 176000 | 2.2803 | | 2.3515 | 0.5 | 184000 | 2.2804 | | 2.3508 | 0.52 | 192000 | 2.2754 | | 2.3508 | 0.55 | 200000 | 2.2767 | | 2.35 | 0.57 | 208000 | 2.2742 | | 2.35 | 0.59 | 216000 | 2.2722 | | 2.3385 | 0.61 | 224000 | 2.2661 | | 2.3385 | 0.63 | 232000 | 2.2706 | | 2.3393 | 0.66 | 240000 | 2.2633 | | 2.3393 | 0.68 | 248000 | 2.2648 | | 2.3392 | 0.7 | 256000 | 2.2656 | | 2.3392 | 0.72 | 264000 | 2.2660 | | 2.3336 | 0.74 | 272000 | 2.2657 | | 2.3336 | 0.76 | 280000 | 2.2605 | | 2.3324 | 0.79 | 288000 | 2.2615 | | 2.3324 | 0.81 | 296000 | 2.2551 | | 2.3312 | 0.83 | 304000 | 2.2581 | | 2.3312 | 0.85 | 312000 | 2.2626 | | 2.3352 | 0.87 | 320000 | 2.2576 | | 2.3352 | 0.9 | 328000 | 2.2553 | | 2.3287 | 0.92 | 336000 | 2.2591 | | 2.3287 | 0.94 | 344000 | 2.2558 | | 2.321 | 0.96 | 352000 | 2.2603 | | 2.321 | 0.98 | 360000 | 2.2569 | | 2.3278 | 1.01 | 368000 | 2.2544 | | 2.3278 | 1.03 | 376000 | 2.2604 | | 2.319 | 1.05 | 384000 | 2.2535 | | 2.319 | 1.07 | 392000 | 2.2420 | | 2.3151 | 1.09 | 400000 | 2.2583 | | 2.3151 | 1.11 | 408000 | 2.2535 | | 2.3144 | 1.14 | 416000 | 2.2582 | | 2.3144 | 1.16 | 424000 | 2.2496 | | 2.3191 | 1.18 | 432000 | 2.2532 | | 2.3191 | 1.2 | 440000 | 2.2515 | | 2.3168 | 1.22 | 448000 | 2.2501 | | 2.3168 | 1.25 | 456000 | 2.2453 | | 2.3156 | 1.27 | 464000 | 2.2445 | | 2.3156 | 1.29 | 472000 | 2.2485 | | 2.3178 | 1.31 | 480000 | 2.2493 | | 2.3178 | 1.33 | 488000 | 2.2443 | | 2.3113 | 1.35 | 496000 | 2.2493 | | 2.3113 | 1.38 | 504000 | 2.2493 | | 2.3116 | 1.4 | 512000 | 2.2483 | | 2.3116 | 1.42 | 520000 | 2.2459 | | 2.3166 | 1.44 | 528000 | 2.2481 | | 2.3166 | 1.46 | 536000 | 2.2542 | | 2.3158 | 1.49 | 544000 | 2.2443 | | 2.3158 | 1.51 | 552000 | 2.2402 | | 2.3148 | 1.53 | 560000 | 2.2449 | | 2.3148 | 1.55 | 568000 | 2.2415 | | 2.3145 | 1.57 | 576000 | 2.2471 | | 2.3145 | 1.6 | 584000 | 2.2469 | | 2.3119 | 1.62 | 592000 | 2.2445 | | 2.3119 | 1.64 | 600000 | 2.2487 | | 2.3045 | 1.66 | 608000 | 2.2456 | | 2.3045 | 1.68 | 616000 | 2.2466 | | 2.3046 | 1.7 | 624000 | 2.2357 | | 2.3046 | 1.73 | 632000 | 2.2448 | | 2.3083 | 1.75 | 640000 | 2.2381 | | 2.3083 | 1.77 | 648000 | 2.2439 | | 2.3065 | 1.79 | 656000 | 2.2402 | | 2.3065 | 1.81 | 664000 | 2.2439 | | 2.307 | 1.84 | 672000 | 2.2409 | | 2.307 | 1.86 | 680000 | 2.2426 | | 2.3026 | 1.88 | 688000 | 2.2387 | | 2.3026 | 1.9 | 696000 | 2.2357 | | 2.2949 | 1.92 | 704000 | 2.2379 | | 2.2949 | 1.95 | 712000 | 2.2408 | | 2.2951 | 1.97 | 720000 | 2.2432 | | 2.2951 | 1.99 | 728000 | 2.2444 | | 2.3011 | 2.01 | 736000 | 2.2382 | | 2.3011 | 2.03 | 744000 | 2.2391 | | 2.3017 | 2.05 | 752000 | 2.2363 | | 2.3017 | 2.08 | 760000 | 2.2444 | | 2.2978 | 2.1 | 768000 | 2.2370 | | 2.2978 | 2.12 | 776000 | 2.2350 | | 2.2961 | 2.14 | 784000 | 2.2347 | | 2.2961 | 2.16 | 792000 | 2.2386 | | 2.2968 | 2.19 | 800000 | 2.2322 | | 2.2968 | 2.21 | 808000 | 2.2403 | | 2.2962 | 2.23 | 816000 | 2.2347 | | 2.2962 | 2.25 | 824000 | 2.2398 | | 2.2984 | 2.27 | 832000 | 2.2358 | | 2.2984 | 2.29 | 840000 | 2.2412 | | 2.3029 | 2.32 | 848000 | 2.2386 | | 2.3029 | 2.34 | 856000 | 2.2346 | | 2.2985 | 2.36 | 864000 | 2.2323 | | 2.2985 | 2.38 | 872000 | 2.2387 | | 2.2922 | 2.4 | 880000 | 2.2303 | | 2.2922 | 2.43 | 888000 | 2.2326 | | 2.2967 | 2.45 | 896000 | 2.2422 | | 2.2967 | 2.47 | 904000 | 2.2350 | | 2.2917 | 2.49 | 912000 | 2.2299 | | 2.2917 | 2.51 | 920000 | 2.2308 | | 2.2912 | 2.54 | 928000 | 2.2345 | | 2.2912 | 2.56 | 936000 | 2.2264 | | 2.2887 | 2.58 | 944000 | 2.2361 | | 2.2887 | 2.6 | 952000 | 2.2319 | | 2.2956 | 2.62 | 960000 | 2.2340 | | 2.2956 | 2.64 | 968000 | 2.2356 | | 2.2927 | 2.67 | 976000 | 2.2366 | | 2.2927 | 2.69 | 984000 | 2.2335 | | 2.2872 | 2.71 | 992000 | 2.2330 | | 2.2872 | 2.73 | 1000000 | 2.2251 | | 2.2936 | 2.75 | 1008000 | 2.2327 | | 2.2936 | 2.78 | 1016000 | 2.2326 | | 2.2899 | 2.8 | 1024000 | 2.2307 | | 2.2899 | 2.82 | 1032000 | 2.2291 | | 2.2931 | 2.84 | 1040000 | 2.2285 | | 2.2931 | 2.86 | 1048000 | 2.2327 | | 2.3042 | 2.88 | 1056000 | 2.2367 | | 2.3042 | 2.91 | 1064000 | 2.2345 | | 2.2864 | 2.93 | 1072000 | 2.2267 | | 2.2864 | 2.95 | 1080000 | 2.2343 | | 2.2933 | 2.97 | 1088000 | 2.2354 | | 2.2933 | 2.99 | 1096000 | 2.2260 | | 2.2909 | 3.02 | 1104000 | 2.2341 | | 2.2909 | 3.04 | 1112000 | 2.2266 | | 2.2889 | 3.06 | 1120000 | 2.2253 | | 2.2889 | 3.08 | 1128000 | 2.2255 | | 2.292 | 3.1 | 1136000 | 2.2194 | | 2.292 | 3.13 | 1144000 | 2.2319 | | 2.282 | 3.15 | 1152000 | 2.2221 | | 2.282 | 3.17 | 1160000 | 2.2273 | | 2.2827 | 3.19 | 1168000 | 2.2296 | | 2.2827 | 3.21 | 1176000 | 2.2332 | | 2.2937 | 3.23 | 1184000 | 2.2302 | | 2.2937 | 3.26 | 1192000 | 2.2262 | | 2.2845 | 3.28 | 1200000 | 2.2318 | | 2.2845 | 3.3 | 1208000 | 2.2291 | | 2.284 | 3.32 | 1216000 | 2.2327 | | 2.284 | 3.34 | 1224000 | 2.2308 | | 2.2923 | 3.37 | 1232000 | 2.2264 | | 2.2923 | 3.39 | 1240000 | 2.2390 | | 2.2859 | 3.41 | 1248000 | 2.2310 | | 2.2859 | 3.43 | 1256000 | 2.2287 | | 2.2879 | 3.45 | 1264000 | 2.2284 | | 2.2879 | 3.47 | 1272000 | 2.2228 | | 2.292 | 3.5 | 1280000 | 2.2296 | | 2.292 | 3.52 | 1288000 | 2.2329 | | 2.2827 | 3.54 | 1296000 | 2.2263 | | 2.2827 | 3.56 | 1304000 | 2.2324 | | 2.2829 | 3.58 | 1312000 | 2.2232 | | 2.2829 | 3.61 | 1320000 | 2.2273 | | 2.2863 | 3.63 | 1328000 | 2.2296 | | 2.2863 | 3.65 | 1336000 | 2.2294 | | 2.2796 | 3.67 | 1344000 | 2.2283 | | 2.2796 | 3.69 | 1352000 | 2.2280 | | 2.2835 | 3.72 | 1360000 | 2.2264 | | 2.2835 | 3.74 | 1368000 | 2.2224 | | 2.2875 | 3.76 | 1376000 | 2.2219 | | 2.2875 | 3.78 | 1384000 | 2.2243 | | 2.2792 | 3.8 | 1392000 | 2.2320 | | 2.2792 | 3.82 | 1400000 | 2.2273 | | 2.2932 | 3.85 | 1408000 | 2.2257 | | 2.2932 | 3.87 | 1416000 | 2.2360 | | 2.2899 | 3.89 | 1424000 | 2.2277 | | 2.2899 | 3.91 | 1432000 | 2.2275 | | 2.2859 | 3.93 | 1440000 | 2.2287 | | 2.2859 | 3.96 | 1448000 | 2.2211 | | 2.2876 | 3.98 | 1456000 | 2.2236 | | 2.2876 | 4.0 | 1464000 | 2.2288 | | 2.2879 | 4.02 | 1472000 | 2.2226 | | 2.2879 | 4.04 | 1480000 | 2.2242 | | 2.282 | 4.06 | 1488000 | 2.2286 | | 2.282 | 4.09 | 1496000 | 2.2210 | | 2.2828 | 4.11 | 1504000 | 2.2304 | | 2.2828 | 4.13 | 1512000 | 2.2310 | | 2.2765 | 4.15 | 1520000 | 2.2295 | | 2.2765 | 4.17 | 1528000 | 2.2276 | | 2.2839 | 4.2 | 1536000 | 2.2260 | | 2.2839 | 4.22 | 1544000 | 2.2255 | | 2.2845 | 4.24 | 1552000 | 2.2200 | | 2.2845 | 4.26 | 1560000 | 2.2228 | | 2.2816 | 4.28 | 1568000 | 2.2322 | | 2.2816 | 4.31 | 1576000 | 2.2250 | | 2.2965 | 4.33 | 1584000 | 2.2242 | | 2.2965 | 4.35 | 1592000 | 2.2295 | | 2.2806 | 4.37 | 1600000 | 2.2198 | | 2.2806 | 4.39 | 1608000 | 2.2301 | | 2.2868 | 4.41 | 1616000 | 2.2309 | | 2.2868 | 4.44 | 1624000 | 2.2270 | | 2.2907 | 4.46 | 1632000 | 2.2291 | | 2.2907 | 4.48 | 1640000 | 2.2269 | | 2.2809 | 4.5 | 1648000 | 2.2261 | | 2.2809 | 4.52 | 1656000 | 2.2318 | | 2.2876 | 4.55 | 1664000 | 2.2252 | | 2.2876 | 4.57 | 1672000 | 2.2248 | | 2.2844 | 4.59 | 1680000 | 2.2223 | | 2.2844 | 4.61 | 1688000 | 2.2250 | | 2.2841 | 4.63 | 1696000 | 2.2278 | | 2.2841 | 4.65 | 1704000 | 2.2226 | | 2.2851 | 4.68 | 1712000 | 2.2274 | | 2.2851 | 4.7 | 1720000 | 2.2247 | | 2.2863 | 4.72 | 1728000 | 2.2239 | | 2.2863 | 4.74 | 1736000 | 2.2227 | | 2.2788 | 4.76 | 1744000 | 2.2234 | | 2.2788 | 4.79 | 1752000 | 2.2293 | | 2.2849 | 4.81 | 1760000 | 2.2199 | | 2.2849 | 4.83 | 1768000 | 2.2309 | | 2.2826 | 4.85 | 1776000 | 2.2235 | | 2.2826 | 4.87 | 1784000 | 2.2292 | | 2.2809 | 4.9 | 1792000 | 2.2248 | | 2.2809 | 4.92 | 1800000 | 2.2187 | | 2.2865 | 4.94 | 1808000 | 2.2331 | | 2.2865 | 4.96 | 1816000 | 2.2244 | | 2.2773 | 4.98 | 1824000 | 2.2246 | | 2.2773 | 5.0 | 1832000 | 2.2315 | | 2.2738 | 5.03 | 1840000 | 2.2319 | | 2.2738 | 5.05 | 1848000 | 2.2258 | | 2.2806 | 5.07 | 1856000 | 2.2241 | | 2.2806 | 5.09 | 1864000 | 2.2228 | | 2.2822 | 5.11 | 1872000 | 2.2218 | | 2.2822 | 5.14 | 1880000 | 2.2276 | | 2.2866 | 5.16 | 1888000 | 2.2233 | | 2.2866 | 5.18 | 1896000 | 2.2266 | | 2.2831 | 5.2 | 1904000 | 2.2231 | | 2.2831 | 5.22 | 1912000 | 2.2241 | | 2.2875 | 5.24 | 1920000 | 2.2263 | | 2.2875 | 5.27 | 1928000 | 2.2234 | | 2.2802 | 5.29 | 1936000 | 2.2269 | | 2.2802 | 5.31 | 1944000 | 2.2253 | | 2.2905 | 5.33 | 1952000 | 2.2191 | | 2.2905 | 5.35 | 1960000 | 2.2217 | | 2.282 | 5.38 | 1968000 | 2.2212 | | 2.282 | 5.4 | 1976000 | 2.2213 | | 2.2798 | 5.42 | 1984000 | 2.2218 | | 2.2798 | 5.44 | 1992000 | 2.2222 | | 2.2864 | 5.46 | 2000000 | 2.2212 | | 2.2864 | 5.49 | 2008000 | 2.2282 | | 2.2867 | 5.51 | 2016000 | 2.2304 | | 2.2867 | 5.53 | 2024000 | 2.2222 | | 2.2834 | 5.55 | 2032000 | 2.2285 | | 2.2834 | 5.57 | 2040000 | 2.2230 | | 2.2851 | 5.59 | 2048000 | 2.2237 | | 2.2851 | 5.62 | 2056000 | 2.2283 | | 2.2774 | 5.64 | 2064000 | 2.2232 | | 2.2774 | 5.66 | 2072000 | 2.2282 | | 2.277 | 5.68 | 2080000 | 2.2271 | | 2.277 | 5.7 | 2088000 | 2.2256 | | 2.2868 | 5.73 | 2096000 | 2.2252 | | 2.2868 | 5.75 | 2104000 | 2.2285 | | 2.2727 | 5.77 | 2112000 | 2.2251 | | 2.2727 | 5.79 | 2120000 | 2.2239 | | 2.2803 | 5.81 | 2128000 | 2.2287 | | 2.2803 | 5.84 | 2136000 | 2.2274 | | 2.2785 | 5.86 | 2144000 | 2.2227 | | 2.2785 | 5.88 | 2152000 | 2.2267 | | 2.2829 | 5.9 | 2160000 | 2.2251 | | 2.2829 | 5.92 | 2168000 | 2.2228 | | 2.2816 | 5.94 | 2176000 | 2.2235 | | 2.2816 | 5.97 | 2184000 | 2.2289 | | 2.283 | 5.99 | 2192000 | 2.2238 | | 2.283 | 6.01 | 2200000 | 2.2245 | | 2.2761 | 6.03 | 2208000 | 2.2297 | | 2.2761 | 6.05 | 2216000 | 2.2300 | | 2.2823 | 6.08 | 2224000 | 2.2268 | | 2.2823 | 6.1 | 2232000 | 2.2252 | | 2.2715 | 6.12 | 2240000 | 2.2240 | | 2.2715 | 6.14 | 2248000 | 2.2233 | | 2.2809 | 6.16 | 2256000 | 2.2238 | | 2.2809 | 6.18 | 2264000 | 2.2204 | | 2.2823 | 6.21 | 2272000 | 2.2218 | | 2.2823 | 6.23 | 2280000 | 2.2295 | | 2.2848 | 6.25 | 2288000 | 2.2298 | | 2.2848 | 6.27 | 2296000 | 2.2299 | | 2.2847 | 6.29 | 2304000 | 2.2246 | | 2.2847 | 6.32 | 2312000 | 2.2230 | | 2.2783 | 6.34 | 2320000 | 2.2260 | | 2.2783 | 6.36 | 2328000 | 2.2176 | | 2.2791 | 6.38 | 2336000 | 2.2211 | | 2.2791 | 6.4 | 2344000 | 2.2262 | | 2.2797 | 6.43 | 2352000 | 2.2293 | | 2.2797 | 6.45 | 2360000 | 2.2219 | | 2.2784 | 6.47 | 2368000 | 2.2249 | | 2.2784 | 6.49 | 2376000 | 2.2216 | | 2.271 | 6.51 | 2384000 | 2.2256 | | 2.271 | 6.53 | 2392000 | 2.2296 | | 2.2787 | 6.56 | 2400000 | 2.2275 | ### Framework versions - Transformers 4.35.0.dev0 - Pytorch 2.0.1+cu117 - Datasets 2.14.5 - Tokenizers 0.14.0