feat: add extractor-based, non-strict parsing#693
Draft
aschierl-xitaso wants to merge 3 commits into
Draft
Conversation
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
CSAF Validation Benchmark
Details
| Benchmark suite | Current: 2f89ec8 | Previous: b723aec | Ratio |
|---|---|---|---|
csaf_2_0_tests/6.1.1 |
132736 ns/iter (± 780) |
125702 ns/iter (± 2732) |
1.06 |
csaf_2_0_tests/6.1.2 |
52314 ns/iter (± 90) |
50378 ns/iter (± 28) |
1.04 |
csaf_2_0_tests/6.1.3 |
13271 ns/iter (± 103) |
12634 ns/iter (± 8) |
1.05 |
csaf_2_0_tests/6.1.4 |
27671 ns/iter (± 40) |
28420 ns/iter (± 143) |
0.97 |
csaf_2_0_tests/6.1.5 |
14165 ns/iter (± 26) |
13613 ns/iter (± 29) |
1.04 |
csaf_2_0_tests/6.1.6 |
39713 ns/iter (± 126) |
39255 ns/iter (± 83) |
1.01 |
csaf_2_0_tests/6.1.7 |
33980 ns/iter (± 49) |
32608 ns/iter (± 341) |
1.04 |
csaf_2_0_tests/6.1.8 |
90467 ns/iter (± 3635) |
85671 ns/iter (± 633) |
1.06 |
csaf_2_0_tests/6.1.9 |
87422 ns/iter (± 129) |
89370 ns/iter (± 184) |
0.98 |
csaf_2_0_tests/6.1.10 |
153870 ns/iter (± 314) |
148760 ns/iter (± 944) |
1.03 |
csaf_2_0_tests/6.1.11 |
14637 ns/iter (± 696) |
14576 ns/iter (± 16) |
1.00 |
csaf_2_0_tests/6.1.12 |
30118 ns/iter (± 26) |
30495 ns/iter (± 120) |
0.99 |
csaf_2_0_tests/6.1.13 |
33016 ns/iter (± 47) |
32445 ns/iter (± 146) |
1.02 |
csaf_2_0_tests/6.1.15 |
12710 ns/iter (± 32) |
12688 ns/iter (± 23) |
1.00 |
csaf_2_0_tests/6.1.16 |
48466 ns/iter (± 64) |
48360 ns/iter (± 48) |
1.00 |
csaf_2_0_tests/6.1.17 |
14131 ns/iter (± 20) |
13949 ns/iter (± 14) |
1.01 |
csaf_2_0_tests/6.1.18 |
16448 ns/iter (± 18) |
15528 ns/iter (± 12) |
1.06 |
csaf_2_0_tests/6.1.19 |
16156 ns/iter (± 904) |
15363 ns/iter (± 18) |
1.05 |
csaf_2_0_tests/6.1.20 |
13965 ns/iter (± 12) |
13590 ns/iter (± 481) |
1.03 |
csaf_2_0_tests/6.1.21 |
44475 ns/iter (± 79) |
50943 ns/iter (± 96) |
0.87 |
csaf_2_0_tests/6.1.22 |
48779 ns/iter (± 39) |
51278 ns/iter (± 104) |
0.95 |
csaf_2_0_tests/6.1.23 |
16248 ns/iter (± 55) |
16116 ns/iter (± 19) |
1.01 |
csaf_2_0_tests/6.1.24 |
17179 ns/iter (± 172) |
15849 ns/iter (± 38) |
1.08 |
csaf_2_0_tests/6.1.25 |
35286 ns/iter (± 75) |
34484 ns/iter (± 53) |
1.02 |
csaf_2_0_tests/6.1.26 |
20517 ns/iter (± 48) |
20701 ns/iter (± 30) |
0.99 |
csaf_2_0_tests/6.1.27.1 |
13655 ns/iter (± 115) |
13524 ns/iter (± 49) |
1.01 |
csaf_2_0_tests/6.1.27.2 |
13719 ns/iter (± 31) |
13702 ns/iter (± 12) |
1.00 |
csaf_2_0_tests/6.1.27.3 |
13381 ns/iter (± 21) |
13375 ns/iter (± 39) |
1.00 |
csaf_2_0_tests/6.1.27.4 |
13804 ns/iter (± 9) |
13558 ns/iter (± 74) |
1.02 |
csaf_2_0_tests/6.1.27.5 |
14296 ns/iter (± 16) |
14400 ns/iter (± 22) |
0.99 |
csaf_2_0_tests/6.1.27.6 |
13786 ns/iter (± 7) |
13862 ns/iter (± 23) |
0.99 |
csaf_2_0_tests/6.1.27.7 |
12741 ns/iter (± 23) |
12739 ns/iter (± 268) |
1.00 |
csaf_2_0_tests/6.1.27.8 |
13140 ns/iter (± 314) |
13056 ns/iter (± 9) |
1.01 |
csaf_2_0_tests/6.1.27.9 |
26075 ns/iter (± 206) |
25865 ns/iter (± 64) |
1.01 |
csaf_2_0_tests/6.1.27.10 |
14299 ns/iter (± 17) |
14413 ns/iter (± 18) |
0.99 |
csaf_2_0_tests/6.1.27.11 |
13372 ns/iter (± 11) |
13372 ns/iter (± 22) |
1 |
csaf_2_0_tests/6.1.28 |
23551 ns/iter (± 71) |
24653 ns/iter (± 81) |
0.96 |
csaf_2_0_tests/6.1.29 |
13008 ns/iter (± 52) |
12770 ns/iter (± 34) |
1.02 |
csaf_2_0_tests/6.1.30 |
47356 ns/iter (± 59) |
48865 ns/iter (± 62) |
0.97 |
csaf_2_0_tests/6.1.31 |
34247 ns/iter (± 108) |
33684 ns/iter (± 68) |
1.02 |
csaf_2_0_tests/6.1.32 |
12939 ns/iter (± 18) |
12820 ns/iter (± 51) |
1.01 |
csaf_2_0_tests/6.1.33 |
19708 ns/iter (± 17) |
19962 ns/iter (± 25) |
0.99 |
csaf_2_0_tests/6.2.1 |
156220 ns/iter (± 283) |
153379 ns/iter (± 333) |
1.02 |
csaf_2_0_tests/6.2.2 |
21880 ns/iter (± 132) |
21563 ns/iter (± 731) |
1.01 |
csaf_2_0_tests/6.2.3 |
21294 ns/iter (± 280) |
20553 ns/iter (± 147) |
1.04 |
csaf_2_0_tests/6.2.4 |
20459 ns/iter (± 76) |
20431 ns/iter (± 30) |
1.00 |
csaf_2_0_tests/6.2.5 |
58554 ns/iter (± 1041) |
59533 ns/iter (± 381) |
0.98 |
csaf_2_0_tests/6.2.6 |
58826 ns/iter (± 153) |
60196 ns/iter (± 113) |
0.98 |
csaf_2_0_tests/6.2.7 |
18579 ns/iter (± 775) |
17565 ns/iter (± 54) |
1.06 |
csaf_2_0_tests/6.2.8 |
37660 ns/iter (± 106) |
36852 ns/iter (± 69) |
1.02 |
csaf_2_0_tests/6.2.9 |
37863 ns/iter (± 49) |
36798 ns/iter (± 199) |
1.03 |
csaf_2_0_tests/6.2.10 |
28207 ns/iter (± 50) |
28811 ns/iter (± 172) |
0.98 |
csaf_2_0_tests/6.2.11 |
151325 ns/iter (± 148) |
157552 ns/iter (± 310) |
0.96 |
csaf_2_0_tests/6.2.12 |
35425 ns/iter (± 163) |
39208 ns/iter (± 39) |
0.90 |
csaf_2_0_tests/6.2.13 |
797947 ns/iter (± 1959) |
804796 ns/iter (± 6336) |
0.99 |
csaf_2_0_tests/6.2.14 |
45274 ns/iter (± 534) |
46276 ns/iter (± 35) |
0.98 |
csaf_2_0_tests/6.2.15 |
35908 ns/iter (± 53) |
36413 ns/iter (± 67) |
0.99 |
csaf_2_0_tests/6.2.16 |
46285 ns/iter (± 55) |
46073 ns/iter (± 88) |
1.00 |
csaf_2_0_tests/6.2.17 |
18134 ns/iter (± 419) |
18052 ns/iter (± 20) |
1.00 |
csaf_2_0_tests/6.2.18 |
29422 ns/iter (± 132) |
29025 ns/iter (± 48) |
1.01 |
csaf_2_0_tests/6.2.19 |
21516 ns/iter (± 20) |
21798 ns/iter (± 268) |
0.99 |
csaf_2_0_tests/6.2.20 |
1743994 ns/iter (± 59837) |
1783015 ns/iter (± 6548) |
0.98 |
csaf_2_0_tests/6.3.1 |
44449 ns/iter (± 192) |
44438 ns/iter (± 43) |
1.00 |
csaf_2_0_tests/6.3.2 |
28897 ns/iter (± 44) |
28621 ns/iter (± 30) |
1.01 |
csaf_2_0_tests/6.3.3 |
28882 ns/iter (± 39) |
28599 ns/iter (± 65) |
1.01 |
csaf_2_0_tests/6.3.4 |
31653 ns/iter (± 62) |
31581 ns/iter (± 54) |
1.00 |
csaf_2_0_tests/6.3.5 |
43621 ns/iter (± 56) |
42728 ns/iter (± 39) |
1.02 |
csaf_2_0_tests/6.3.6 |
20989 ns/iter (± 18) |
21051 ns/iter (± 165) |
1.00 |
csaf_2_0_tests/6.3.7 |
20997 ns/iter (± 45) |
20869 ns/iter (± 199) |
1.01 |
csaf_2_0_tests/6.3.8 |
21078 ns/iter (± 342) |
21591 ns/iter (± 259) |
0.98 |
csaf_2_0_tests/6.3.9 |
36909 ns/iter (± 61) |
37106 ns/iter (± 54) |
0.99 |
csaf_2_0_tests/6.3.10 |
33560 ns/iter (± 439) |
32867 ns/iter (± 66) |
1.02 |
csaf_2_0_tests/6.3.11 |
34540 ns/iter (± 62) |
34254 ns/iter (± 28) |
1.01 |
csaf_2_1_tests/6.1.1 |
662615 ns/iter (± 8576) |
655242 ns/iter (± 3336) |
1.01 |
csaf_2_1_tests/6.1.2 |
253742 ns/iter (± 495) |
252748 ns/iter (± 1178) |
1.00 |
csaf_2_1_tests/6.1.3 |
71584 ns/iter (± 121) |
68431 ns/iter (± 381) |
1.05 |
csaf_2_1_tests/6.1.4 |
126614 ns/iter (± 256) |
129662 ns/iter (± 174) |
0.98 |
csaf_2_1_tests/6.1.5 |
49345 ns/iter (± 1038) |
48272 ns/iter (± 83) |
1.02 |
csaf_2_1_tests/6.1.6 |
121416 ns/iter (± 217) |
122198 ns/iter (± 166) |
0.99 |
csaf_2_1_tests/6.1.7 |
73965 ns/iter (± 84) |
76610 ns/iter (± 79) |
0.97 |
csaf_2_1_tests/6.1.8 |
72889 ns/iter (± 126) |
74853 ns/iter (± 731) |
0.97 |
csaf_2_1_tests/6.1.9 |
566137 ns/iter (± 2258) |
584156 ns/iter (± 1518) |
0.97 |
csaf_2_1_tests/6.1.10 |
725305 ns/iter (± 5067) |
718934 ns/iter (± 3976) |
1.01 |
csaf_2_1_tests/6.1.11 |
80653 ns/iter (± 133) |
79101 ns/iter (± 174) |
1.02 |
csaf_2_1_tests/6.1.12 |
82886 ns/iter (± 1458) |
84716 ns/iter (± 52) |
0.98 |
csaf_2_1_tests/6.1.13 |
199254 ns/iter (± 554) |
192288 ns/iter (± 548) |
1.04 |
csaf_2_1_tests/6.1.15 |
43355 ns/iter (± 77) |
43506 ns/iter (± 48) |
1.00 |
csaf_2_1_tests/6.1.16 |
163549 ns/iter (± 858) |
152643 ns/iter (± 294) |
1.07 |
csaf_2_1_tests/6.1.17 |
47915 ns/iter (± 175) |
47050 ns/iter (± 110) |
1.02 |
csaf_2_1_tests/6.1.18 |
56494 ns/iter (± 159) |
51799 ns/iter (± 143) |
1.09 |
csaf_2_1_tests/6.1.19 |
53668 ns/iter (± 82) |
51128 ns/iter (± 90) |
1.05 |
csaf_2_1_tests/6.1.20 |
48089 ns/iter (± 74) |
46325 ns/iter (± 96) |
1.04 |
csaf_2_1_tests/6.1.21 |
151597 ns/iter (± 292) |
162958 ns/iter (± 791) |
0.93 |
csaf_2_1_tests/6.1.22 |
148368 ns/iter (± 637) |
152392 ns/iter (± 245) |
0.97 |
csaf_2_1_tests/6.1.23 |
54305 ns/iter (± 85) |
52473 ns/iter (± 52) |
1.03 |
csaf_2_1_tests/6.1.24 |
52006 ns/iter (± 588) |
49683 ns/iter (± 41) |
1.05 |
csaf_2_1_tests/6.1.25 |
169521 ns/iter (± 1005) |
168812 ns/iter (± 533) |
1.00 |
csaf_2_1_tests/6.1.26 |
73982 ns/iter (± 153) |
73106 ns/iter (± 155) |
1.01 |
csaf_2_1_tests/6.1.27.1 |
48048 ns/iter (± 110) |
48145 ns/iter (± 247) |
1.00 |
csaf_2_1_tests/6.1.27.2 |
48079 ns/iter (± 102) |
47494 ns/iter (± 102) |
1.01 |
csaf_2_1_tests/6.1.27.3 |
48708 ns/iter (± 87) |
48661 ns/iter (± 51) |
1.00 |
csaf_2_1_tests/6.1.27.4 |
48946 ns/iter (± 62) |
48800 ns/iter (± 30) |
1.00 |
csaf_2_1_tests/6.1.27.5 |
49933 ns/iter (± 63) |
49819 ns/iter (± 121) |
1.00 |
csaf_2_1_tests/6.1.27.6 |
49446 ns/iter (± 115) |
50524 ns/iter (± 184) |
0.98 |
csaf_2_1_tests/6.1.27.7 |
45465 ns/iter (± 848) |
45662 ns/iter (± 156) |
1.00 |
csaf_2_1_tests/6.1.27.8 |
45771 ns/iter (± 53) |
46691 ns/iter (± 32) |
0.98 |
csaf_2_1_tests/6.1.27.9 |
69463 ns/iter (± 510) |
67684 ns/iter (± 103) |
1.03 |
csaf_2_1_tests/6.1.27.10 |
46821 ns/iter (± 55) |
48083 ns/iter (± 219) |
0.97 |
csaf_2_1_tests/6.1.27.11 |
48095 ns/iter (± 6521) |
48330 ns/iter (± 82) |
1.00 |
csaf_2_1_tests/6.1.27.12 |
48438 ns/iter (± 727) |
48753 ns/iter (± 42) |
0.99 |
csaf_2_1_tests/6.1.27.13 |
70255 ns/iter (± 88) |
72463 ns/iter (± 653) |
0.97 |
csaf_2_1_tests/6.1.27.14 |
49002 ns/iter (± 52) |
48967 ns/iter (± 300) |
1.00 |
csaf_2_1_tests/6.1.27.15 |
48517 ns/iter (± 67) |
48649 ns/iter (± 55) |
1.00 |
csaf_2_1_tests/6.1.27.16 |
45250 ns/iter (± 196) |
48944 ns/iter (± 458) |
0.92 |
csaf_2_1_tests/6.1.27.17 |
52456 ns/iter (± 241) |
52002 ns/iter (± 52) |
1.01 |
csaf_2_1_tests/6.1.27.18 |
55026 ns/iter (± 58) |
54858 ns/iter (± 172) |
1.00 |
csaf_2_1_tests/6.1.27.19 |
53938 ns/iter (± 60) |
54301 ns/iter (± 95) |
0.99 |
csaf_2_1_tests/6.1.28 |
70402 ns/iter (± 120) |
72058 ns/iter (± 76) |
0.98 |
csaf_2_1_tests/6.1.29 |
47719 ns/iter (± 151) |
45958 ns/iter (± 137) |
1.04 |
csaf_2_1_tests/6.1.30 |
163674 ns/iter (± 289) |
156128 ns/iter (± 298) |
1.05 |
csaf_2_1_tests/6.1.31 |
163822 ns/iter (± 188) |
159454 ns/iter (± 401) |
1.03 |
csaf_2_1_tests/6.1.32 |
47558 ns/iter (± 99) |
46086 ns/iter (± 112) |
1.03 |
csaf_2_1_tests/6.1.33 |
85427 ns/iter (± 140) |
87766 ns/iter (± 226) |
0.97 |
csaf_2_1_tests/6.1.34 |
59087 ns/iter (± 129) |
57229 ns/iter (± 374) |
1.03 |
csaf_2_1_tests/6.1.35 |
79449 ns/iter (± 252) |
77695 ns/iter (± 379) |
1.02 |
csaf_2_1_tests/6.1.36 |
87654 ns/iter (± 367) |
84289 ns/iter (± 384) |
1.04 |
csaf_2_1_tests/6.1.37 |
86274 ns/iter (± 142) |
84492 ns/iter (± 160) |
1.02 |
csaf_2_1_tests/6.1.38 |
47149 ns/iter (± 93) |
46384 ns/iter (± 43) |
1.02 |
csaf_2_1_tests/6.1.39 |
47560 ns/iter (± 99) |
46856 ns/iter (± 181) |
1.02 |
csaf_2_1_tests/6.1.40 |
47885 ns/iter (± 31) |
47389 ns/iter (± 37) |
1.01 |
csaf_2_1_tests/6.1.41 |
48021 ns/iter (± 35) |
46890 ns/iter (± 149) |
1.02 |
csaf_2_1_tests/6.1.42 |
206938 ns/iter (± 1638) |
197882 ns/iter (± 205) |
1.05 |
csaf_2_1_tests/6.1.43 |
174563 ns/iter (± 265) |
168308 ns/iter (± 214) |
1.04 |
csaf_2_1_tests/6.1.44 |
175239 ns/iter (± 339) |
168616 ns/iter (± 403) |
1.04 |
csaf_2_1_tests/6.1.45 |
163461 ns/iter (± 1239) |
164307 ns/iter (± 586) |
0.99 |
csaf_2_1_tests/6.1.46 |
283535 ns/iter (± 804) |
269182 ns/iter (± 250) |
1.05 |
csaf_2_1_tests/6.1.47 |
507750 ns/iter (± 961) |
476012 ns/iter (± 799) |
1.07 |
csaf_2_1_tests/6.1.48 |
88819 ns/iter (± 113) |
87292 ns/iter (± 77) |
1.02 |
csaf_2_1_tests/6.1.49 |
89124 ns/iter (± 170) |
87572 ns/iter (± 102) |
1.02 |
csaf_2_1_tests/6.1.50 |
89254 ns/iter (± 181) |
87752 ns/iter (± 128) |
1.02 |
csaf_2_1_tests/6.1.51 |
179958 ns/iter (± 216) |
170806 ns/iter (± 376) |
1.05 |
csaf_2_1_tests/6.1.52 |
171305 ns/iter (± 564) |
175526 ns/iter (± 145) |
0.98 |
csaf_2_1_tests/6.1.53 |
56566 ns/iter (± 471) |
54651 ns/iter (± 82) |
1.04 |
csaf_2_1_tests/6.1.54 |
64053 ns/iter (± 158) |
63319 ns/iter (± 88) |
1.01 |
csaf_2_1_tests/6.1.55 |
65077 ns/iter (± 181) |
64151 ns/iter (± 77) |
1.01 |
csaf_2_1_tests/6.1.56 |
98879 ns/iter (± 220) |
95440 ns/iter (± 198) |
1.04 |
csaf_2_1_tests/6.1.57 |
131577 ns/iter (± 151) |
136174 ns/iter (± 1009) |
0.97 |
csaf_2_1_tests/6.1.58 |
92423 ns/iter (± 914) |
87032 ns/iter (± 999) |
1.06 |
csaf_2_1_tests/6.1.59 |
91383 ns/iter (± 1176) |
90185 ns/iter (± 110) |
1.01 |
csaf_2_1_tests/6.1.60.1 |
71567 ns/iter (± 329) |
74629 ns/iter (± 918) |
0.96 |
csaf_2_1_tests/6.1.60.2 |
71578 ns/iter (± 80) |
76609 ns/iter (± 654) |
0.93 |
csaf_2_1_tests/6.1.60.3 |
72442 ns/iter (± 89) |
73026 ns/iter (± 156) |
0.99 |
csaf_2_1_tests/6.1.61 |
178917 ns/iter (± 439) |
168337 ns/iter (± 270) |
1.06 |
csaf_2_1_tests/6.2.1 |
723374 ns/iter (± 704) |
723945 ns/iter (± 908) |
1.00 |
csaf_2_1_tests/6.2.2 |
82537 ns/iter (± 675) |
79222 ns/iter (± 254) |
1.04 |
csaf_2_1_tests/6.2.3 |
78326 ns/iter (± 125) |
76631 ns/iter (± 193) |
1.02 |
csaf_2_1_tests/6.2.4 |
70768 ns/iter (± 415) |
66374 ns/iter (± 83) |
1.07 |
csaf_2_1_tests/6.2.5 |
212928 ns/iter (± 472) |
195840 ns/iter (± 383) |
1.09 |
csaf_2_1_tests/6.2.6 |
216924 ns/iter (± 264) |
201513 ns/iter (± 465) |
1.08 |
csaf_2_1_tests/6.2.7 |
63087 ns/iter (± 425) |
63063 ns/iter (± 145) |
1.00 |
csaf_2_1_tests/6.2.8 |
186725 ns/iter (± 396) |
175894 ns/iter (± 839) |
1.06 |
csaf_2_1_tests/6.2.9 |
186891 ns/iter (± 256) |
187554 ns/iter (± 713) |
1.00 |
csaf_2_1_tests/6.2.11 |
561400 ns/iter (± 753) |
571951 ns/iter (± 1120) |
0.98 |
csaf_2_1_tests/6.2.12 |
126950 ns/iter (± 220) |
124412 ns/iter (± 168) |
1.02 |
csaf_2_1_tests/6.2.13 |
3708349 ns/iter (± 13531) |
3581048 ns/iter (± 23245) |
1.04 |
csaf_2_1_tests/6.2.14 |
133336 ns/iter (± 2474) |
126403 ns/iter (± 635) |
1.05 |
csaf_2_1_tests/6.2.15 |
110970 ns/iter (± 2240) |
108757 ns/iter (± 90) |
1.02 |
csaf_2_1_tests/6.2.16 |
235622 ns/iter (± 364) |
219378 ns/iter (± 824) |
1.07 |
csaf_2_1_tests/6.2.17 |
72552 ns/iter (± 91) |
69998 ns/iter (± 112) |
1.04 |
csaf_2_1_tests/6.2.18 |
158867 ns/iter (± 1448) |
151768 ns/iter (± 488) |
1.05 |
csaf_2_1_tests/6.2.19 |
84363 ns/iter (± 74) |
83528 ns/iter (± 139) |
1.01 |
csaf_2_1_tests/6.2.20 |
7928372 ns/iter (± 21975) |
8142952 ns/iter (± 138139) |
0.97 |
csaf_2_1_tests/6.2.21 |
364555 ns/iter (± 1192) |
350545 ns/iter (± 576) |
1.04 |
csaf_2_1_tests/6.2.22 |
167957 ns/iter (± 155) |
163996 ns/iter (± 1063) |
1.02 |
csaf_2_1_tests/6.2.23 |
85102 ns/iter (± 160) |
83200 ns/iter (± 169) |
1.02 |
csaf_2_1_tests/6.2.24 |
87104 ns/iter (± 414) |
83462 ns/iter (± 57) |
1.04 |
csaf_2_1_tests/6.2.25 |
84629 ns/iter (± 138) |
83944 ns/iter (± 595) |
1.01 |
csaf_2_1_tests/6.2.26 |
85666 ns/iter (± 43) |
84170 ns/iter (± 582) |
1.02 |
csaf_2_1_tests/6.2.27 |
85914 ns/iter (± 576) |
84162 ns/iter (± 121) |
1.02 |
csaf_2_1_tests/6.2.28 |
72062 ns/iter (± 99) |
73430 ns/iter (± 263) |
0.98 |
csaf_2_1_tests/6.2.29 |
72086 ns/iter (± 83) |
71173 ns/iter (± 299) |
1.01 |
csaf_2_1_tests/6.2.30 |
72698 ns/iter (± 118) |
71643 ns/iter (± 313) |
1.01 |
csaf_2_1_tests/6.2.31 |
86759 ns/iter (± 97) |
85412 ns/iter (± 124) |
1.02 |
csaf_2_1_tests/6.2.32 |
88245 ns/iter (± 115) |
85572 ns/iter (± 189) |
1.03 |
csaf_2_1_tests/6.2.33 |
253739 ns/iter (± 432) |
231105 ns/iter (± 1035) |
1.10 |
csaf_2_1_tests/6.2.34 |
86181 ns/iter (± 690) |
85916 ns/iter (± 467) |
1.00 |
csaf_2_1_tests/6.2.35 |
89880 ns/iter (± 237) |
86168 ns/iter (± 131) |
1.04 |
csaf_2_1_tests/6.2.36 |
87745 ns/iter (± 101) |
86616 ns/iter (± 329) |
1.01 |
csaf_2_1_tests/6.2.37 |
88078 ns/iter (± 193) |
86845 ns/iter (± 98) |
1.01 |
csaf_2_1_tests/6.2.38 |
385690 ns/iter (± 3704) |
100298 ns/iter (± 184) |
3.85 |
csaf_2_1_tests/6.2.39.1 |
71592 ns/iter (± 281) |
75349 ns/iter (± 695) |
0.95 |
csaf_2_1_tests/6.2.39.2 |
71630 ns/iter (± 142) |
75728 ns/iter (± 150) |
0.95 |
csaf_2_1_tests/6.2.39.3 |
71573 ns/iter (± 791) |
75061 ns/iter (± 695) |
0.95 |
csaf_2_1_tests/6.2.39.4 |
71610 ns/iter (± 108) |
74700 ns/iter (± 177) |
0.96 |
csaf_2_1_tests/6.2.39.5 |
72382 ns/iter (± 45) |
75236 ns/iter (± 646) |
0.96 |
csaf_2_1_tests/6.2.40 |
88617 ns/iter (± 193) |
90499 ns/iter (± 208) |
0.98 |
csaf_2_1_tests/6.2.41 |
213462 ns/iter (± 771) |
196650 ns/iter (± 466) |
1.09 |
csaf_2_1_tests/6.2.42 |
88915 ns/iter (± 740) |
87777 ns/iter (± 178) |
1.01 |
csaf_2_1_tests/6.2.43 |
89166 ns/iter (± 498) |
88032 ns/iter (± 92) |
1.01 |
csaf_2_1_tests/6.2.44 |
89489 ns/iter (± 241) |
88331 ns/iter (± 90) |
1.01 |
csaf_2_1_tests/6.2.45 |
89805 ns/iter (± 3169) |
88712 ns/iter (± 200) |
1.01 |
csaf_2_1_tests/6.2.46 |
92648 ns/iter (± 780) |
88829 ns/iter (± 239) |
1.04 |
csaf_2_1_tests/6.2.47 |
90052 ns/iter (± 85) |
89007 ns/iter (± 144) |
1.01 |
csaf_2_1_tests/6.2.48 |
176021 ns/iter (± 153) |
171995 ns/iter (± 725) |
1.02 |
csaf_2_1_tests/6.2.49 |
90856 ns/iter (± 556) |
89580 ns/iter (± 478) |
1.01 |
csaf_2_1_tests/6.2.50.1 |
71614 ns/iter (± 100) |
75207 ns/iter (± 600) |
0.95 |
csaf_2_1_tests/6.2.50.2 |
71574 ns/iter (± 58) |
75100 ns/iter (± 809) |
0.95 |
csaf_2_1_tests/6.2.50.3 |
71643 ns/iter (± 70) |
73370 ns/iter (± 549) |
0.98 |
csaf_2_1_tests/6.2.51 |
90882 ns/iter (± 162) |
89832 ns/iter (± 541) |
1.01 |
csaf_2_1_tests/6.2.52 |
209836 ns/iter (± 262) |
200178 ns/iter (± 251) |
1.05 |
csaf_2_1_tests/6.2.53 |
82662 ns/iter (± 709) |
80875 ns/iter (± 268) |
1.02 |
csaf_2_1_tests/6.2.54.1 |
71667 ns/iter (± 1391) |
74046 ns/iter (± 253) |
0.97 |
csaf_2_1_tests/6.2.54.2 |
71602 ns/iter (± 500) |
75034 ns/iter (± 166) |
0.95 |
csaf_2_1_tests/6.2.54.3 |
71613 ns/iter (± 68) |
75131 ns/iter (± 60) |
0.95 |
csaf_2_1_tests/6.2.54.4 |
71598 ns/iter (± 263) |
74863 ns/iter (± 102) |
0.96 |
csaf_2_1_tests/6.3.1 |
73199 ns/iter (± 625) |
75154 ns/iter (± 226) |
0.97 |
csaf_2_1_tests/6.3.2 |
100077 ns/iter (± 109) |
98477 ns/iter (± 473) |
1.02 |
csaf_2_1_tests/6.3.3 |
73434 ns/iter (± 277) |
75733 ns/iter (± 81) |
0.97 |
csaf_2_1_tests/6.3.4 |
73413 ns/iter (± 262) |
75726 ns/iter (± 58) |
0.97 |
csaf_2_1_tests/6.3.5 |
72623 ns/iter (± 68) |
74554 ns/iter (± 103) |
0.97 |
csaf_2_1_tests/6.3.6 |
73665 ns/iter (± 60) |
75428 ns/iter (± 733) |
0.98 |
csaf_2_1_tests/6.3.7 |
73656 ns/iter (± 103) |
77299 ns/iter (± 311) |
0.95 |
csaf_2_1_tests/6.3.8 |
73957 ns/iter (± 367) |
76630 ns/iter (± 276) |
0.97 |
csaf_2_1_tests/6.3.9 |
190517 ns/iter (± 1071) |
192895 ns/iter (± 234) |
0.99 |
csaf_2_1_tests/6.3.10 |
89227 ns/iter (± 342) |
88081 ns/iter (± 82) |
1.01 |
csaf_2_1_tests/6.3.11 |
89548 ns/iter (± 182) |
88056 ns/iter (± 97) |
1.02 |
csaf_2_1_tests/6.3.12 |
251085 ns/iter (± 539) |
236631 ns/iter (± 1773) |
1.06 |
csaf_2_1_tests/6.3.13 |
89875 ns/iter (± 123) |
88677 ns/iter (± 88) |
1.01 |
csaf_2_1_tests/6.3.14 |
90248 ns/iter (± 877) |
92576 ns/iter (± 101) |
0.97 |
csaf_2_1_tests/6.3.15 |
90351 ns/iter (± 103) |
89197 ns/iter (± 72) |
1.01 |
csaf_2_1_tests/6.3.16 |
90569 ns/iter (± 163) |
89504 ns/iter (± 298) |
1.01 |
csaf_2_1_tests/6.3.17 |
90783 ns/iter (± 765) |
89519 ns/iter (± 44) |
1.01 |
csaf_2_1_tests/6.3.18 |
101437 ns/iter (± 374) |
99454 ns/iter (± 219) |
1.02 |
csaf_2_1_tests/6.3.19.1 |
71585 ns/iter (± 110) |
74520 ns/iter (± 592) |
0.96 |
csaf_2_1_tests/6.3.19.2 |
71607 ns/iter (± 90) |
74660 ns/iter (± 336) |
0.96 |
csaf_2_1_tests/6.3.19.3 |
71633 ns/iter (± 82) |
75125 ns/iter (± 339) |
0.95 |
csaf_2_1_tests/6.3.19.4 |
71570 ns/iter (± 183) |
75444 ns/iter (± 92) |
0.95 |
csaf_2_1_tests/6.3.19.5 |
71562 ns/iter (± 87) |
73059 ns/iter (± 445) |
0.98 |
csaf_2_1_tests/6.3.20 |
101960 ns/iter (± 624) |
100245 ns/iter (± 141) |
1.02 |
csaf_2_1_tests/6.3.21.1 |
71568 ns/iter (± 90) |
75159 ns/iter (± 101) |
0.95 |
csaf_2_1_tests/6.3.21.2 |
71576 ns/iter (± 268) |
75438 ns/iter (± 213) |
0.95 |
csaf_2_1_tests/6.3.21.3 |
71586 ns/iter (± 146) |
76552 ns/iter (± 2004) |
0.94 |
csaf_2_1_tests/6.3.21.4 |
71635 ns/iter (± 1736) |
74545 ns/iter (± 864) |
0.96 |
csaf_2_1_tests/6.3.21.5 |
71658 ns/iter (± 155) |
73216 ns/iter (± 1560) |
0.98 |
csaf_2_1_tests/6.3.21.6 |
71592 ns/iter (± 50) |
74526 ns/iter (± 45) |
0.96 |
csaf_2_1_tests/6.3.21.7 |
71584 ns/iter (± 84) |
74181 ns/iter (± 72) |
0.96 |
csaf_2_1_tests/6.3.21.8 |
71566 ns/iter (± 108) |
74520 ns/iter (± 38) |
0.96 |
csaf_2_1_tests/6.3.21.9 |
71626 ns/iter (± 122) |
74541 ns/iter (± 212) |
0.96 |
csaf_2_1_tests/6.3.22 |
91315 ns/iter (± 110) |
90416 ns/iter (± 135) |
1.01 |
full_validation/csaf_2_0_full_preset |
7570565 ns/iter (± 17037) |
7110941 ns/iter (± 82517) |
1.06 |
full_validation/csaf_2_1_full_preset |
51625764 ns/iter (± 1406529) |
50067356 ns/iter (± 292705) |
1.03 |
parse_only/csaf_2_0 |
1150895 ns/iter (± 7233) |
1191124 ns/iter (± 1858) |
0.97 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
⚠️ Performance Alert ⚠️
Possible performance regression was detected for benchmark 'CSAF Validation Benchmark'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.50.
| Benchmark suite | Current: 2f89ec8 | Previous: b723aec | Ratio |
|---|---|---|---|
csaf_2_1_tests/6.2.38 |
385690 ns/iter (± 3704) |
100298 ns/iter (± 184) |
3.85 |
This comment was automatically generated by workflow using github-action-benchmark.
aschierl-xitaso
force-pushed
the
feat/non-strict-parsing
branch
from
96fc1df to
4aa1491
Compare
aschierl-xitaso
force-pushed
the
feat/non-strict-parsing
branch
from
4aa1491 to
80c9a22
Compare
aschierl-xitaso
force-pushed
the
feat/non-strict-parsing
branch
from
ef355b8 to
2f89ec8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add infrastructure to do non-strict JSON data extraction for test cases, supporting to work on
serde_json::Value(not necessarily conforming to the CSAF schema, in contrast to the previous typify approach)String/Reader(not necessarily containing syntactically valid JSON, parsing up to the first error)Test cases then consist of two steps:
This PR also adds a temporary wrapper that allows using the
TwoStageValidationas aRawValidator(in the future, the extraction steps for all tests could be executed within one JSON traversal)
and prototypically implements one test using the new mechanism.
For the rough idea, see #571, although some details have been implemented differently)