forked from zeek/zeek
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathCHANGES
22366 lines (14224 loc) · 771 KB
/
CHANGES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
3.2.0-dev.475 | 2020-05-07 17:15:23 -0700
* GH-958: Fix crash when trying to redef non-existing enum (Johanna Amann, Corelight)
3.2.0-dev.473 | 2020-05-06 10:40:09 -0700
* Revert addition of final modifier to JSON formatter (Tim Wojtulewicz, Corelight)
3.2.0-dev.471 | 2020-05-06 10:00:58 -0700
* Fix global buffer over-read in POP3 analyzer (Justin Azoff, Corelight)
* Fix SSL scripting error leading to access of unitialized field (Jon Siwek, Corelight)
Reported by Justin Azoff
* Remove outdated comment on set_to_regex. (Johanna Amann, Corelight)
We can add patterns at runtime since 2.6.
3.2.0-dev.467 | 2020-05-04 18:00:35 -0700
* GH-952: Correct spelling of DCE/RPC operation string NetrLogonSameLogonWithFlags
(Jon Siwek, Corelight)
3.2.0-dev.466 | 2020-05-04 17:50:14 -0700
* Add network_time_init() event. (Jan Grashoefer)
This event is generated upon first initialization of network_time.
3.2.0-dev.461 | 2020-05-04 17:08:46 -0700
* Avoid scheduling multiple inactivity timers (Justin Azoff and Jon Siwek, Corelight)
Also updated language.expire_subnet btest which is unduly sensitive to
timer-related changes
3.2.0-dev.459 | 2020-05-01 17:46:20 -0700
* Extend CI config to cover building with libmaxminddb support (Jon Siwek, Corelight)
* Ensure time continues moving forward if a pcap source is suspended (Tim Wojtulewicz, Corelight)
3.2.0-dev.455 | 2020-05-01 09:44:30 -0700
* GH-938: fix IO loop iterations sometimes skipping offline pcap sources (Jon Siwek, Corelight)
3.2.0-dev.451 | 2020-04-29 16:28:34 -0700
* Organized and added to the shipped file identification signatures. (Seth Hall, Corelight)
- Added ISO 9660 disk image
- Created new files for categorizing signatures better.
- executable.sig - Executable (and bytecode) files.
- java.sig - Java related files (class/jar, etc).
- programming.sig - Mostly scripting language identification
3.2.0-dev.447 | 2020-04-29 15:55:03 -0700
* GH-713: Fixed misc/stats.zeek skipping a log entry on termination (Brittany Donowho)
3.2.0-dev.445 | 2020-04-29 15:25:03 -0700
* Add warning message for unknown Broker statuses (Jon Siwek, Corelight)
There's now a couple placeholder/unimplemented status values in Broker
related to upcoming routing features that we don't want to handle
explicitly for compatibility reasons, but also don't want the compiler
warning about unhandled values in the switch.
3.2.0-dev.443 | 2020-04-28 17:10:38 -0700
* GH-941: Fix build when configured to use libmaxminddb (Jon Siwek, Corelight)
3.2.0-dev.441 | 2020-04-27 13:34:22 -0700
* Fix a few more IntrusivePtr deprecation warnings (Tim Wojtulewicz, Corelight)
* Fix cloning of TypeType values (Vern Paxson, Corelight)
3.2.0-dev.437 | 2020-04-27 19:30:24 +0000
* GH-854: provide access to original HTTP/MIME header names
The "http_header" event now has an "original_name" parameter that allows
access to the original header name (the "name" parameter reamins the
same as before: it's the uppercased header name).
The "mime_header_rec" record type now also includes an "original_name"
field to similarly provide access to original header name in the
following events: "http_all_headers", "mime_one_header", and
"mime_all_headers". (Jon Siwek, Corelight)
* Remove error message from empty bloomfilter lookups
If a bloomfilter doesn't have a type, that just means no
bloomfilter_add() has been called yet, so seems undesirable to emit an
error for a lookup against something that's known to be empty. (Jon Siwek, Corelight)
* unused variables found via use-def analysis (plus an indentation micro-nit) (Vern Paxson, Corelight)
3.2.0-dev.431 | 2020-04-27 12:09:30 -0700
* Update various BIFs to return IntrusivePtr (Jon Siwek, Corelight)
3.2.0-dev.428 | 2020-04-24 16:19:45 -0700
* Deprecate returning Val* from BIFs (Jon Siwek, Corelight)
* Deprecate binpac::string_to_val (Jon Siwek, Corelight)
* Deprecate binpac::bytestring_to_val, replace with binpac::to_stringval (Jon Siwek, Corelight)
* Update deprecated BifEvent::generate_* usages (Jon Siwek, Corelight)
* Deprecate Connection::Event and Analyzer::Event methods
And update usages to the "EnqueueEvent" methods. (Jon Siwek, Corelight)
* Deprecate BuildConnVal() methods and update usages to ConnVal()
The later being a new method that returns IntrusivePtr (Jon Siwek, Corelight)
* Update all BIFs to return IntrusivePtr instead of Val* (Jon Siwek, Corelight)
* Update deprecated ValManager::GetPort usages (Jon Siwek, Corelight)
* Update deprecated ValManager::GetEmptyString usages (Jon Siwek, Corelight)
* Update deprecated ValManager::GetCount usages (Jon Siwek, Corelight)
* Update deprecated ValManager::GetInt usages (Jon Siwek, Corelight)
* Update deprecated ValManager::GetBool usages (Jon Siwek, Corelight)
* Update deprecated ValManager GetTrue/GetFalse usages (Jon Siwek, Corelight)
* Deprecate all ValManager "Get" methods
Alternate methods that return IntrusivePtr are available in similarly
named methods that omit the "Get" prefix. (Jon Siwek, Corelight)
* Change BIFs to return a wrapper object
That allows returning either Val* or IntrusivePtr<T>. The former could
eventually be deprecated, but it's used extensively at the moment. (Jon Siwek, Corelight)
3.2.0-dev.412 | 2020-04-22 10:43:39 -0700
* Fix buffer over-read in Ident analyzer (Max Kellermann)
3.2.0-dev.410 | 2020-04-21 10:25:29 -0700
* Remove "connection_external" event (Johanna Amann, Corelight)
This event was forgotten in our broccoli cleanup. It cannot be
raised by anything anymore
3.2.0-dev.405 | 2020-04-16 15:38:46 -0700
* Remove ineffective &default in netcontrol cluster event handler args (Jon Siwek, Corelight)
3.2.0-dev.404 | 2020-04-16 14:12:20 -0700
* GH-895: Remove use of Variable-Length-Arrays (Jon Siwek, Corelight)
3.2.0-dev.402 | 2020-04-16 13:00:59 -0700
* Add default function for Kerberos constant-lookup-tables (Jon Siwek, Corelight)
* check for the existance of f?$conns in file_sniff event in policy/protocols/ssl/log-hostcerts-only.zeek
(SG)
* Update README.md to fix the logo and one of the links (Tim Wojtulewicz, Corelight)
3.2.0-dev.396 | 2020-04-14 15:45:54 -0700
* A few minor cleanups in Dict (Tim Wojtulewicz, Corelight)
3.2.0-dev.394 | 2020-04-14 15:29:00 -0700
* Fix a confusing variable name shadowing (Jon Siwek, Corelight)
3.2.0-dev.392 | 2020-04-14 11:06:04 -0700
* Fix Stack Overflow in POP3_Analyzer::ProcessRequest. (Johanna Amann, Corelight)
The VLA can overflow given a large enough string. As a small fix, this
commit gets rid of the VLA and assigns the password directly to the
target string.
This was reported by Matteo Rizzo (Google).
3.2.0-dev.391 | 2020-04-13 14:59:50 -0700
* Add warning for ineffective &default arguments in handlers
For event/hook handlers that had a previous declaration, any &default
arguments are ineffective. Only &default uses in the initial
prototype's arguments have an effect (that includes if the handler
is actually the site at which the declaration occurs). (Jon Siwek, Corelight)
* Fix frame size allocation of alternate event/hook handlers
When the only existing handler doesn't use the full argument list, still
have to ensure the full frame is allocated because all arguments get set
regardless. (Jon Siwek, Corelight)
* Emit error for alternate event/hook prototype args with attributes
Argument attributes are only allowed in the canonical prototype. (Jon Siwek, Corelight)
* Improve alternate event/hook prototype matching
This fixes it to again allow the old behavior of matching a handler
against the canonical prototype as long as all argument types, but not
necessarily names, match. (Jon Siwek, Corelight)
* Allow alternate event/hook prototype declarations
The alternates must be some subset of the canonical prototype (the one
that's first declared) and allows users to define handlers for any
such prototype. Example:
# Prototype declarations
global my_event: event(s: string, c: count);
global my_event: event(c: count);
global my_event: event();
# Handler definitions
event my_event(s: string, c: count) { print s, c; }
event my_event(c: count) { print c; }
event my_event() { }
This allows handlers to consume a subset of the arguments or even
re-order them. This makes it easier to either extend an existing
event/hook's arguments and/or deprecate usages of certain prototypes. (Jon Siwek, Corelight)
* Fix Broker topics used to uniquely identify cluster nodes
Node-specific topic prefix subscriptions/publications now add a trailing
slash like "zeek/cluster/node/<name>/". Without the trailing slash,
messages attempting to target "proxy-10" may also be sent to "proxy-1"
since subscription matching is prefix-based. (Jon Siwek, Corelight)
* Update submodule(s)
[nomail] (Jon Siwek, Corelight)
* Fix include in X509Common.h
Give full path to Analyzer.h, so that X509Common.h is include-able by
plugins. (Johanna Amann, Corelight)
3.2.0-dev.382 | 2020-04-09 13:17:03 -0700
* Remove 'using namespace std' as well as other using statements from headers.
This unfortunately cuases a ton of flow-down changes because a lot of other
code was depending on that definition existing. This has a fairly large chance
to break builds of external plugins, considering how many internal ones it broke. (Tim Wojtulewicz, Corelight)
3.2.0-dev.378 | 2020-04-09 08:47:44 -0700
* Replace most of the uses of 0 or NULL to indicate null pointers with nullptr.
This change does not change any calls to syscalls, in the interest of passing
what the API for those calls says to pass. (Tim Wojtulewicz, Corelight)
3.2.0-dev.372 | 2020-04-08 14:00:28 -0700
* Lazy-initalize some of the fields in Frame to reduce size (Tim Wojtulewicz, Corelight)
* Set InternalHashTag to a uint16_t so CompositeHash doesn't have a gap in it.
Resizes CompositeHash from 32 bytes to 24 bytes. (Tim Wojtulewicz, Corelight)
* Mark constants in List constexpr so they don't actually take up space in created objects
This resizes List from 24 bytes to 16 bytes. (Tim Wojtulewicz, Corelight)
* Reorder some class variables to fill in gaps in structure packing (Tim Wojtulewicz, Corelight)
* Mark classes as final, where appropriate. (Tim Wojtulewicz, Corelight)
These classes were recommended by gcc's -Wsugggest-final-types as places where
large numbers of virtual functions could be optimized away.
3.2.0-dev.360 | 2020-04-07 14:12:27 -0700
* Add "udp_content_ports" option
Any port in that set found as either source or destination port
of a UDP packet will cause the "udp_contents" event to be raised. (Jon Siwek, Corelight)
* Add new "udp_content_delivery_ports_use_resp" option
This controls whether ports given by "udp_content_delivery_ports_orig" and
"udp_content_delivery_ports_orig" are in terms of the UDP packet's
destination port or by the Connection's "responder" port (the former is
the unchanged default behavior). (Jon Siwek, Corelight)
3.2.0-dev.357 | 2020-04-07 13:43:38 -0700
* Fix connection duration thresholding
Connection durations were being compared "less than" instead of "greater
than" the desired duration threshold as provided to
set_current_conn_bytes_threshold() or
ConnThreshold::set_duration_threshold() (Jon Siwek, Corelight)
3.2.0-dev.355 | 2020-04-07 12:05:17 -0700
* Fix using patterns stored as table/set indices
Pattern values used as table/set indices are stored in serialized form
using just the pattern text, but re-creating the value from that didn't
fully initialize/compile the regex matcher after (re-)creating it from
that pattern text. (Jon Siwek, Corelight)
3.2.0-dev.353 | 2020-04-06 17:27:35 -0700
* Improve parsing of RDPEUDP/RDPEUDP2 (Anthony Kasza, Corelight)
3.2.0-dev.338 | 2020-04-02 18:29:23 -0700
* Add RDP over UDP analyzer (Anthony Kasza, Corelight)
3.2.0-dev.336 | 2020-04-02 15:03:04 -0700
* Fix uses of GetBool in bifs to use GetTrue/GetFalse (Tim Wojtulewicz, Corelight)
* Change Sessions::dump_this_packet to a bool (Tim Wojtulewicz, Corelight)
3.2.0-dev.332 | 2020-03-31 15:34:52 -0700
* GH-725: fix logic for finding a lambda's usage of outer IDs (Jon Siwek, Corelight)
* Change record field anonymous functions to use lambda expressions
There was an alternate syntax to assign anonymous functions to record
fields that was never migrated to use the new lambda expression
machinery (and so didn't allow referencing variables in outer scope):
type myrec: record {
foo: function(a: string);
};
local o = "o";
local mr = myrec($foo(a: string) = { print a + o; });
(Jon Siwek, Corelight)
3.2.0-dev.329 | 2020-03-31 08:48:10 -0700
* Mark input/output message classes as final, since nothing should be inheriting from them (Tim Wojtulewicz, Corelight)
* Add missing override specifier to a number of methods, remove virtual from some as well (Tim Wojtulewicz, Corelight)
* Add override specifier to Configure() method in almost all of the internal plugins (Tim Wojtulewicz, Corelight)
3.2.0-dev.325 | 2020-03-31 07:04:59 +0000
* Fix bug in intel framework letting deletes of mixed-cased entries
fail. (Antonio Nappa, Corelight)
3.2.0-dev.322 | 2020-03-31 06:44:57 +0000
* A large number of functions had return values and/or arguments
changed to use ``bool`` types instead of ``int``. (Tim
Wojtulewicz, Corelight)
3.2.0-dev.320 | 2020-03-30 14:09:23 -0700
* GH-887: improve GRE/ERSPAN parsing of non-IPv4/IPv6 inner payload
This changes the decapsulation logic for GRE/ERSPAN payloads to re-use
existing Layer 2 parsing logic that already handles things like 802.1Q
tags correctly before going on to process the inner IPv4/IPv6 payload. (Jon Siwek, Corelight)
* Fix ERSPAN III optional sub-header parsing (Jon Siwek, Corelight)
* Many small cleanups of Expr.cc (Tim Wojtulewicz, Corelight)
* Remove unused cq queue code (Tim Wojtulewicz, Corelight)
* Add a "reserved" field to tcp_hdr record
This record is available by handling either raw_packet or new_packet
events. (Jon Siwek, Corelight)
* GH-878: Make RocksDB usage opt-in and add --enable-rocksdb configure flag (Jon Siwek, Corelight)
3.2.0-dev.300 | 2020-03-27 11:48:46 -0700
* Func::DescribeDebug: move a NumFields() call out of loop (Jon Siwek, Corelight)
* Use const-ref parameter for zeek::val_list_to_args()
It ended up being used a bit more than initially expected and this
is closer to the style we're generally aiming for. (Jon Siwek, Corelight)
* Fix missing IntrusivePtr.h include and ambiguous ODesc::Add call (Jon Siwek, Corelight)
* Remove TimerMgr arg from event queuing/scheduling methods
It's not useful for anything since there's only ever a single TimerMgr. (Jon Siwek, Corelight)
* Deprecate Analyzer::ConnectionEvent()
And update usages to Analyzer::EnqueueConnEvent() (Jon Siwek, Corelight)
* Deprecate file_analysis::File::FileEvent methods using val_list args
And update usages to the overload that takes a zeek::Args instead. (Jon Siwek, Corelight)
* Deprecate Connection::ConnectionEvent methods
And update usages to Connection::EnqueueEvent (Jon Siwek, Corelight)
* Deprecate EventMgr::QueueEventFast() and update usages to Enqueue() (Jon Siwek, Corelight)
* Deprecate EventMgr::QueueEvent() and update usages to Enqueue() (Jon Siwek, Corelight)
* Deprecate Func::Call(val_list*, ...)
The version taking a vector of intrusive pointers should be used
instead. A variadic version is also added that forwards all arguments. (Jon Siwek, Corelight)
* Use vector<IntrusivePtr<Val>> for Func::Call and Event queuing args
This change may break BIFs that use @ARGS@, @ARG@, or @ARGC@ since their
types have changed. (Jon Siwek, Corelight)
* Fix memory leak in Zeek when-statement bodies with runtime errors (Jon Siwek, Corelight)
* Change TableVal::RecoverIndex() to return IntrusivePtr (Jon Siwek, Corelight)
* Use IntrusivePtr in TableVal::CallExpireFunc (Jon Siwek, Corelight)
* Fix memory leak when runtime error occurs in a Zeek for-loop (Jon Siwek, Corelight)
* Enable leak checks for btests that produce runtime exceptions
These were previously reporting leaks due to various allocations not
getting cleaned up during the stack unwind, but at the current state of
the transition toward IntrusivePtr usage, theses tests no longer leak. (Jon Siwek, Corelight)
3.2.0-dev.280 | 2020-03-24 13:49:43 -0700
* Add test case for binpac flowbuffer frame length parsing bug (Jon Siwek, Corelight)
3.2.0-dev.277 | 2020-03-20 11:29:36 -0700
* Initialize RecordVal default fields when redef'd (Jon Siwek, Corelight)
If a RecordVal had been created, but later its RecordType redef'd to
contain fields with &default, those fields were incorrectly left
uninitialized.
* GH-857: fix redefining record types used to index tables (Jon Siwek, Corelight)
This change tracks all TableVals created at parse-time whose index
depends on a given RecordType. Should that RecordType be redef'd, those
TableVals are immediately rebuilt such that they are valid to
subsequently use in either parse-time initializations or eventually in
any arbitrary run-time expression.
* Change RecordVals to get resized at time of RecordType redef (Jon Siwek, Corelight)
Opposed to unconditionally checking all RecordVals whether they need to
be resized after parsing ends.
3.2.0-dev.273 | 2020-03-20 10:05:21 -0700
* GH-865: fix parsing of SMB NegotiateContextList
* The compression capability was incorrectly set to 0x0004 instead of 0x0003
* The padding was 4-byte instead of 8-byte aligned and also the spec.
does not strictly require the padding for the last item in the list.
* Add a default case to handle parsing of unknown context types. (Jon Siwek, Corelight)
3.2.0-dev.271 | 2020-03-17 22:52:40 -0700
* Scope: store IntrusivePtr in `local` (Max Kellermann)
* Scope: pass IntrusivePtr to AddInit() (Max Kellermann)
* DNS_Mgr: use class IntrusivePtr (Max Kellermann)
* Scope: use class IntrusivePtr (Max Kellermann)
* Attr: use class IntrusivePtr (Max Kellermann)
* Expr: check_and_promote_expr() returns IntrusivePtr (Max Kellermann)
Instead of returning a pseudo-boolean integer, it now returns a
referenced object or nullptr on error. The old API was very error
prone because of its obscure reference counting semantics.
* Frame: use class IntrusivePtr (Max Kellermann)
* Val: RecordVal::LookupWithDefault() returns IntrusivePtr (Max Kellermann)
* Type: RecordType::FieldDefault() returns IntrusivePtr (Max Kellermann)
Fixes memory leak in EventHandler::NewEvent().
* Val: TableVal::Delete() returns IntrusivePtr (Max Kellermann)
* Type: base_type() returns IntrusivePtr (Max Kellermann)
* Type: init_type() returns IntrusivePtr (Max Kellermann)
* Type: merge_types() returns IntrusivePtr (Max Kellermann)
* Type: use class IntrusivePtr in VectorType (Max Kellermann)
* Type: use class IntrusivePtr in EnumType (Max Kellermann)
* Type: use class IntrusivePtr in FileType (Max Kellermann)
* Type: use class IntrusivePtr in TypeDecl (Max Kellermann)
* Type: make TypeDecl `final` and the dtor non-`virtual` (Max Kellermann)
* Type: use class IntrusivePtr in TypeType (Max Kellermann)
* Type: use class IntrusivePtr in FuncType (Max Kellermann)
* Type: use class IntrusivePtr in TypeList (Max Kellermann)
* Type: use class IntrusivePtr in IndexType (Max Kellermann)
* Val: use class IntrusivePtr in class TableVal (Max Kellermann)
* Val: use class IntrusivePtr in class TableEntryVal (Max Kellermann)
* CompHash: return IntrusivePtr (Max Kellermann)
* Type: return IntrusivePtr (Max Kellermann)
* Val: add TableVal::Assign() overload with IntrusivePtr (Max Kellermann)
* zeekygen/IdentifierInfo: use class IntrusivePtr more (Max Kellermann)
* Func: use class IntrusivePtr (Max Kellermann)
* GH-845: reference `id_list` before passing to new `BroFunc` (Max Kellermann)
Passing the `id_list` pointer to `BroFunc` transfers ownership of the
contained `ID` instances, because `~BroFunc()` unreferences them.
Therefore, we need to increase the reference counters for each
`BroFunc` instance to fix the use-after-free bug.
* input/Manager: fix three use-after-free bugs (Max Kellermann)
3.2.0-dev.237 | 2020-03-17 16:54:35 -0700
* Cleanup in iosource/Packet (Tim Wojtulewicz, Corelight)
* Add ability for packet sources to flag a packet's l2 or l3 checksum as valid.
This lets packet source plugins implement handling of hardware checksum
offloading, if available. Setting the flags will skip the internal checksumming
for either layer 2 and/or layer 3. (Tim Wojtulewicz, Corelight)
3.2.0-dev.234 | 2020-03-16 12:37:17 -0700
* Remove an unused smb2_negotiate_response event handler. (M.Shirk)
3.2.0-dev.231 | 2020-03-13 13:41:23 -0700
* Add branch whitelist for CI notifications (Jon Siwek, Corelight)
3.2.0-dev.230 | 2020-03-12 13:33:09 -0700
* Introduce X509 certificate caching.
This change introduces caching of X509 certificates. If a certificate is
encountered more than 10 times in a minute, it will be cached for the future
(until it is not seen for more than a minute). The caching means that the
base scripts cache the certificate information and use cached information
to raise events.
This means certificates do not have to be repeatedly parsed; furthermore
the events are passed "recycled" Vals (so data is not repeatedly copied around).
This new feature is highly configureable - see the x509 main.zeek for full
information. You can both change the timeouts, disable the feature, or
even stop X509 events from being raised (by hooking X509::x509_certificate_cache_replay).
(Johanna Amann, Corelight)
3.2.0-dev.223 | 2020-03-12 12:03:48 -0700
* Update pointer to external test repo (Jon Siwek, Corelight)
3.2.0-dev.222 | 2020-03-11 16:39:53 -0700
* Fix whitespace in connection record comments (Jon Siwek, Corelight)
3.2.0-dev.221 | 2020-03-11 11:21:20 -0700
* Made additional MySQL fixes. (Vlad Grigorescu)
1) There are a couple more places where the new protocol uses and OK
packet instead of the deprecated EOF.
2) With > 255 results, we could end up in an situation where the uint8
sequence number would wrap, and we'd naively think it'd be a new
handshake.
Now, we track the previous sequence number, and assume overflow if it
was 255 previously and 0 now.
We also reset the previous sequence number to 0 in various packets
that we'd expect at the end of other commands.
* Add support to MySQL for deprecation of EOF packets. (Vlad Grigorescu)
From the docs: "As of MySQL 5.7.5, OK packes are also used to indicate
EOF, and EOF packets are deprecated."
The client sets a capability flag (CLIENT_DEPRECATE_EOF) to indicate
that it expects an OK instead of an EOF after the resultset rows.
* MySQL analyzer whitespace cleanup (Vlad Grigorescu)
* Fix EOF detection in the MySQL protocol analyzer. (Vlad Grigorescu)
The MySQL documentation
(https://dev.mysql.com/doc/dev/mysql-server/latest/page_protocol_basic_eof_packet.html)
warns us that "You must check whether the packet length is less than 9 to
make sure that it is a EOF_Packet packet."
While we were doing this in two places, we were comparing the total
packet length, which includes the 4-byte header. Changed to compare to
13 instead.
3.2.0-dev.214 | 2020-03-09 13:35:26 -0700
* Stop running GitHub Actions in forked repos (Jon Siwek, Corelight)
* Remove Travis CI configuration (Jon Siwek, Corelight)
* Add GitHub Action for Coverity Scan (Jon Siwek, Corelight)
3.2.0-dev.209 | 2020-03-06 16:27:29 -0800
* Add urgent flag to tcp_flags documentation (Tim Wojtulewicz, Corelight)
3.2.0-dev.205 | 2020-03-06 07:59:08 -0800
* Move libkqueue submodule to the aux directory (Jon Siwek, Corelight)
* Move rapidjson submodule to aux directory
This also now installs the full rapidjson include/ tree in order to
allow external plugins to properly build if they include
threading/formatters/JSON.h (Jon Siwek, Corelight)
* GH-831: Fix the "bro" symlink for binary packaging mode. (Jon Siwek, Corelight)
3.2.0-dev.198 | 2020-03-05 14:18:10 -0800
* Fix a btest to order zeek_init() events via &priority (Jon Siwek, Corelight)
3.2.0-dev.192 | 2020-02-28 11:36:12 -0800
* threading::Queue::WakeUp(): lock mutex before notifying condition_variable (Max Kellermann)
Not locking the associated mutex can create race conditions and lockups.
3.2.0-dev.190 | 2020-02-28 00:42:17 -0800
* Stmt: use class IntrusivePtr (Max Kellermann)
* Stmt: remove unused default constructors and `friend` declarations (Max Kellermann)
* Val: remove unimplemented prototype recover_val() (Max Kellermann)
* Val: cast_value_to_type() returns IntrusivePtr (Max Kellermann)
* Val: use IntrusivePtr in check_and_promote() (Max Kellermann)
* Val: use nullptr instead of 0 (Max Kellermann)
* zeekygen: use class IntrusivePtr (Max Kellermann)
* ID: use class IntrusivePtr (Max Kellermann)
* Expr: use class IntrusivePtr (Max Kellermann)
* Var: copy Location to stack, to fix use-after-free crash bug (Max Kellermann)
* Scope: lookup_ID() and install_ID() return IntrusivePtr<ID> (Max Kellermann)
* Scope: delete duplicate locals (Max Kellermann)
* EventRegistry: automatically delete EventHandlers (Max Kellermann)
* main: destroy event_registry after iosource_mgr (Max Kellermann)
Fixes use-after-free bugs because PcapSource::Close() queues an event.
* zeekygen/IdentifierInfo: delete duplicate fields (Max Kellermann)
* main: free the global scope in terminate_bro() (Max Kellermann)
Make valgrind a bit happier.
* Scope: pop_scope() returns IntrusivePtr<> (Max Kellermann)
* Scope: unref all inits in destructor (Max Kellermann)
* Var: pass IntrusivePtr to add_global(), add_local() etc. (Max Kellermann)
* plugin/ComponentManager: hold a reference to the EnumType (Max Kellermann)
Use class IntrusivePtr<>.
* Type: fix use-after-free bug in VectorType::ShallowClone() (Max Kellermann)
The new VectorType instance needs a new reference to the `yield_type`.
* Var: fix crash when redeclaring a function with different args (Max Kellermann)
3.2.0-dev.167 | 2020-02-27 14:24:55 -0800
* Expr: use fmt instead of sprintf (Tim Wojtulewicz, Corelight)
* Expr: other minor initialization cleanup (Tim Wojtulewicz, Corelight)
* Expr: use List::empty() (Tim Wojtulewicz, Corelight)
* Expr: Convert a bunch of methods returning ints to return bools (Tim Wojtulewicz, Corelight)
* IPAddr: minor cleanup (Tim Wojtulewicz, Corelight)
- Mark empty constructors/destructors as default
- Initialization cleanup
- Remove unnecessary elses from before returns
* PriorityQueue: initialization cleanup (Tim Wojtulewicz, Corelight)
* IP: Cleanup initialization, make a few functions consistent (Tim Wojtulewicz, Corelight)
3.2.0-dev.159 | 2020-02-26 19:51:24 -0800
* Pop global frame stack on exception. (Johanna Amann, Corelight)
When changing this code for 3.0.0, the code-flow changed significantly.
One of the changes lead to exception being re-thrown from Call - which
leads to the function cleanup not always being run. While the Unref of
the framepointer was (correctly) moved into the catch block, the global
frame stack was not cleaned up - and contains an invalid pointer after
encountering an interpreter exception.
This small commit pops the stack and removes the framepointer that was
pushed onto it before re-throwing the exception.
I also think that this change (re-throwing the exception) breaks the
Zeek debugger in significant ways (you cannot debug a script that causes
a Runtime error anymore).
3.2.0-dev.157 | 2020-02-26 10:04:32 -0800
* IntrusivePtr: overload std::swap() (Max Kellermann)
* IntrusivePtr: eliminate setPtr() (Max Kellermann)
There are only two call sites, and those hard-code the `add_ref`
parameter.
* IntrusivePtr: optimize release() using std::exchange() (Max Kellermann)
* Expr: remove unused default constructors and `friend` declarations (Max Kellermann)
* Remove useless override: RecordAssignExpr::Eval() (Max Kellermann)
* Type: don't pass reference to pointer to MatchesIndex() (Max Kellermann)
No implementation modifies the pointer value. To guard against this,
this commit changes `&` to `const`.
* Expr: don't pass reference to pointer to check_and_promote*() (Max Kellermann)
The function never modifies the pointer value.
check_and_promote_expr() is left untouched because it really does
modify the pointer.
* ID: remove unused default constructor (Max Kellermann)
* Scope: simplify PList access, use pop_back() and back() (Max Kellermann)
* List: add method empty() (Max Kellermann)
3.2.0-dev.146 | 2020-02-26 09:40:18 -0800
* Type: fix double free bug in SetType::ShallowClone() (Max Kellermann)
3.2.0-dev.144 | 2020-02-25 19:52:57 -0800
* Add a faster siphash24 implementation than the reference one (Tim Wojtulewicz, Corelight)
- Average of 10 runs of 2009-M57-day11-18.trace (release build at -O3):
- Master: 6.027s 93650 bytes max RSS
- Commit: 5.950s 93271 bytes max RSS
- Hashing a fixed 32-byte payload 10 million times with a fixed key:
- Master: 1.397411s
- Commit: 0.998211s
3.2.0-dev.142 | 2020-02-25 19:27:28 -0800
* Updates to Broker to build CAF as sub-project (Dominik Charousset, Corelight)
The embedded CAF now builds directly as a CMake subdirectory rather
than an external project.
3.2.0-dev.137 | 2020-02-24 20:41:43 -0800
* parse.y: fix memory leak in FieldAssignExpr call (Max Kellermann)
* parse.y: fix use-after-free bug in open-ended index_slice (Max Kellermann)
* Type: fix use-after-free bug in init_type() (Max Kellermann)
* Expr: fix potential memory leak in RecordCoerceExpr::Fold() (Max Kellermann)
* Expr: fix memory leak in RecordCoerceExpr::InitVal() (Max Kellermann)
* zeekygen/IdentifierInfo: fix memory leak in operator=() (Max Kellermann)
* Func: fix memory leaks in get_func_priority() (Max Kellermann)
* parse.y: fix several memory leaks after lookup_ID() (Max Kellermann)
* Func: fix memory leaks in check_built_in_call() (Max Kellermann)
* Var: fix memory leaks in add_global() and add_local() (Max Kellermann)
* Var: add missing references to `init` in add{,_and_assign}_local() (Max Kellermann)
* parse.y: hold reference on init_expr for zeekygen::Manager::Redef() (Max Kellermann)
* Expr: fix two memory leaks in AssignExpr::InitVal() (Max Kellermann)
* parse.y: fix memory leak after "&deprecated" without string (Max Kellermann)
* RuleMatcher: delete PatternSet instances in destructor (Max Kellermann)
* Fix reference counting in Option::set_change_handler() (Max Kellermann)
3.2.0-dev.120 | 2020-02-24 18:13:04 -0800
* Update zeek-testing commit (Jon Siwek, Corelight)
3.2.0-dev.119 | 2020-02-24 18:02:10 -0800
* Use IntrusivePtr in TableVal::ExpandAndInit() to clarify ownership (Max Kellermann)
3.2.0-dev.117 | 2020-02-24 18:53:03 -0700
* Switch to using [[deprecated]] attribute (Jon Siwek, Corelight)
3.2.0-dev.115 | 2020-02-24 18:48:50 -0700
* Improve formatting of doubles that are close to integers
Now checks for approximate floating point equality so that more doubles
get properly disambiguated from integers (Jon Siwek, Corelight)
* Improve HTTP version number comparisons
Previous use of floating point comparisons was not always stable. (Jon Siwek, Corelight)
* Add a 32-bit task to Cirrus CI config (Jon Siwek, Corelight)
* Replace va_list fmt() overload with vfmt()
Using an overload that takes a va_list argument potentially causes
accidental misuse on platforms (e.g. 32-bit) where va_list is
implemented as a type that may collide with commonly-used argument
types.
For example:
char* c = copy_string("hi");
fmt("%s", (const char*)c);
fmt("%s", c);
The first fmt() call correctly goes through fmt(const char*, ...) first,
but the second mistakenly goes through fmt(const char*, va_list) first
because variadic function overloads have lower priority during overload
resolution and va_list on a 32-bit system happens to be defined as a
pointer type that can match with "char*" but not "const char*". (Jon Siwek, Corelight)
* Format tables indexed by patterns consistently across 32-bit/64-bit
Uses a full 64 bit integer for length values regardless of actual size_t
to get consistent results between either 32-bit and 64-bit platforms. (Jon Siwek, Corelight)
* Format interval values consistently across 32-bit/64-bit platforms (Jon Siwek, Corelight)
3.2.0-dev.108 | 2020-02-24 17:24:07 -0800
* Change OpaqueVal/HashVal APIs to use IntrusivePtr (Max Kellermann)
3.2.0-dev.106 | 2020-02-24 17:07:10 -0800
* ScannedFile: Identify already scanned files by device and inode (Arne Welzel)
Zeek scripts located on separate filesystems, but sharing the same inode
number leads to scripts not being loaded. The reason is that a `ScannedFile`
is only identified by `st_ino` which is not enough to uniquely identify a
file in a system.
* GH-808: Add ZEEK_VERSION_NUMBER definition to zeek-config.h (Jon Siwek, Corelight)
This is the result of (major * 10000 + minor * 100 + patch), for example
3.1.2 becomes 30102. This definition may be helpful for external code
that requires conditional compilation to support multiple Zeek
versions with differing APIs.
* Fix CI to checkout right commit of zeek-testing-private (Jon Siwek, Corelight)
3.2.0-dev.99 | 2020-02-21 21:23:52 -0800
* Update a URL in CI README (Jon Siwek, Corelight)
3.2.0-dev.98 | 2020-02-21 20:04:30 -0800
* Val: use C++ initializers (Max Kellermann)
* Val: add BroValUnion constructors (Max Kellermann)
* Val: reduce duplicate code by using delegating constructors (Max Kellermann)
* Val: remove unused default constructors and `friend` declarations (Max Kellermann)
* Val: remove the unnecessary BroValUnion typedef (Max Kellermann)
* Type: remove unnecessary enum typedefs (Max Kellermann)
* Type: use C++ initializers (Max Kellermann)
* Type: move code from BroType::BroType() to constexpr functions (Max Kellermann)
Prepare to inline the constructor, which will one day be `constexpr`
(requires moving the `std::string name` field somewhere else).
* Type: remove useless BroType destructor (Max Kellermann)
* Obj: disallow copying BroObj (Max Kellermann)
Copying a BroObj is dangerous, and should only be done with dedicated
(virtual) methods which are implemented by all derived classes. This
commit avoids unintentional copies.
* Obj: use C++ initializers (Max Kellermann)
* Obj: make `no_location` constexpr (Max Kellermann)
This ensures that the variable is initialized at compile time and may
allow the compiler to apply more initializations.
3.2.0-dev.85 | 2020-02-21 15:29:45 -0800
* threading/MsgThread: add [[noreturn]] to InternalError() (Max Kellermann)
* Flare: add [[noreturn]] to bad_pipe_op() (Max Kellermann)
* Obj: add [[noreturn]] attributes to Internal(), bad_ref() (Max Kellermann)
* Reporter: add [[noreturn]] attribute to several methods (Max Kellermann)
3.2.0-dev.80 | 2020-02-21 10:14:05 -0800
* OpaqueVal: remove misplaced `virtual` keywords (Max Kellermann)
These methods are not meant to be overridden.
* CompHash: use class IntrusivePtr for the `type` field (Max Kellermann)
* IntrusivePtr: replace the "add_ref" parameter with tag structs (Max Kellermann)
Using a runtime parameter is obscure and error-prone. Avoiding
error-prone code and getting reference counting right is the whole
point of this class.
* IntrusivePtr: remove reset(), nobody uses it (Max Kellermann)