bugfix: Treemap.andnot would not include items with no container on rhs

[Dr-Emann]

And add a test which validates the expected behavior. This also found a doc test which was accidentally incorrect.

[saulius]

Weird CI error. Any thoughts @Dr-Emann? I do not have access to a Windows machine to verify this.

[Dr-Emann]

Hmm. It's not reproducing on my windows machine..

[saulius]

I tempted to call this an intermittent issue with Github runners. I think we're good to go 👍

[Dr-Emann]

@lemire any thoughts on this, windows is failing seemingly consistently (this PR, several builds of #125) failing in CI with an illegal instruction on windows, running roaring_bitmap_or_many against two very simple bitmaps:

let bitmap1 = Bitmap::from([500, 1000]);
let bitmap2 = Bitmap::from([1000, 2000]);

It doesn't reproduce on my personal windows machine, not sure where to look here. Only thought is some simd thing, could we be somehow getting SIMD instructions the cpu doesn't actually support? The build and run take place on the same machine..

[lemire]

@Dr-Emann

1. There was a bug in earlier CRoaring versions where it would detect the ISA based on the CPU, ignoring OS overrides, but you are using a recent version that has been patched. It is not impossible that I got it wrong, but I have similar code in other projects that are part of Node.js which is in broad use, so I expect it to be correct.
2. One way in which the Visual Studio builds differ is with respect to thread safety. Visual Studio does not yet (at least last time I checked) supports standard atomics. We try our best, but I expect that data races under Visual Studio are possible. So if your tests are multithreaded, you could have a data race. Looking quickly at croaring-rs, it does not look like a data race is likely.
3. An obvious question is: can we reproduce it in CRoaring itself (taking rust out of the equation)?
4. When did the issue begin?

The build and run take place on the same machine..

That's not necessarily relevant because we use runtime dispatching. So we will build AVX-512 support even if the local machine does not support AVX-512. Same with AVX. To make things more complicated, even if your CPU supports AVX-512 or AVX, the OS can disable support.

To help debugging, I recommend exposing croaring_hardware_support(). It returns an integer. The first two bits tell you what is detected:

enum {
  ROARING_SUPPORTS_AVX2 = 1,
  ROARING_SUPPORTS_AVX512 = 2,
};

So 3 would indicate AVX-512 and AVX2. Printing this out could be useful information (whether you need it for this issue or not).

You could also try to build with #define CROARING_COMPILER_SUPPORTS_AVX512 0, which would disable forcefully AVX-512. If it solves your problems, then you know that the trouble is with AVX-512. Disabling AVX-512 under Windows would be entirely acceptable.

[lemire]

An obvious issue is that this should cause problems with CRoaring itself. It seems that it does not (our CI is green). Why would Rust cause a different behaviour under Windows? It should not matter, right? I expect that you are building the C code, and then the Rust code just interfaces with it. So what is up? Intriguing.

[Dr-Emann]

In case anyone else is following, see answers/further investegation in #128, and I believe RoaringBitmap/CRoaring#573