Add numpy 2 support

[jparismorgan]

What

Updates to support NumPy 2. NumPy 2.0 as a major release changes the C ABI, so any package that builds against the NumPy C API like shapely will have to be rebuilt with numpy 2.0 to be able to run with numpy 2.0.

Specifically:

• We require that packages are built with numpy>=2.0.0
• We require that at runtime numpy>=1.25.0 is present

See:

• https://numpy.org/doc/stable/dev/depending_on_numpy.html#numpy-2-0-specific-advice
• https://github.com/shapely/shapely/pull/2033/files#diff-50c86b7ed8ac2cf95bd48334961bf0530cdc77b5a56f852c5c61b89d735fd711R9

Testing

• Existing tests pass.

TODO

In a future PR we could follow this advice and test against the earliest numpy version which we support:

Opened SC-50896 to track this.

[jparismorgan]

We make this change b/c of changes to numpy data type promotion.

With numpy 1 and the original code we were getting:

[ingestion@read_source_metadata] schema.domain.dim(1).domain[1]: 2147483647 <class 'numpy.int32'>
[ingestion@read_source_metadata] size: 2147483648 <class 'numpy.int64'>
[ingestion@read_source_metadata] dimensions: 3 <class 'numpy.int64'>

With numpy 2 and the original code we instead were getting:

[ingestion@read_source_metadata] schema.domain.dim(1).domain[1]: 2147483647 <class 'numpy.int32'>
[ingestion@read_source_metadata] size: -2147483648 <class 'numpy.int32'>
[ingestion@read_source_metadata] dimensions: 3 <class 'numpy.int32'>

This is because in size = schema.domain.dim(1).domain[1] + 1 the + 1 used to cause a cast to int64 (which is required because 2147483647 is int32 max).

As mentioned in https://numpy.org/devdocs/numpy_2_0_migration_guide.html#changes-to-numpy-data-type-promotion, numpy no longer casts automatically:

So here we explicitly cast to int64 so that we return the same value as we did before.

[jparismorgan]

This change is also because numpy 2 does not do data type promotion.

With numpy 1 and the original code we were getting:

[ingestion@consolidate_partition_udf] partial_indexes [  0   0   1   1   5   6  14  14  14  18  23  28  42  44  44  44  50  55
  58  60  61  61  61  66  66  81  81  84  84  86  91  94 101 108 110 114
 118 125 126 127 127 127 135 135 141 142 143 143 149 154 157 161 178 191
 200 201 209 214 214 230 233 236 240 242 243 248 257 257 275 276 278 282
 283 290 290 291 298 316 324 324 332 335 335 343 343 347 350 353 356 373
 374 379 382 391 391 391 398 399 405 412 421] <class 'numpy.ndarray'> uint64
[ingestion@consolidate_partition_udf] prev_index 0 <class 'numpy.uint64'>
[ingestion@consolidate_partition_udf] partial_index 0 <class 'numpy.uint64'>
[ingestion@consolidate_partition_udf] s slice(0, -1, None) <class 'slice'>

With numpy 2 and the original code we were getting:

[ingestion@consolidate_partition_udf] partial_indexes [  0   0   1   1   5   6  14  14  14  18  23  28  42  44  44  44  50  55
  58  60  61  61  61  66  66  81  81  84  84  86  91  94 101 108 110 114
 118 125 126 127 127 127 135 135 141 142 143 143 149 154 157 161 178 191
 200 201 209 214 214 230 233 236 240 242 243 248 257 257 275 276 278 282
 283 290 290 291 298 316 324 324 332 335 335 343 343 347 350 353 356 373
 374 379 382 391 391 391 398 399 405 412 421] <class 'numpy.ndarray'> uint64
[ingestion@consolidate_partition_udf] prev_index 0 <class 'numpy.uint64'>
[ingestion@consolidate_partition_udf] partial_index 0 <class 'numpy.uint64'>
[ingestion@consolidate_partition_udf] s slice(0, 18446744073709551615, None) <class 'slice'>

Notice that we get slice(0, 18446744073709551615, None) instead of slice(0, -1, None). To fix this we can cast before subtracting, which we do here.

[ihnorton]

Thanks!