Failing to fit DecisionTree

[kurtlawrence]

Trying to fit a DecisionTree is failing because the comparator doesn't implement total order.
I believe this is due to using Rust > 1.81 with this change to the sort function.

I am fitting a decision tree with f64 records and bool targets.

thread 'main' panicked at core/src/slice/sort/shared/smallsort.rs:860:5:
user-provided comparison function does not correctly implement a total order
stack backtrace:
   0: rust_begin_unwind
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/std/src/panicking.rs:662:5
   1: core::panicking::panic_fmt
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/panicking.rs:74:14
   2: core::slice::sort::shared::smallsort::panic_on_ord_violation
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/slice/sort/shared/smallsort.rs:860:5
   3: core::slice::sort::shared::smallsort::bidirectional_merge
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/slice/sort/shared/smallsort.rs:838:13
   4: core::slice::sort::shared::smallsort::sort8_stable
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/slice/sort/shared/smallsort.rs:678:9
   5: core::slice::sort::shared::smallsort::small_sort_general_with_scratch
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/slice/sort/shared/smallsort.rs:243:13
   6: <T as core::slice::sort::shared::smallsort::StableSmallSortTypeImpl>::small_sort
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/slice/sort/shared/smallsort.rs:61:9
   7: core::slice::sort::stable::quicksort::quicksort
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/slice/sort/stable/quicksort.rs:25:13
   8: core::slice::sort::stable::drift::create_run
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/slice/sort/stable/drift.rs:255:9
   9: core::slice::sort::stable::drift::sort
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/slice/sort/stable/drift.rs:64:17
  10: core::slice::sort::stable::driftsort_main
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/slice/sort/stable/mod.rs:84:5
  11: core::slice::sort::stable::sort
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/slice/sort/stable/mod.rs:45:5
  12: alloc::slice::stable_sort
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/alloc/src/slice.rs:882:5
  13: alloc::slice::<impl [T]>::sort_by
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/alloc/src/slice.rs:288:9
  14: linfa_trees::decision_trees::algorithm::SortedIndex<F>::of_array_column
             at /home/kurt/.cargo/registry/src/index.crates.io-6f17d22bba15001f/linfa-trees-0.7.0/src/decision_trees/algorithm.rs:103:9
  15: linfa_trees::decision_trees::algorithm::<impl linfa::traits::Fit<ndarray::ArrayBase<D,ndarray::dimension::dim::Dim<[usize; 2]>>,T,linfa::error::Error> for linfa_trees::decision_trees::hyperparams::DecisionTreeValidParams<F,L>>::fit::{{closure}}
             at /home/kurt/.cargo/registry/src/index.crates.io-6f17d22bba15001f/linfa-trees-0.7.0/src/decision_trees/algorithm.rs:530:17
  16: core::iter::adapters::map::map_fold::{{closure}}
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/iter/adapters/map.rs:88:28
  17: core::iter::traits::iterator::Iterator::fold
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/iter/traits/iterator.rs:2583:21
  18: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::fold
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/iter/adapters/map.rs:128:9
  19: core::iter::traits::iterator::Iterator::for_each
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/iter/traits/iterator.rs:813:9
  20: alloc::vec::Vec<T,A>::extend_trusted
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/alloc/src/vec/mod.rs:3121:17
  21: <alloc::vec::Vec<T,A> as alloc::vec::spec_extend::SpecExtend<T,I>>::spec_extend
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/alloc/src/vec/spec_extend.rs:26:9
  22: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter_nested::SpecFromIterNested<T,I>>::from_iter
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/alloc/src/vec/spec_from_iter_nested.rs:60:9
  23: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/alloc/src/vec/spec_from_iter.rs:33:9
  24: <alloc::vec::Vec<T> as core::iter::traits::collect::FromIterator<T>>::from_iter
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/alloc/src/vec/mod.rs:2985:9
  25: core::iter::traits::iterator::Iterator::collect
             at /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/core/src/iter/traits/iterator.rs:2000:9
  26: linfa_trees::decision_trees::algorithm::<impl linfa::traits::Fit<ndarray::ArrayBase<D,ndarray::dimension::dim::Dim<[usize; 2]>>,T,linfa::error::Error> for linfa_trees::decision_trees::hyperparams::DecisionTreeValidParams<F,L>>::fit
             at /home/kurt/.cargo/registry/src/index.crates.io-6f17d22bba15001f/linfa-trees-0.7.0/src/decision_trees/algorithm.rs:528:38
  27: linfa::param_guard::<impl linfa::traits::Fit<R,T,E> for P>::fit
             at /home/kurt/.cargo/registry/src/index.crates.io-6f17d22bba15001f/linfa-0.7.0/src/param_guard.rs:64:9

[coolstudio1678]

I am fitting a decision tree with f64 records and str targets, rust version is 1.82, The program runs correctly.

[kurtlawrence]

I wonder if the error occurs if there are Nan's in the records

[coolstudio1678]

My data is stored in polars data. When converted to ndarray, if there is a nan value, an error is reported directly. Therefore, there is no nan value in the tree.