Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error executing Benchmarch #13

Open
dbanshee opened this issue Jan 20, 2020 · 2 comments
Open

Error executing Benchmarch #13

dbanshee opened this issue Jan 20, 2020 · 2 comments

Comments

@dbanshee
Copy link 

root@547a227b1517:~/numba-examples# numba_bench -o results -r gpu
Scanning /root/numba-examples for benchmarks
Writing results to /root/numba-examples/results
/usr/local/lib/python3.6/dist-packages/numba_bench-0.1-py3.6.egg/numba_bench/benchmark.py:54: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  config = yaml.load(f)

  Running Histogram [/root/numba-examples/examples/density_estimation/histogram]
    numpy: bins10, float32 - 1000 => 3 reps, 1000 iter per rep, 169.493665 usec per call
    numba: bins10, float32 - 1000 => 3 reps, 10000 iter per rep, 15.088746 usec per call
    numba_gpu: bins10, float32 - 1000 => 3 reps, 100 iter per rep, 1593.281750 usec per call
    numpy: bins10, float32 - 10000 => 3 reps, 1000 iter per rep, 265.618970 usec per call
    numba: bins10, float32 - 10000 => 3 reps, 1000 iter per rep, 141.332854 usec per call
    numba_gpu: bins10, float32 - 10000 => 3 reps, 100 iter per rep, 1603.700330 usec per call
    numpy: bins10, float32 - 100000 => 3 reps, 100 iter per rep, 1179.389010 usec per call
    numba: bins10, float32 - 100000 => 3 reps, 100 iter per rep, 1402.002540 usec per call
    numba_gpu: bins10, float32 - 100000 => 3 reps, 100 iter per rep, 1681.287160 usec per call
    numpy: bins10, float32 - 300000 => 3 reps, 100 iter per rep, 3271.321700 usec per call
    numba: bins10, float32 - 300000 => 3 reps, 100 iter per rep, 4213.909610 usec per call
    numba_gpu: bins10, float32 - 300000 => 3 reps, 100 iter per rep, 1949.394360 usec per call
    numpy: bins10, float32 - 3000000 => 3 reps, 10 iter per rep, 32000.128600 usec per call
    numba: bins10, float32 - 3000000 => 3 reps, 10 iter per rep, 42029.814400 usec per call
    numba_gpu: bins10, float32 - 3000000 => 3 reps, 100 iter per rep, 4975.962050 usec per call
    numpy: bins10, float64 - 1000 => 3 reps, 1000 iter per rep, 160.448296 usec per call
    numba: bins10, float64 - 1000 => 3 reps, 10000 iter per rep, 14.967072 usec per call
    numba_gpu: bins10, float64 - 1000 => 3 reps, 100 iter per rep, 1591.028610 usec per call
    numpy: bins10, float64 - 10000 => 3 reps, 1000 iter per rep, 273.850549 usec per call
    numba: bins10, float64 - 10000 => 3 reps, 1000 iter per rep, 137.559821 usec per call
    numba_gpu: bins10, float64 - 10000 => 3 reps, 100 iter per rep, 1585.167370 usec per call
    numpy: bins10, float64 - 100000 => 3 reps, 100 iter per rep, 1402.316260 usec per call
    numba: bins10, float64 - 100000 => 3 reps, 100 iter per rep, 1365.159980 usec per call
    numba_gpu: bins10, float64 - 100000 => 3 reps, 100 iter per rep, 1778.616570 usec per call
    numpy: bins10, float64 - 300000 => 3 reps, 100 iter per rep, 4086.320090 usec per call
    numba: bins10, float64 - 300000 => 3 reps, 100 iter per rep, 4103.344970 usec per call
    numba_gpu: bins10, float64 - 300000 => 3 reps, 100 iter per rep, 2086.206040 usec per call
    numpy: bins10, float64 - 3000000 => 3 reps, 10 iter per rep, 37877.584500 usec per call
    numba: bins10, float64 - 3000000 => 3 reps, 10 iter per rep, 40958.335600 usec per call
    numba_gpu: bins10, float64 - 3000000 => 3 reps, 100 iter per rep, 6885.126960 usec per call
    numpy: bins1000, float32 - 1000 => 3 reps, 1000 iter per rep, 180.907137 usec per call
    numba: bins1000, float32 - 1000 => 3 reps, 10000 iter per rep, 16.114160 usec per call
    numba_gpu: bins1000, float32 - 1000 => 3 reps, 100 iter per rep, 1586.353680 usec per call
    numpy: bins1000, float32 - 10000 => 3 reps, 1000 iter per rep, 275.862535 usec per call
    numba: bins1000, float32 - 10000 => 3 reps, 1000 iter per rep, 142.604908 usec per call
    numba_gpu: bins1000, float32 - 10000 => 3 reps, 100 iter per rep, 1589.610960 usec per call
    numpy: bins1000, float32 - 100000 => 3 reps, 100 iter per rep, 1223.783610 usec per call
    numba: bins1000, float32 - 100000 => 3 reps, 100 iter per rep, 1404.859190 usec per call
    numba_gpu: bins1000, float32 - 100000 => 3 reps, 100 iter per rep, 1684.227960 usec per call
    numpy: bins1000, float32 - 300000 => 3 reps, 100 iter per rep, 3347.087800 usec per call
    numba: bins1000, float32 - 300000Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/numba_bench-0.1-py3.6.egg/numba_bench/benchmark.py", line 184, in _run_and_validate_results
    self.validator(input_args, input_kwargs, actual_results)
  File "/root/numba-examples/examples/density_estimation/histogram/impl.py", line 73, in validator
    np.testing.assert_array_equal(expected_hist, actual_hist)
  File "/usr/local/lib/python3.6/dist-packages/numpy/testing/_private/utils.py", line 918, in assert_array_equal
    verbose=verbose, header='Arrays are not equal')
  File "/usr/local/lib/python3.6/dist-packages/numpy/testing/_private/utils.py", line 841, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

Mismatch: 0.4%
Max absolute difference: 1
Max relative difference: 0.00301205
 x: array([   1,    0,    0,    0,    0,    0,    0,    0,    0,    1,    0,
          0,    0,    1,    0,    0,    0,    0,    0,    0,    1,    0,
          1,    0,    0,    0,    0,    0,    0,    0,    0,    0,    0,...
 y: array([   1,    0,    0,    0,    0,    0,    0,    0,    0,    1,    0,
          0,    0,    1,    0,    0,    0,    0,    0,    0,    1,    0,
          1,    0,    0,    0,    0,    0,    0,    0,    0,    0,    0,...

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/numba_bench", line 4, in <module>
    __import__('pkg_resources').run_script('numba-bench==0.1', 'numba_bench')
  File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 666, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 1462, in run_script
    exec(code, namespace, namespace)
  File "/usr/local/lib/python3.6/dist-packages/numba_bench-0.1-py3.6.egg/EGG-INFO/scripts/numba_bench", line 7, in <module>
    sys.exit(main(sys.argv))
  File "/usr/local/lib/python3.6/dist-packages/numba_bench-0.1-py3.6.egg/numba_bench/main.py", line 62, in main
    verify_only=args.verify_only)
  File "/usr/local/lib/python3.6/dist-packages/numba_bench-0.1-py3.6.egg/numba_bench/benchmark.py", line 290, in discover_and_run_benchmarks
    results = benchmark.run_benchmark(verify_only=verify_only)
  File "/usr/local/lib/python3.6/dist-packages/numba_bench-0.1-py3.6.egg/numba_bench/benchmark.py", line 229, in run_benchmark
    self._run_and_validate_results(input_dict, impl_dict)
  File "/usr/local/lib/python3.6/dist-packages/numba_bench-0.1-py3.6.egg/numba_bench/benchmark.py", line 186, in _run_and_validate_results
    self._raise_benchmark_error('Implementation %s failed validation on input %s' % (impl_dict['name'], input_dict['x']))
  File "/usr/local/lib/python3.6/dist-packages/numba_bench-0.1-py3.6.egg/numba_bench/benchmark.py", line 59, in _raise_benchmark_error
    raise BenchmarkError(self.benchmark_dir, message)
numba_bench.benchmark.BenchmarkError: [/root/numba-examples/examples/density_estimation/histogram]: Implementation numba failed validation on input 300000

Runing on:

root@547a227b1517:~/numba-examples# numba -s
System info:
--------------------------------------------------------------------------------
__Time Stamp__
2020-01-20 18:47:17.782535

__Hardware Information__
Machine                                       : x86_64
CPU Name                                      : ivybridge
Number of accessible CPU cores                : 4
Listed accessible CPUs cores                  : 0-3
CFS restrictions                              : None
CPU Features                                  : 
64bit aes avx cmov cx16 f16c fsgsbase mmx pclmul popcnt rdrnd sahf sse sse2 sse3
sse4.1 sse4.2 ssse3 xsave xsaveopt

__OS Information__
Platform                                      : Linux-5.0.0-38-generic-x86_64-with-Ubuntu-18.04-bionic
Release                                       : 5.0.0-38-generic
System Name                                   : Linux
Version                                       : #41-Ubuntu SMP Tue Dec 3 00:27:35 UTC 2019
OS specific info                              : Ubuntu18.04bionic
glibc info                                    : glibc 2.25

__Python Information__
Python Compiler                               : GCC 8.3.0
Python Implementation                         : CPython
Python Version                                : 3.6.8
Python Locale                                 : en_US UTF-8

__LLVM information__
LLVM version                                  : 8.0.0

__CUDA Information__
Found 1 CUDA devices
id 0      b'GeForce GTX 760'                              [SUPPORTED]
                      compute capability: 3.0
                           pci device id: 0
                              pci bus id: 1
Summary:
        1/1 devices are supported
CUDA driver version                           : 10010
CUDA libraries:
Finding cublas from System
        named  libcublas.so.10.0.130
        trying to open library...       ok
Finding cusparse from System
        named  libcusparse.so.10.0.130
        trying to open library...       ok
Finding cufft from System
        named  libcufft.so.10.0.145
        trying to open library...       ok
Finding curand from System
        named  libcurand.so.10.0.130
        trying to open library...       ok
Finding nvvm from System
        named  libnvvm.so.3.3.0
        trying to open library...       ok
Finding libdevice from System
        searching for compute_20...     ok
        searching for compute_30...     ok
        searching for compute_35...     ok
        searching for compute_50...     ok

__ROC Information__
ROC available                                 : False
Error initialising ROC due to                 : No ROC toolchains found.
No HSA Agents found, encountered exception when searching:
Error at driver init: 
NUMBA_HSA_DRIVER /opt/rocm/lib/libhsa-runtime64.so is not a valid file path.  Note it must be a filepath of the .so/.dll/.dylib or the driver:

__SVML Information__
SVML state, config.USING_SVML                 : False
SVML library found and loaded                 : False
llvmlite using SVML patched LLVM              : True
SVML operational                              : False

__Threading Layer Information__
TBB Threading layer available                 : True
OpenMP Threading layer available              : False
+--> Disabled due to                          : Unknown import problem.
Workqueue Threading layer available           : True

__Numba Environment Variable Information__
None set.

__Conda Information__
Conda not present/not working.
Error was [Errno 2] No such file or directory: 'conda': 'conda'

--------------------------------------------------------------------------------
If requested, please copy and paste the information between
the dashed (----) lines, or from a given specific section as
appropriate.

=============================================================
IMPORTANT: Please ensure that you are happy with sharing the
contents of the information present, any information that you
wish to keep private you should remove before sharing.
=============================================================
@abelsiqueira
Copy link

abelsiqueira commented Jun 28, 2022 •
edited
Loading

I have a similar error using numba_bench -o results histogram, but with 100000 instead of 300000:

Implementation numba failed validation on input 100000

@animator
Copy link

animator commented Aug 6, 2022 •
edited
Loading

Looks like the issue is with density_estimation/histogram example. Other benchmarks ran fine.

  Running Histogram [/Documents/GitHub/numba-examples/examples/density_estimation/histogram]
    numpy: bins10, float32 - 1000 => 3 reps, 10000 iter per rep, 88.135271 usec per call
    numba: bins10, float32 - 1000 => 3 reps, 100000 iter per rep, 4.443364 usec per call
    numpy: bins10, float32 - 10000 => 3 reps, 1000 iter per rep, 144.338375 usec per call
    numba: bins10, float32 - 10000 => 3 reps, 10000 iter per rep, 29.171375 usec per call
    numpy: bins10, float32 - 100000 => 3 reps, 1000 iter per rep, 687.286959 usec per call
    numba: bins10, float32 - 100000 => 3 reps, 1000 iter per rep, 279.511208 usec per call
    numpy: bins10, float32 - 300000 => 3 reps, 100 iter per rep, 1895.402500 usec per call
    numba: bins10, float32 - 300000 => 3 reps, 1000 iter per rep, 843.883041 usec per call
    numpy: bins10, float32 - 3000000 => 3 reps, 10 iter per rep, 18824.854100 usec per call
    numba: bins10, float32 - 3000000 => 3 reps, 100 iter per rep, 8644.399160 usec per call
    numpy: bins10, float64 - 1000 => 3 reps, 10000 iter per rep, 85.318975 usec per call
    numba: bins10, float64 - 1000 => 3 reps, 100000 iter per rep, 4.416603 usec per call
    numpy: bins10, float64 - 10000 => 3 reps, 1000 iter per rep, 148.214792 usec per call
    numba: bins10, float64 - 10000 => 3 reps, 10000 iter per rep, 29.376937 usec per call
    numpy: bins10, float64 - 100000 => 3 reps, 1000 iter per rep, 768.946458 usec per call
    numba: bins10, float64 - 100000 => 3 reps, 1000 iter per rep, 283.016125 usec per call
    numpy: bins10, float64 - 300000 => 3 reps, 100 iter per rep, 2161.592920 usec per call
    numba: bins10, float64 - 300000 => 3 reps, 1000 iter per rep, 853.712167 usec per call
    numpy: bins10, float64 - 3000000 => 3 reps, 10 iter per rep, 21563.341600 usec per call
    numba: bins10, float64 - 3000000 => 3 reps, 100 iter per rep, 8740.612910 usec per call
    numpy: bins1000, float32 - 1000 => 3 reps, 10000 iter per rep, 92.946225 usec per call
    numba: bins1000, float32 - 1000 => 3 reps, 100000 iter per rep, 4.994264 usec per call
    numpy: bins1000, float32 - 10000 => 3 reps, 1000 iter per rep, 139.874917 usec per call
    numba: bins1000, float32 - 10000 => 3 reps, 10000 iter per rep, 28.367758 usec per call
    numpy: bins1000, float32 - 100000 => 3 reps, 1000 iter per rep, 634.296291 usec per call
    numba: bins1000, float32 - 100000Traceback (most recent call last):
  File "/Documents/GitHub/numba-examples/numba_bench/benchmark.py", line 184, in _run_and_validate_results
    self.validator(input_args, input_kwargs, actual_results)
  File "/Documents/GitHub/numba-examples/examples/density_estimation/histogram/impl.py", line 73, in validator
    np.testing.assert_array_equal(expected_hist, actual_hist)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/numpy/testing/_private/utils.py", line 934, in assert_array_equal
    assert_array_compare(operator.__eq__, x, y, err_msg=err_msg,
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/numpy/testing/_private/utils.py", line 844, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Arrays are not equal

Mismatched elements: 2 / 1000 (0.2%)
Max absolute difference: 1
Max relative difference: 0.00595238
 x: array([  1,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   1,   0,   0,   0,   0,   0,   0,   0,   0,...
 y: array([  1,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
         0,   0,   0,   0,   1,   0,   0,   0,   0,   0,   0,   0,   0,...

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/bin/numba_bench", line 7, in <module>
    exec(compile(f.read(), __file__, 'exec'))
  File "/Documents/GitHub/numba-examples/bin/numba_bench", line 7, in <module>
    sys.exit(main(sys.argv))
  File "/Documents/GitHub/numba-examples/numba_bench/main.py", line 61, in main
    discover_and_run_benchmarks(root, output, match_substrings, skip_existing=args.skip_existing, resources=resources,
  File "/Documents/GitHub/numba-examples/numba_bench/benchmark.py", line 290, in discover_and_run_benchmarks
    results = benchmark.run_benchmark(verify_only=verify_only)
  File "/Documents/GitHub/numba-examples/numba_bench/benchmark.py", line 229, in run_benchmark
    self._run_and_validate_results(input_dict, impl_dict)
  File "/Documents/GitHub/numba-examples/numba_bench/benchmark.py", line 186, in _run_and_validate_results
    self._raise_benchmark_error('Implementation %s failed validation on input %s' % (impl_dict['name'], input_dict['x']))
  File "/Documents/GitHub/numba-examples/numba_bench/benchmark.py", line 59, in _raise_benchmark_error
    raise BenchmarkError(self.benchmark_dir, message)
numba_bench.benchmark.BenchmarkError: [/Documents/GitHub/numba-examples/examples/density_estimation/histogram]: Implementation numba failed validation on input 100000

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@animator @abelsiqueira @dbanshee