Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Beginnings of the oneAPI backend #955

Merged
merged 117 commits into from
Oct 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
117 commits
Select commit Hold shift + click to select a range
ce287f0
snapshot adding oneapi
jmitrevs Dec 21, 2023
cd0a2b8
fix reduce constexpr
jmitrevs Dec 21, 2023
3b3d40d
further updates
jmitrevs Dec 23, 2023
b742901
update the bridge and testbench
jmitrevs Dec 26, 2023
8f6ef78
fix issues discovered when compiling
jmitrevs Dec 28, 2023
2e56be4
update bridge writing files
jmitrevs Jan 8, 2024
db780f0
Merge remote-tracking branch 'upstream/main' into oneapi_backend
jmitrevs Jan 8, 2024
b90021f
build library (but not tested)
jmitrevs Jan 9, 2024
f086aa2
fix a bug in testbench
jmitrevs Jan 10, 2024
1f28cbf
snapshot after some debugging
jmitrevs Jan 11, 2024
3e69b9a
remove forgotten debug printing
jmitrevs Jan 11, 2024
17e6856
add build
jmitrevs Jan 11, 2024
2766a6e
pre-commit fixes
jmitrevs Jan 12, 2024
c4ce138
fix more pre-commit
jmitrevs Jan 12, 2024
354d708
fix more pre-commit errors
jmitrevs Jan 12, 2024
8119029
snapshot of work before reworking types
jmitrevs Jan 21, 2024
cae1a8a
Use using to decide array type, some preliminary updates
jmitrevs Feb 12, 2024
06a8c27
snapshot unifying types
jmitrevs Feb 14, 2024
8f58778
fix the testbench and bridge
jmitrevs Feb 14, 2024
86b0f4b
snapshot updating nnet_utils (not finished)
jmitrevs Feb 14, 2024
62c5ecb
define array in nnet_types for oneAPI
jmitrevs Feb 14, 2024
d203b42
fix parallel conv2d
jmitrevs Feb 14, 2024
f983ece
add back the streaming versions of algs, most unconverted
jmitrevs Feb 14, 2024
5dd9282
tentatively complete streaming for dense but not functional
jmitrevs Feb 15, 2024
09b9513
first version that compiles streaming
jmitrevs Feb 15, 2024
0e3f9ba
change how the pipe value type is extracted
jmitrevs Feb 16, 2024
e9f49ad
Merge remote-tracking branch 'upstream/main' into oneapi_backend
jmitrevs Feb 16, 2024
99038eb
fix pre-commit error
jmitrevs Feb 16, 2024
3d555ac
always treat elu as ELU class
jmitrevs Feb 26, 2024
68c6a51
fix batchnorm
jmitrevs Feb 26, 2024
a3f5b3c
snapshot towards fixing conv
jmitrevs Feb 27, 2024
0cbf5be
snapshot fixing test for streaming
jmitrevs Feb 28, 2024
75c9301
fix conv1d
jmitrevs Feb 28, 2024
ba2e283
fix conv2d
jmitrevs Feb 28, 2024
a7c08d3
fix reshape and flatten for oneAPI
jmitrevs Feb 29, 2024
fa05c8b
initial oneAPI tests
jmitrevs Feb 29, 2024
36d5c85
remove nnet_dense_compressed from oneAPI
jmitrevs Mar 14, 2024
de8b76d
add merge functionality (untested)
jmitrevs Mar 16, 2024
058adb4
fix merge for oneAPI
jmitrevs Mar 16, 2024
0a7c761
fix merge for oneAPI (missing commit)
jmitrevs Mar 16, 2024
4c847b2
add zeropadding
jmitrevs Mar 17, 2024
f690c98
standardize paralellization spelling
jmitrevs Mar 17, 2024
262bc0c
fix pointwise for oneAPI
jmitrevs Apr 15, 2024
a8da30e
remove references to quartus
jmitrevs Apr 15, 2024
46ccc1d
more replace quartus with oneapi
jmitrevs Apr 15, 2024
8c9313b
snapshot on the way towards implementing pooling
jmitrevs Apr 15, 2024
0498d44
fix io_stream pooling for oneAPI
jmitrevs Apr 16, 2024
7bd7ba5
add fix for Conv2DBatchnorm
jmitrevs Apr 16, 2024
4ff035f
accidentally committed CMakeLists.txt in my debug setup
jmitrevs Apr 16, 2024
b754f76
reshaping, not fully tested
jmitrevs Apr 17, 2024
2e5a05e
fix cloning of streams
jmitrevs Apr 18, 2024
77c5672
Merge remote-tracking branch 'upstream/main' into oneapi_backend
jmitrevs Apr 19, 2024
8470a6c
fix pytest library loading
jmitrevs Apr 22, 2024
20128bb
remove unused template
jmitrevs Apr 22, 2024
efb6a7a
fix some activation bugs
jmitrevs Apr 22, 2024
6f439d5
fix the overwriting of directories in the pytest
jmitrevs Apr 22, 2024
637e192
update version of test repository
jmitrevs Apr 23, 2024
0f12c96
try to fix docker issue
jmitrevs Apr 23, 2024
a5aac2a
bump hls4ml-testing tag to 0.5.2
jmitrevs Apr 24, 2024
412bd43
try not restricting tensorflow-model-optimizatoin
jmitrevs Apr 24, 2024
5cffadf
Update to 0.5.3 for testing
jmitrevs Apr 24, 2024
d156339
bump to docker image 0.5.4, suggested by Ben
jmitrevs Apr 25, 2024
924af07
fix pre-commit warning
jmitrevs Apr 25, 2024
7ded550
dial down N_TESTS_PER_YAML to 4
jmitrevs Apr 25, 2024
e966b18
revert tensorflow-model-optimization change
jmitrevs Apr 25, 2024
e649f34
fix issue of saving in "obsolete" h5 format
jmitrevs Apr 25, 2024
4743a5d
Merge remote-tracking branch 'upstream/main' into oneapi_backend
jmitrevs May 3, 2024
bf68958
fix embedding for oneAPI
jmitrevs May 8, 2024
d07985d
First attempt at adding RNNs to oneAPI
jmitrevs May 8, 2024
a58e4f5
fix bug in array size
jmitrevs May 9, 2024
eb9575a
fix order or indices
jmitrevs May 9, 2024
04e0fcf
Merge branch 'main' into oneapi_backend
jmitrevs May 31, 2024
b4ed5bc
make queues static in bridge
jmitrevs Jun 6, 2024
ba55211
fix logic error in repack stream
jmitrevs Jun 27, 2024
9b790c5
changing the style, but functionally identical
jmitrevs Jun 27, 2024
b4e8873
Merge remote-tracking branch 'upstream/main' into oneapi_backend
jmitrevs Jul 17, 2024
60fe56b
Merge branch 'main' into oneapi_backend
jmitrevs Jul 25, 2024
056765e
update pointwise optimizer for oneAPI
jmitrevs Jul 25, 2024
ee6817d
add oneAPI to test_multi_dense.py
jmitrevs Jul 25, 2024
5a5b015
fix updating weight types
jmitrevs Jul 29, 2024
1d72aa8
initial changes of templates, for testing
jmitrevs Jul 29, 2024
106d578
fix weight naming, product selection
jmitrevs Jul 29, 2024
80902d7
make im2col the default; fix winograd size
jmitrevs Jul 29, 2024
ea213a3
fix up streaming dense and convolution
jmitrevs Jul 30, 2024
5ba9a29
fix prelu, some batchnorm
jmitrevs Jul 30, 2024
fdd0baf
fix weight array of exponential types
jmitrevs Jul 31, 2024
3ff54a9
move ACExponentialPrecisionDefinition to oneapi_types
jmitrevs Jul 31, 2024
d6604f0
attempt to fix batchnorm and recurrent
jmitrevs Aug 3, 2024
0f74122
Merge branch 'main' into oneapi_backend
jmitrevs Aug 22, 2024
9ffd18e
fixed BatchNormalizationQuantizedTanhConfigTemplate template selection
jmitrevs Aug 22, 2024
be08ad0
fix embedding_stream
jmitrevs Aug 22, 2024
c06beda
fix lstm and simple rnn
jmitrevs Aug 22, 2024
5452fab
fix GRU
jmitrevs Aug 22, 2024
e39e867
fix winograd, and also disable it by default
jmitrevs Aug 22, 2024
cfe229f
fix threshold name
jmitrevs Aug 23, 2024
70617e1
split bn_quant to be backend-specific
jmitrevs Aug 23, 2024
5bc6cbe
add type inference to oneAPI
jmitrevs Aug 25, 2024
c0cf580
add oneAPI to pytorch tests
jmitrevs Aug 27, 2024
8c827b8
fix pooling with padding for oneAPI and Quartus
jmitrevs Aug 29, 2024
a4f4bd9
Merge branch 'main' into oneapi_backend
jmitrevs Sep 12, 2024
f1c0301
Merge branch 'main' into oneapi_backend
jmitrevs Sep 13, 2024
7e0a8ca
Compilation for larger models enabled by increasing -fconstexpr-steps
laurilaatu Sep 13, 2024
acdc363
Merge pull request #6 from laurilaatu/oneapi_constexpr_fix
jmitrevs Sep 13, 2024
d1e14de
add oneapi clone tests; remove reduntand multi_clone test
jmitrevs Sep 18, 2024
1b78e57
remove some attributes to avoid overwrite warnings
jmitrevs Sep 24, 2024
865e2c8
Merge branch 'main' into oneapi_backend
jmitrevs Oct 1, 2024
f9a71f1
make extra handling for oneAPI like others (as in PR #1067)
jmitrevs Oct 1, 2024
320615d
remove warnings for extra optimizers that are not scheduled on purpose
jmitrevs Oct 1, 2024
5d13de5
update parametrized activations
jmitrevs Oct 2, 2024
ae9266f
fix reference to alpha that had not been switched to param
jmitrevs Oct 2, 2024
8084fd9
Merge branch 'main' into oneapi_backend
jmitrevs Oct 2, 2024
641be56
add oneapi documentation
jmitrevs Oct 21, 2024
5761125
Merge branch 'main' into oneapi_backend
jmitrevs Oct 21, 2024
15c6cc3
Merge remote-tracking branch 'upstream/main' into oneapi_backend
jmitrevs Oct 22, 2024
0f475fe
add parallelization factor to the attributes for oneAPI
jmitrevs Oct 22, 2024
31678e9
Merge remote-tracking branch 'upstream/main' into oneapi_backend
jmitrevs Oct 24, 2024
85a6b16
Merge branch 'main' into oneapi_backend
JanFSchulte Oct 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions docs/advanced/oneapi.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
==============
oneAPI Backend
==============

The ``oneAPI`` backend of hls4ml is designed for deploying NNs on Intel/Altera FPGAs. It will eventually
replace the ``Quartus`` backend, which should really have been called the Intel HLS backend. (The actual Quartus
program continues to be used with IP produced by the ``oneAPI`` backend.)
This section discusses details of the ``oneAPI`` backend.

The ``oneAPI`` code uses SYCL kernels to implement the logic that is deployed on FPGAs. It naturally leads to the
accelerator style of programming. In the IP Component flow, which is currently the only flow supported, the
kernel becomes the IP, and the "host code" becomes the testbench. An accelerator flow, with easier deployment on
PCIe accelerator boards, is planned to be added in the future.

The produced work areas use cmake to build the projects in a style based
`oneAPI-samples <https://github.com/oneapi-src/oneAPI-samples/tree/main/DirectProgramming/C%2B%2BSYCL_FPGA>`_.
The standard ``fpga_emu``, ``report``, ``fpga_sim``, and ``fpga`` are supported. Additionally, ``make lib``
produces the library used for calling the ``predict`` function from hls4ml. The ``compile`` and ``build`` commands
in hls4ml interact with the cmake system, so one does not need to manually use the build system, but it there
if desired.

The ``oneAPI`` backend, like the ``Quartus`` backend, only implements the ``Resource`` strategy for the layers. There
is no ``Latency`` implementation of any of the layers.

Note: currently tracing and external weights (i.e. setting BramFactor) are not supported.

io_parallel and io_stream
=========================

As mentioned in the :ref:`I/O Types` section, ``io_parallel`` is for small models, while ``io_stream`` is for
larger models. In ``oneAPI``, there is an additional difference: ``io_stream`` implements each layer on its
own ``task_sequence``. Thus, the layers run in parallel, with pipes connecting the inputs and outputs. This
is similar in style to the `dataflow` implementation on Vitis, but more explicit. On the other hand, ``io_parallel``
always uses a single task, relying on pipelining within the task for good performance. In contrast, the Vitis
backend sometimes uses dataflow with ``io_parallel``.
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@

advanced/fifo_depth
advanced/extension
advanced/oneapi
advanced/accelerator
advanced/model_optimization

Expand Down
2 changes: 2 additions & 0 deletions hls4ml/backends/__init__.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
from hls4ml.backends.backend import Backend, get_available_backends, get_backend, register_backend # noqa: F401
from hls4ml.backends.fpga.fpga_backend import FPGABackend # noqa: F401
from hls4ml.backends.oneapi.oneapi_backend import OneAPIBackend
from hls4ml.backends.quartus.quartus_backend import QuartusBackend
from hls4ml.backends.symbolic.symbolic_backend import SymbolicExpressionBackend
from hls4ml.backends.vivado.vivado_backend import VivadoBackend
Expand All @@ -16,3 +17,4 @@
register_backend('Quartus', QuartusBackend)
register_backend('Catapult', CatapultBackend)
register_backend('SymbolicExpression', SymbolicExpressionBackend)
register_backend('oneAPI', OneAPIBackend)
Empty file.
Loading
Loading