-
Notifications
You must be signed in to change notification settings - Fork 227
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rust bindings refactor #6309
Rust bindings refactor #6309
Conversation
Moves a bunch of stuff out of src/types.rs that did not belong: - Confidence - Variable - Function specific stuff - Refactored InstructionInfo, see the msp430 and riscv examples. - Renamed Function::from_raw to Function::ref_from_raw and fixed places where the ref was incremented twice - Fixed FunctionRecognizer leaking functions (see above) - Fixed some apis incorrectly returning Result where Option is clearer - Started to move destructured types to the From trait for converting to an from ffi types, see Location for an example - Started to remove bad module level imports (importing std modules like mem all over the place) - Moved some wrapper handle types to named handle field (this improves readability), see CoreArchitecture for an example - Removed some unintuitive getters, this is bad practice for Rust code, just access the field directly, see DataVariable for an example - General code cleanup, purposely did not run rustfmt, that will be a single seperate commit
- Fixed invalid views being able to invoke UB when dealing with databases - Cleaned up some helper code in dwarf_import - Fixed inverted is_null checks causing crashes! Oops!
Still a WIP, I think branch info is still invalid, need to figure out the issue there. - Fixed some invalid Ref lifetimes when constructing indirectly, see Array<DataVariable> for example - Added some more comments - Removed some "magic" functions like MLIL Function::ssa_variables There are still a bunch of invalid lifetimes that aren't crashing us due to the usage of those API's not living long enough. But they ARE an issue.
Trying to comment more TODO's as I go along. - Renamed llil::Function to llil::LowLevelILFunction for clarity and consistency - Take base structures by ref in StructureBuilder::set_base_structures to prevent premature drops - Added more raw to wrapper conversions - Removed UB prone apis - Getting rid of more core module references, use std! - Removed improper Guard usage in array impls for wrapper types with no context - Untangling the UB of the Label api, still only half done :/
- Misc formatting - Made Logger ref counted - Fixed leaking name of logger every time something was logged - Fixed the last (hopefully) of the unresolved labels - Simplified some code
componentArray was never freed
When linking you must depend on the -sys crate. This is because linker arguments (what says where to find binaryninjacore) are NOT transitive. The top level application crate MUST provide it. For more information see: - rust-lang/cargo#9554 (comment) - oxidecomputer/omicron#225
Use cargo to manage the git repo ref instead
This is where all shipped public plugins that are not arch/view/platform/lang will be at from now on Originally they were in the rust workspace, meaning they all shared a Cargo.lock which is ill-advised.
- More clarification on plugin/executable requirements - Made examples actually rust examples - Add Display impl for InstructionTextToken - Renamed feature "noexports" to "no_exports"
This really did bother me
We don't register a compatible log sink so they will just get sent into the void
This is being done in the hopes of curbing the multi-thousand line files that will occur once we flesh out the tests
Still need to add support for running tests in ci
- Architecture id's are now typed accordingly - Fix some clippy lints - Make instruction index public in LLIL - Removed useless helper functions - LLIL expressions and instruction indexes are now typed accordingly
This should show binaryninjacore-sys alongside binaryninja crate
- Remove lazy_static dependency - Remove hacky impl Debug for Type and just use the display impl - Add more debug impls - Reorder some top level namespace items - Add first type test
- Added main thread handler api - Register a headless main thread handler by default in initialization - Refactor QualifiedName to be properly owned - Loosened some type constraints on some apis involving QualifiedName - Fixed some apis that were crashing due to incorrect param types - Removed extern crate cruft for log crate - Simplified headless initialization using more wrapper apis - Fixed segments leaking because of no ref wrapper, see BinaryViewExt::segment_at - Added rstest to manage headless init in unit tests - Added some more unit tests - Refactored demangler api to be more ergonomic - Fixed minidump plugin not building
- Fixup usage of QualifiedName in plugins - Make QualifiedName more usable
- Make EdgeStyle type not wrap raw - Regression tests for WARP will run on all bins in the out dir now
Makes collaboration more in line with the refactor. Still a lot of work to do. Namely still need: - Proper errors - _with_opts functions - More ergonomic api - Better connection procedure - Updated documentation - A LOT of unit tests - An example - Typed id's for everything (i dont want BnString as the id!!!) - NEED to refactor the progress callbacks into the new progress api, but we should pull in some of the stuff the collab progress has - Elimination of apis that are dumb helpers
Because I have tomorrow and Friday to finish this up I went ahead and pulled in #5773, I think I should have enough time to refactor it and add some tests and examples. |
@emesare Thanks that solves my use case. BTW I hope in the future we can add a function to the coreAPI to recover the |
pull_request_target allows PR's to access the headless license, for this to be safe we need to prevent people from running the job. To prevent the job from being ran we add an environment requirement on testing that a reviewer must review the code and then manually approve it to run.
- Use GroupId instead of u64 - Use ProgressCallback in place of ProgressExecutor - Misc cleanup of FileMetadata - Add `save_to_path` and `save_to_accessor` to save modified binaries - Added binary_view unit tests - Added collaboration unit tests - Fixed a few issues with the collaboration apis - Renamed Command registration functions so that there is no import ambiguity - Split out RemoteUndoEntry - Collaboration apis now have a explicit `_with_progress` set of apis - Misc clippy lint fixes
- Add extra info to README.md - Refactor components api - Add components unit test
The rolled up commit will be on https://github.com/Vector35/binaryninja-api/tree/rust_refactor |
Community rust plugins to update: |
Merged with 6a63c17 |
@emesare are you going to add a load with progress func for the headless session? |
That sounds like a good idea, I'll add that today! Thank you. |
Thank you! I know this is most likely an inappropriate place to ask, but I'm trying to understand how to run a plugin via the Rust API (BinExport) specifically. Are there any examples anywhere on how to do this? I can also ask on Slack if that's a better place? |
No problem, by run you mean run a command that BinExport registered? If so than we would need to add the rust equivalent of binaryninja-api/binaryninjaapi.h Lines 14146 to 14168 in 8767400
In rust we can register commands but we cannot call them yet. I unfortunately have a few things in front of adding that, specifically I need to implement the BinaryDataNotification API. If you would like to make an issue for fleshing out plugin commands to support calling them from rust, I can assign it to myself. |
Also this is assuming the plugin command that you want to call does not then invoke some user interaction, that would require us to also add the interaction callback API. There is a TODO for that in the flow graph example binaryninja-api/rust/examples/flowgraph.rs Lines 52 to 53 in 6a63c17
|
@emesare I guess this is how it currently works in python? https://infosec.exchange/@binaryninja/113827475796582876 |
Ah I see, thank you for this explanation. I will wait for PluginCommand running to be added to the Rust API. Thanks again! |
Yea that would be through the Here are the issues:
|
Opening this PR for public discussion. This needs more testing and also lacks a few major changes that make sense to land alongside the already existing set of breaking changes.
The Rust bindings are being heavily refactored for the 4.3 release. This is to address many of the shortcomings and historical issues, namely around maintaining and safety.
Some of the additions of this branch include:
rustfmt
)cargo run --example
)RegisterId
)Reviewer Guide
TODO