Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use wazero #582

Draft
wants to merge 154 commits into
base: main
Choose a base branch
from
Draft

use wazero #582

wants to merge 154 commits into from

Conversation

faddat
Copy link
Contributor

@faddat faddat commented Dec 19, 2024

status

  • Testfloats now runs. Working on getting tests that do instantiation to work correctly.

problems

  • gas system immature
  • somewhere, pages is being read as bytes or vice versa
  • Testfloats is getting different results depending on where it is run (m1 vs x86)

what?

To ensure that contracts can run reliably without stopping chains, we shall endeavor to swap out wasmer for wazero. Wasmer is a great wasm vm written in rust, a great programming language. Wazero is a great web assembly virtual machine written in go, a great programming language (with horrible c interop). Both languages get spicy around interop, so it is best to code an entire blockchain application in just one programming language. Additionally, Go memory management breaks when using cgo to import C.

why?

Because cgo doesn't work all that well, and messes up some basic features of Go. Also, similar ABI issues and c interop issues on the rust side, so it is probably not a good idea to use cgo / unsafe in go / unsafe in rust on >40 production networks, even if we are just accounting for the extreme increase in random explosions cause by cgo. But in fact we will be able to reduce explosions in many other ways too.

Probably right now this is a proof of concept.

How?

When the lint PR's are merged to main, they can be merged here also.

  • clarify structure

  • Create an interface for runtimes so that the runtime interface is clear

  • implement wazero in the runtime folder

  • implement the

whats the trick?

The trick is that cosmwasm's vm package is imported by libwasmvm and provides a lot of functions. We will need to recreate that for wazero and use the host functions feature.

there's a pretty clear migration path to wazero insrtead of wasmer in wasmvm.

When I began to work on cosmwasgo, I made a design mistake and made things less modular.

Today, I began to go through all of the code file by file, and I was able to get to a pretty good place. I still need to reconcile your last commit though.

Although I have hardcoded the runtime, I think you can see how using the runtime folder and the runtime interface allows for a pretty singificant simplification of wasmvm.

There are still remaining questions around gas metering, and a few other things.

cheers!

-Jacob

Note: even if you do not reveiw or merge this, I would appreciate not closing it. Among other things I am using the PR to compare.

Random notes that I will add to as I go:

  • the code inside libwasmvm folder defines interfaces for contracts. It needs to be reimplemented in go using wazero.
  • I'm going one by one through libwasmvm/* and setting that up in Go
    • TestAnalyzeCode done
    • TestStoreCode done
    • Memory management (memory.rs) is now in memory.go
    • Various db methods added to the host
  • This way, we will no longer need a huge binary blob to run cosmwasm contracts in chains using go, and performance and reliability will be improved.
  • It's possible that this may enable us to just use the SDK's gas meter
  • This pull request updates none of the tests. The goal is to ensure that this code passes every single test used by the wasmer version, without exception.

@faddat faddat changed the title merged chaos use wazero, and intraoduce an abstraction for the virtual macine Dec 19, 2024
@faddat faddat marked this pull request as draft December 20, 2024 07:06
@faddat faddat changed the title use wazero, and intraoduce an abstraction for the virtual macine use wazero, and introduce an abstraction for the virtual macine Dec 20, 2024
- Updated `MockEnv` to use `types.Uint64` for time representation.
- Introduced helper functions for memory operations: `readMemory` and `writeMemory`.
- Replaced direct memory read/write calls in `hostGet` and other functions with the new helpers.
- Removed the `MemoryAllocator` struct and related functions, simplifying memory management.
- Cleaned up unused code and deleted obsolete files in the `libwasmvm` directory.
- Adjusted function signatures and error handling for better clarity and consistency.

This commit enhances the overall structure and readability of the codebase while ensuring proper memory handling.
run tests
- Simplified `allocateInContract` with reduced debug logging
- Streamlined `hostDbRead` function with more concise memory handling
- Extracted memory management logic into a new `instantiate.go` file
- Removed verbose debug print statements in runtime functions
- Improved error handling and reduced code complexity in memory allocation routines
- Simplified memory and export validation in `analyzeForValidation`
- Removed commented-out code and unnecessary complexity
- Updated `allocate` and `deallocate` host functions with improved error handling
- Streamlined interface version and export checks
- Removed verbose comments and simplified function implementations
faddat added 19 commits January 30, 2025 18:14
- Deleted unused mutex in test file
- Reordered import statements in runtime files
- Removed unused functions like `readAndValidateResult`
- Simplified host function implementations by removing unnecessary context modifications
- Simplified host function registration in `registerhostfunctions.go`
- Removed redundant function registration checks and logging
- Improved context handling for runtime environment in host functions
- Streamlined iterator and crypto function registration
- Enhanced error handling and debug logging for host functions
- Deleted unused `prettyJSON` helper function from `internal/api/api_test.go`
- Simplified test file by removing unnecessary JSON formatting utility
- Kept `prettyPrint` function for struct formatting
This commit introduces several key changes:
- Moved host functions to a separate package
- Simplified host function registration
- Updated memory management and region handling
- Added String() method to Checksum type for hex encoding
- Improved error handling and logging in host functions
- Restructured runtime interfaces for better modularity
Modify the Secp256k1RecoverPubkey method call to match the updated function signature, which now returns an additional value. This change ensures compatibility with the latest API implementation.
This commit extends the cryptographic capabilities of the library by:
- Adding BLS12-381 cryptographic function signatures to the GoAPI type
- Updating host crypto functions to use new runtime and memory management
- Implementing gas charging for BLS12-381 operations
- Refactoring import paths and function signatures for crypto-related methods
@faddat
Copy link
Contributor Author

faddat commented Feb 14, 2025

Yesterday I broke this Branch but it was intentional. There really wasn't any way for me to break changes up into pieces and I wanted to break this into multiple packages

This commit introduces comprehensive gas management modules for the WasmVM runtime:
- Created gas package with core GasState implementation
- Added gas configuration modules for different CosmWasm versions (v1.x and v2.x)
- Implemented default gas configuration methods
- Structured gas tracking and consumption logic
- Prepared foundation for flexible gas metering across runtime versions
…anagement

This commit introduces a new WasmRuntime interface that defines a comprehensive set of methods for managing WebAssembly smart contract lifecycles, including:
- Code storage and management operations
- Contract instantiation, execution, and migration methods
- IBC (Inter-Blockchain Communication) entry points
- Caching and resource management
- Metrics retrieval

The interface provides a flexible and extensible approach to handling WASM contract interactions across different runtime environments.
This commit introduces several improvements to the internal API library:
- Simplified runtime initialization and cache management
- Enhanced error handling with more descriptive error messages
- Removed unused methods and simplified function signatures
- Updated import paths to reflect new package structure
- Added more descriptive comments for public functions
- Streamlined code storage and contract lifecycle methods

The changes improve code readability, error reporting, and overall maintainability of the WASM runtime library.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant