Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split BPFMap API into managed and unmanaged #356

Merged
merged 7 commits into from
Aug 17, 2023

Conversation

geyslan
Copy link
Member

@geyslan geyslan commented Aug 4, 2023

Close: #354

commit 9cb8ba9 (HEAD -> map-effort, origin/map-effort)
Author: Geyslan Gregório [email protected]
Date: Mon Aug 7 17:38:03 2023 -0300

feat(map): introduce ReuseFD() to reuse a map fd

The `ReuseFD` function links the current BPFMap instance to a provided
map file descriptor. This enables the reuse of a map that was initiated
by another process.

Initial effort: #346

Co-authored-by: Francisco Javier Honduvilla Coto <[email protected]>

commit fe71839
Author: Geyslan Gregório [email protected]
Date: Wed Aug 9 16:46:41 2023 -0300

chore(map): logic refactoring and standardization

Introduced struct helpers to manage the C struct lifecycle on the C side,
which avoids problems with structs which may contain bitfields. See #244.

  These are replacements for 'BPFMapBatchOpts' and 'bpfMapBatchOptsToC()'.
  - cgo_bpf_map_batch_opts_new
  - cgo_bpf_map_batch_opts_free

  These are replacements for 'bpfMapCreateOptsToC()'
  - cgo_bpf_map_create_opts_new
  - cgo_bpf_map_create_opts_free

Restructured libbpf BPF map logic into separate files:
- 'map-common.go' contains generic BPF map types, constants, and helpers
  like 'GetMapInfoByFD()'.
- 'map-low.go' includes 'BPFMapLow' and respective logic and helpers
  like 'CreateMap()'.
- 'map-iterator.go' contains related types and logic.

In the 'BPFMap' struct:
- Removed fields like 'fd' and 'name' since they must be retrieved
  directly from 'libbpf'.
- Added 'bpfMapLow' instance for access to low-level methods.
- Deprecated 'BPFMap.GetMaxEntries' in favor of using 'MaxEntries'.
- Exposed 'InitialValue' and 'SetInitialValue' as public wrappers.
- Introduced MapFlags(), IfIndex() and MapExtra() methods.

Introduced 'misc.go' as a place for miscellaneous generic helpers.

Bug Fixes:
- Fixed cases where 'BPFMap.ValueSize()', possibly 0, was being passed
  directly to 'make()' - detected on the map of maps effort #343.

Further Refactoring and Standardization on the map related logic and on
lines touched by other changes:
- Removed check for 'BPFMap.Name()' against nil, as 'C.GoString(C.NULL)'
  returns "".
- Changed error returns to 'retC' in functions that return error values
  to avoid confusion with possible use of errno.
- Changed returns to 'valueC' in functions that return values or
  pointers to values.
- Suffixed variable names that are C types with 'C'.
- Applied the use of 'defer' for better resource management.

commit 7b73fc0
Author: Geyslan Gregório [email protected]
Date: Wed Aug 9 15:42:49 2023 -0300

fix(c): check against NULL before memcpy

commit 8b1190a
Author: Geyslan Gregório [email protected]
Date: Wed Aug 9 14:17:13 2023 -0300

chore(c): prefix helpers with 'cgo_'

Prefix all C helpers used in Go with 'cgo_', renaming two cases
where the functions does not reflect the libbpf API:

set_print_fn -> cgo_libbpf_set_print_fn
get_internal_map_init_value -> cgo_bpf_map__initial_value

commit f58df61
Author: Geyslan Gregório [email protected]
Date: Wed Aug 9 14:09:17 2023 -0300

chore(c): move C definitions to 'libbpfgo.c'

Leaving 'libbpfgo.h' only with declarations to avoid multiple
declaration errors when importing 'libbpfgo.h' from different Go files.

commit c9b7e75
Author: Geyslan Gregório [email protected]
Date: Wed Aug 9 13:53:53 2023 -0300

chore(map): relocate map logic to map.go

commit d45e8f6
Author: Geyslan Gregório [email protected]
Date: Tue Aug 8 16:12:26 2023 -0300

chore: del unneeded assignment to blank identifier

@geyslan geyslan force-pushed the map-effort branch 2 times, most recently from 167031a to 6883fbb Compare August 7, 2023 21:30
@geyslan geyslan self-assigned this Aug 7, 2023
@geyslan geyslan added the chore label Aug 7, 2023
@geyslan geyslan force-pushed the map-effort branch 2 times, most recently from a8c562b to 9acf966 Compare August 7, 2023 21:46
@javierhonduco
Copy link
Contributor

javierhonduco commented Aug 8, 2023

Thanks for doing this! Tested func (m *Module) GetMap() and things seem to work fine!

@geyslan geyslan force-pushed the map-effort branch 2 times, most recently from 0a289a3 to dd84a62 Compare August 8, 2023 23:08
@geyslan
Copy link
Member Author

geyslan commented Aug 8, 2023

Since this involved creating new files and relocating logic, it required making major changes in a single commit.

@geyslan geyslan marked this pull request as ready for review August 8, 2023 23:12
@geyslan
Copy link
Member Author

geyslan commented Aug 8, 2023

When this get merged, the next to come to light is the map of maps.

Despite the discussion in #343 (comment), I'm not so sure about creating another type like BPFMapOfMaps, as that would require another module method like GetMapOfMaps(). Perhaps best is to leave BPFMap and BPFMapLow with the same libbpf APIs and make usage assumptions based on respective selftest. Let's see.

@geyslan geyslan added the bug Something isn't working label Aug 9, 2023
@geyslan
Copy link
Member Author

geyslan commented Aug 9, 2023

Since this involved creating new files and relocating logic, it required making major changes in a single commit.

I'm going redo the bigger commit splitting it by subject for better understanding and review.

Leaving 'libbpfgo.h' only with declarations to avoid multiple
declaration errors when importing 'libbpfgo.h' from different Go files.
@geyslan
Copy link
Member Author

geyslan commented Aug 9, 2023

I'm going redo the bigger commit splitting it by subject for better understanding and review.

@yanivagman I've broken it down a bit more and left GetMapByID() and GetMapFDByID() implementations for an upcoming PR (map of maps). ReuseFD(), co-authored by @javierhonduco, is coming with this since it's a good example of map high and low level separation.

rwArray_test.go Show resolved Hide resolved
@geyslan geyslan mentioned this pull request Aug 10, 2023
map-low.go Show resolved Hide resolved
libbpfgo.go Show resolved Hide resolved
@geyslan geyslan added the feature New feature or request label Aug 16, 2023
Copy link
Collaborator

@yanivagman yanivagman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM,
The last commit (ReuseFD) should be feat and not chore,
Also two small questions in the following comments

libbpfgo.c Outdated Show resolved Hide resolved
selftest/reuse-fd/main.go Show resolved Hide resolved
@geyslan
Copy link
Member Author

geyslan commented Aug 17, 2023

Overall LGTM, The last commit (ReuseFD) should be feat and not chore, Also two small questions in the following comments

Thanks for reviewing this. I'm going to address all requests and bugs.

@geyslan geyslan force-pushed the map-effort branch 2 times, most recently from 70d093b to 25ed660 Compare August 17, 2023 21:05
geyslan and others added 4 commits August 17, 2023 18:08
Prefix all C helpers used in Go with 'cgo_', renaming two cases
where the functions does not reflect the libbpf API:

set_print_fn -> cgo_libbpf_set_print_fn
get_internal_map_init_value -> cgo_bpf_map__initial_value
Introduced struct helpers to manage the C struct lifecycle on the C side,
which avoids problems with structs which may contain bitfields. See aquasecurity#244.

  These are replacements for 'BPFMapBatchOpts' and 'bpfMapBatchOptsToC()'.
  - cgo_bpf_map_batch_opts_new
  - cgo_bpf_map_batch_opts_free

  These are replacements for 'bpfMapCreateOptsToC()'
  - cgo_bpf_map_create_opts_new
  - cgo_bpf_map_create_opts_free

Restructured libbpf BPF map logic into separate files:
- 'map-common.go' contains generic BPF map types, constants, and helpers
  like 'GetMapInfoByFD()'.
- 'map-low.go' includes 'BPFMapLow' and respective logic and helpers
  like 'CreateMap()'.
- 'map-iterator.go' contains related types and logic.

In the 'BPFMap' struct:
- Removed fields like 'fd' and 'name' since they must be retrieved
  directly from 'libbpf'.
- Added 'bpfMapLow' instance for access to low-level methods.
- Deprecated 'BPFMap.GetMaxEntries' in favor of using 'MaxEntries'.
- Exposed 'InitialValue' and 'SetInitialValue' as public wrappers.
- Introduced MapFlags(), IfIndex() and MapExtra() methods.

Introduced 'misc.go' as a place for miscellaneous generic helpers.

Bug Fixes:
- Fixed cases where 'BPFMap.ValueSize()', possibly 0, was being passed
  directly to 'make()' - detected on the map of maps effort aquasecurity#343.

Further Refactoring and Standardization on the map related logic and on
lines touched by other changes:
- Removed check for 'BPFMap.Name()' against nil, as 'C.GoString(C.NULL)'
  returns "".
- Changed error returns to 'retC' in functions that return error values
  to avoid confusion with possible use of errno.
- Changed returns to 'valueC' in functions that return values or
  pointers to values.
- Suffixed variable names that are C types with 'C'.
- Applied the use of 'defer' for better resource management.
The `ReuseFD` function links the current BPFMap instance to a provided
map file descriptor. This enables the reuse of a map that was initiated
by another process.

Initial effort: aquasecurity#346

Co-authored-by: Francisco Javier Honduvilla Coto <[email protected]>
@geyslan geyslan merged commit 21cf435 into aquasecurity:main Aug 17, 2023
9 checks passed
javierhonduco referenced this pull request in parca-dev/parca-agent Sep 15, 2023
…-libbpf-1.2 (#2013)

[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
|
[github.com/aquasecurity/libbpfgo](https://togithub.com/aquasecurity/libbpfgo)
| require | minor | `v0.4.9-libbpf-1.2.0.0.20230817212518-21cf435d454e`
-> `v0.5.0-libbpf-1.2` |

---

### Release Notes

<details>
<summary>aquasecurity/libbpfgo
(github.com/aquasecurity/libbpfgo)</summary>

###
[`v0.5.0-libbpf-1.2`](https://togithub.com/aquasecurity/libbpfgo/releases/tag/v0.5.0-libbpf-1.2)

#### What's Changed

- revive: update revive rules by
[@&#8203;rafaeldtinoco](https://togithub.com/rafaeldtinoco) in
[https://github.com/aquasecurity/libbpfgo/pull/333](https://togithub.com/aquasecurity/libbpfgo/pull/333)
- Wrap `bpf_prog_attach` and `bpf_prog_detach2` by
[@&#8203;mozillazg](https://togithub.com/mozillazg) in
[https://github.com/aquasecurity/libbpfgo/pull/335](https://togithub.com/aquasecurity/libbpfgo/pull/335)
- ci: running tests on multiple go versions by
[@&#8203;mozillazg](https://togithub.com/mozillazg) in
[https://github.com/aquasecurity/libbpfgo/pull/336](https://togithub.com/aquasecurity/libbpfgo/pull/336)
- libbpf: bump to v1.2.2 by
[@&#8203;geyslan](https://togithub.com/geyslan) in
[https://github.com/aquasecurity/libbpfgo/pull/342](https://togithub.com/aquasecurity/libbpfgo/pull/342)
- fix: vagrant image install by
[@&#8203;geyslan](https://togithub.com/geyslan) in
[https://github.com/aquasecurity/libbpfgo/pull/341](https://togithub.com/aquasecurity/libbpfgo/pull/341)
- Add SkipMemlockBump to NewModuleArgs by
[@&#8203;bobrik](https://togithub.com/bobrik) in
[https://github.com/aquasecurity/libbpfgo/pull/337](https://togithub.com/aquasecurity/libbpfgo/pull/337)
- fix: trim quote characters in osinfo by
[@&#8203;NDStrahilevitz](https://togithub.com/NDStrahilevitz) in
[https://github.com/aquasecurity/libbpfgo/pull/344](https://togithub.com/aquasecurity/libbpfgo/pull/344)
- helpers: handle non numeric kernel versions by
[@&#8203;NDStrahilevitz](https://togithub.com/NDStrahilevitz) in
[https://github.com/aquasecurity/libbpfgo/pull/345](https://togithub.com/aquasecurity/libbpfgo/pull/345)
- osinfo: add support for rhel by
[@&#8203;NDStrahilevitz](https://togithub.com/NDStrahilevitz) in
[https://github.com/aquasecurity/libbpfgo/pull/347](https://togithub.com/aquasecurity/libbpfgo/pull/347)
- osinfo: support additional alma linux id by
[@&#8203;NDStrahilevitz](https://togithub.com/NDStrahilevitz) in
[https://github.com/aquasecurity/libbpfgo/pull/348](https://togithub.com/aquasecurity/libbpfgo/pull/348)
- chore: increase timeout for selftests by
[@&#8203;geyslan](https://togithub.com/geyslan) in
[https://github.com/aquasecurity/libbpfgo/pull/349](https://togithub.com/aquasecurity/libbpfgo/pull/349)
- fix(selftest): handle arm64 function names by
[@&#8203;geyslan](https://togithub.com/geyslan) in
[https://github.com/aquasecurity/libbpfgo/pull/350](https://togithub.com/aquasecurity/libbpfgo/pull/350)
- dependabot add by
[@&#8203;gitworkflows](https://togithub.com/gitworkflows) in
[https://github.com/aquasecurity/libbpfgo/pull/351](https://togithub.com/aquasecurity/libbpfgo/pull/351)
- build(deps): bump github.com/stretchr/testify from 1.8.2 to 1.8.4 by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/aquasecurity/libbpfgo/pull/353](https://togithub.com/aquasecurity/libbpfgo/pull/353)
- build(deps): bump actions/checkout from 2 to 3 by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/aquasecurity/libbpfgo/pull/352](https://togithub.com/aquasecurity/libbpfgo/pull/352)
- chore: minor clang format and tidy adjustments by
[@&#8203;geyslan](https://togithub.com/geyslan) in
[https://github.com/aquasecurity/libbpfgo/pull/358](https://togithub.com/aquasecurity/libbpfgo/pull/358)
- chore: add helpers to dependabot by
[@&#8203;geyslan](https://togithub.com/geyslan) in
[https://github.com/aquasecurity/libbpfgo/pull/359](https://togithub.com/aquasecurity/libbpfgo/pull/359)
- build(deps): bump github.com/stretchr/testify from 1.8.0 to 1.8.4 in
/helpers by [@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/aquasecurity/libbpfgo/pull/360](https://togithub.com/aquasecurity/libbpfgo/pull/360)
- build(deps): bump golang.org/x/sys from 0.1.0 to 0.11.0 in /helpers by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/aquasecurity/libbpfgo/pull/361](https://togithub.com/aquasecurity/libbpfgo/pull/361)
- fix(selftest): kern_version logic by
[@&#8203;geyslan](https://togithub.com/geyslan) in
[https://github.com/aquasecurity/libbpfgo/pull/362](https://togithub.com/aquasecurity/libbpfgo/pull/362)
- fix(selftest): use channel for sync by
[@&#8203;geyslan](https://togithub.com/geyslan) in
[https://github.com/aquasecurity/libbpfgo/pull/364](https://togithub.com/aquasecurity/libbpfgo/pull/364)
- Split BPFMap API into managed and unmanaged by
[@&#8203;geyslan](https://togithub.com/geyslan) in
[https://github.com/aquasecurity/libbpfgo/pull/356](https://togithub.com/aquasecurity/libbpfgo/pull/356)
- fix(map): fix error wrap by
[@&#8203;geyslan](https://togithub.com/geyslan) in
[https://github.com/aquasecurity/libbpfgo/pull/368](https://togithub.com/aquasecurity/libbpfgo/pull/368)
- fix(helpers): lint err regarding return statement by
[@&#8203;geyslan](https://togithub.com/geyslan) in
[https://github.com/aquasecurity/libbpfgo/pull/369](https://togithub.com/aquasecurity/libbpfgo/pull/369)
- chore(github): add workflow test by
[@&#8203;geyslan](https://togithub.com/geyslan) in
[https://github.com/aquasecurity/libbpfgo/pull/370](https://togithub.com/aquasecurity/libbpfgo/pull/370)
- fix: `GetMap()` comment by
[@&#8203;geyslan](https://togithub.com/geyslan) in
[https://github.com/aquasecurity/libbpfgo/pull/371](https://togithub.com/aquasecurity/libbpfgo/pull/371)
- Map of maps by [@&#8203;geyslan](https://togithub.com/geyslan) in
[https://github.com/aquasecurity/libbpfgo/pull/366](https://togithub.com/aquasecurity/libbpfgo/pull/366)
- build(deps): bump golang.org/x/sys from 0.11.0 to 0.12.0 in /helpers
by [@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/aquasecurity/libbpfgo/pull/375](https://togithub.com/aquasecurity/libbpfgo/pull/375)
- build(deps): bump actions/checkout from 3 to 4 by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/aquasecurity/libbpfgo/pull/376](https://togithub.com/aquasecurity/libbpfgo/pull/376)

#### New Contributors

- [@&#8203;bobrik](https://togithub.com/bobrik) made their first
contribution in
[https://github.com/aquasecurity/libbpfgo/pull/337](https://togithub.com/aquasecurity/libbpfgo/pull/337)
- [@&#8203;gitworkflows](https://togithub.com/gitworkflows) made their
first contribution in
[https://github.com/aquasecurity/libbpfgo/pull/351](https://togithub.com/aquasecurity/libbpfgo/pull/351)
- [@&#8203;dependabot](https://togithub.com/dependabot) made their first
contribution in
[https://github.com/aquasecurity/libbpfgo/pull/353](https://togithub.com/aquasecurity/libbpfgo/pull/353)

**Full Changelog**:
aquasecurity/libbpfgo@v0.4.9-libbpf-1.2.0...v0.5.0-libbpf-1.2

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/parca-dev/parca-agent).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi44My4wIiwidXBkYXRlZEluVmVyIjoiMzYuODMuMCIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->
@geyslan geyslan deleted the map-effort branch November 28, 2023 12:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working chore feature New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Split BPFMap API into managed and unmanaged
3 participants