Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce new CSAL ftl bdev #1771

Open
wants to merge 3 commits into
base: develop
Choose a base branch
from

Conversation

MaisenbacherD
Copy link

For this PR to be merged the PR openebs/spdk-rs#67
for spdk-rs needs to be resolved and the submodule reference must be
updated accordingly. :)

The SPDK ftl bdev allows to create a layered device with a (fast) cache
device for buffering writes that get eventually flushed out sequentially
to a base device. SDPK ftl is also known as the Cloud Storage
Acceleration Layer (CSAL).

This kind of device potentially enables the use of emerging storage
interfaces like Zoned Namespace (ZNS) or Flexible Data Placement (FDP)
capable NVMe devices. Up to this point, those NVMe command sets are not
yet supported in SPDK upstream. However, the acceleration aspect of a
fast cache device already adds value.
With future support of new devices for SDPK ftl, Mayastor would already
be capable of utilizing those features simply by upgrading SDPK.

For now, the ftl_mount_fs test cases are hidden behind the ignore
attribute because PCIe devices are required for this test until
SPDK v24.09 is picked up, which introduces variable sector size emulation
for ftl devices.
To run these tests use the following:

RUST_LOG=TRACE cargo test -- --test-threads 1 --test ftl_mount_fs --nocapture --ignored

This patch introduces a new ftl device uri scheme:

ftl:///<ftl-device-name>?bbdev=<dev-uri-nested-encoding>&cbdev=<dev-uri-nested-encoding>

<dev-uri-nested-encoding> can be any already valid device uri where '?'
are replaced with '!' and '&' are replaced with '|'.
With SPDK v24.05 only PCIe devices will work where the cache device is
formatted to 4KiB+64B LBA format and the base device to 4KiB LBA format.

From SPDK v24.09 on, any device with a block size of 4KiB will work for
both cache and base device.

CSAL paper reference:
https://dl.acm.org/doi/10.1145/3627703.3629566

@MaisenbacherD
Copy link
Author

Discussion question: The spdk_ftl_conf allows us to set configurations like core_mask and l2p_dram_limit. Are those configurations worth exposing through the ftl uri, or should we go with the default configuration?

MaisenbacherD and others added 3 commits November 14, 2024 08:37
The SPDK ftl bdev allows to create a layered device with a (fast) cache
device for buffering writes that get eventually flushed out sequentially
to a base device. SDPK ftl is also known as the Cloud Storage
Acceleration Layer (CSAL).

This kind of device potentially enables the use of emerging storage
interfaces like Zoned Namespace (ZNS) or Flexible Data Placement (FDP)
capable NVMe devices. Up to this point, those NVMe command sets are not
yet supported in SPDK upstream. However, the acceleration aspect of a
fast cache device already adds value.
With future support of new devices for SDPK ftl, Mayastor would already
be capable of utilizing those features simply by upgrading SDPK.

For now, the `ftl_mount_fs` test cases are hidden behind the `ignore`
attribute because PCIe devices are required for this test until
SPDK v24.09 is picked up, which introduces variable sector size emulation
for ftl devices.
To run these tests use the following:
```
RUST_LOG=TRACE cargo test -- --test-threads 1 --test ftl_mount_fs --nocapture --ignored
```

This patch introduces a new ftl device uri scheme:
```
ftl:///<ftl-device-name>?bbdev=<dev-uri-nested-encoding>&cbdev=<dev-uri-nested-encoding>
```

<dev-uri-nested-encoding> can be any already valid device uri where '?'
are replaced with '!' and '&' are replaced with '|'.
With SPDK v24.05 only PCIe devices will work where the cache device is
formatted to 4KiB+64B LBA format and the base device to 4KiB LBA format.

From SPDK v24.09 on, any device with a block size of 4KiB will work for
both cache and base device.

CSAL paper reference:
https://dl.acm.org/doi/10.1145/3627703.3629566

Co-authored-by: Indraneel M <[email protected]>
Signed-off-by: Dennis Maisenbacher <[email protected]>
Init SPDK tracing when `MayastorEnvironment.num_enties` is set to a
positive integer. This can be helpfull when developing new features.
Traces can be found in `/dev/shm` and have the pid or if not negative
the `MayastorEnvironment.shm_id` as a suffix.

Further information about traces and how to read the captured traces can
be found here: https://spdk.io/doc/nvmf_tgt_tracepoints.html

Signed-off-by: Dennis Maisenbacher <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant