pmemfile

⚠️ Discontinuation of the project

The pmemfile project will no longer be maintained by Intel.

• Intel has ceased development and contributions including, but not limited to, maintenance, bug fixes, new releases, or updates, to this project.
• Intel no longer accepts patches to this project.
• If you have an ongoing need to use this project, are interested in independently developing it, or would like to maintain patches for the open source software community, please create your own fork of this project.
• You will find more information here.

Introduction

Pmemfile project's goal is to provide low-overhead userspace implementation of file APIs using persistent memory. It consists of 2 major compoments:

• libpmemfile-posix - provides syscall-like API which can be used by applications
• libpmemfile - provides transparent access to libpmemfile-posix pools

How to build

Requirements:

• cmake >= 3.3
• libpmemobj-dev(el) >= 1.3 (http://pmem.io/nvml/)
• libsyscall_intercept-dev(el) (https://github.com/pmem/syscall_intercept)

$ mkdir build
$ cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX=/usr
$ make
$ make install

When developing:

$ ...
$ cmake .. -DCMAKE_BUILD_TYPE=Debug -DDEVELOPER_MODE=1 -DTEST_DIR=/mnt/pmem/pmemfile-tests
$ ...
$ ctest --output-on-failure

Note that in Debug mode "make install" installs only debug libraries.

Pmemfile-specific cmake variables:

• BUILD_LIBPMEMFILE=0 - disables building of libpmemfile.so
• BUILD_LIBPMEMFILE_POP=1 - builds tests using with libpmemfile-posix-over-POSIX library which imitates libpmemfile-posix.so, but uses POSIX functions provided by OS
• DEVELOPER_MODE=1 - enables coding style, whitespace, license checks and enables fail-on-warning flags
• LONG_TESTS=1 - enables tests which take much more time
• TEST_DIR=/mnt/pmem/test_dir - provides directory where tests will create its pools
• TRACE_TESTS=1 - dumps more info when test fails (requires cmake >= 3.4)
• TESTS_USE_FORCED_PMEM=1 - allows tests to force enable or force disable use of optimized flush in libpmemobj (to speed them up)
• USE_ASAN=1 - enables AddressSanitizer (only for debugging)
• USE_UBSAN=1 - enables UndefinedBehaviorSanitizer (only for debugging)
• LIBPMEMFILE_VALIDATE_POINTERS=1 - build-in support for PMEMFILE_PRELOAD_VALIDATE_POINTERS environment variable

Package for Debian-based distros

$ ./utils/build-deb.sh

Resulting packages will be in ./build-deb/release and ./build-deb/debug

Package for RPM-based distros

$ ./utils/build-rpm.sh

Resulting packages will be in ~/rpmbuild/RPMS.

Note: Before 1.0 pmemfile-posix API and ABI is considered unstable. We reserve the right to change the on-media layout without warning.

Description:

Example:

$ mkfs.pmemfile /dev/dax0.0 0
$ mkdir /tmp/mountpoint

$ sudo pmemfile-mount /dev/dax0.0 /tmp/mountpoint
$ alias pf='LD_PRELOAD=libpmemfile.so'

# or if you don't have root access:

$ alias pf='LD_PRELOAD=libpmemfile.so PMEMFILE_POOLS=/tmp/mountpoint:/dev/dax0.0'

# now all commands prefixed with 'pf' will see files under /tmp/mountpoint,
# but files will be stored in file system backed by /dev/dax0.0
$ pf mkdir /tmp/mountpoint/dir_in_pmemfile
$ pf cp README.md /tmp/mountpoint/dir_in_pmemfile
$ pf ls -l /tmp/mountpoint/
total 0
drwxrwxrwx 2 user group 4008 Feb 16 17:46 dir_in_pmemfile
$ pf ls -l /tmp/mountpoint/dir_in_pmemfile
total 16
-rw-r--r-- 1 user group 1014 Feb 16 17:46 README.md
$ pf cat /tmp/mountpoint/dir_in_pmemfile/README.md | wc -c
1014
# verify that files are stored in pmemfile pool:
$ ls -l /tmp/mountpoint/
total 0
$ ls -l /tmp/mountpoint/dir_in_pmemfile
ls: cannot access '/tmp/mountpoint/dir_in_pmemfile': No such file or directory

Limitations:

• libpmemfile[-posix].so does not support multiple processes accessing the pool (libpmemobj limitation)
• libpmemfile.so supports Linux only (other UNIX-like systems could be supported)
• libpmemfile.so supports x86_64 only (libsyscall_intercept, libpmem and libpmemobj limitation)
• libpmemfile.so is not safe with remotely replicated pool (libpmemfile-posix.so has no such limitation)

Debugging:

Environment variables:

libsyscall_intercept.so:

• INTERCEPT_LOG - log file for all intercepted syscalls (default: none)
• INTERCEPT_LOG_TRUNC - when set to 0, the above log file is not truncated (default: 1)
• INTERCEPT_HOOK_CMDLINE_FILTER - intercept syscalls only when process name matches this variable (used to skip gdb from intercepting) (default: empty)

libpmemobj.so:

• PMEMOBJ_LOG_FILE - log file (default: none)
• PMEMOBJ_LOG_LEVEL - logging level (default: 0)

libpmemfile-posix.so:

• PMEMFILE_POSIX_LOG_FILE - log file (default: none)
• PMEMFILE_POSIX_LOG_LEVEL - logging level (default: 0)

libpmemfile.so:

• PMEMFILE_PRELOAD_LOG - log file (default: none)
• PMEMFILE_PRELOAD_LOG_TRUNC - when set to 0, the above log file is not truncated (default: 1)
• PMEMFILE_EXIT_ON_NOT_SUPPORTED - when set to 1, aborts an application which uses unsupported syscall (default: 0)
• PMEMFILE_PRELOAD_PAUSE_AT_START - pauses initialization of pmemfile until debugger is attached (default: 0)

Other variables:

• PMEMFILE_BLOCK_SIZE - forces one block size (default: dynamic)
• PMEMFILE_CD - performs early chdir() to specified directory, used as a workaround for missing multi-process support when application must start from pmemfile-backed directory (default: none)
• PMEMFILE_IGNORE_INODE_FREE_ERRORS - when set to 1, disables abort() when freeing inode's metadata fails (it defers freeing to the next application start) - can be used to get out of out-of-space situations (default: 0)
• PMEMFILE_OVERALLOCATE_ON_APPEND - when set to 0, disables allocation of more space than required (default: 1)
• PMEMFILE_PRELOAD_PROCESS_SWITCHING - when set to 1, enables VERY slow emulation of multi-process support, used for testing pmemfile with file system test suites (default: 0)
• PMEMFILE_PRELOAD_VALIDATE_POINTERS - when set to 1, verifies memory reaching libpmemfile through syscall arguments is accessible; it's very slow, so it should never be used in production for non-buggy applications

Other stuff

• vltrace - tool for tracing applications and evaluating whether libpmemfile.so supports them (https://github.com/pmem/vltrace)