Lock for single process restriction

[ethe]

Currently, Tonbo only supports single-process read and write operations. Accessing the database file from multiple processes can lead to undefined behavior. However, Tonbo does not prevent this. Therefore, Tonbo should implement a mechanism to lock access to the database files.

[irenjj]

I'd like to take it :)

[PhVHoang]

Hi @ethe , a silly question as I'm new to the project as well.

Should this locking mechanism be implemented in https://github.com/tonbo-io/fusio ? And will the lock prevent both read & write operations from multiple processes? Or only writes will be locked?

[ethe]

Hi @PhVHoang , I think it is different level behavior. Linux file system allows access a single file across multiple process, fusio does not promise much more safety than OS (or any other storage backend). locking db file (folder) to prevent logical error is the Tonbo requirement. Therefore, I think it might be better to lock folder on Tonbo (or other application) level.

[KKould]

The locks between multiple processes are often fslock, which should depend on the underlying implementation of each system such as libc::flock, so I think this issue depends on fusio.

• ref: https://github.com/brunoczim/fslock/blob/master/src/unix.rs#L323

[ethe]

Sure, fusio promises operation atomicity on local fs (fusio just pass through operations to local fs), however, the "lock" logic of Tonbo should be implmented in Tonbo, what Tonbo needs is db instance lock cross multiple files (wal log, version log, parquet files, etc.)

I was misunderstanding, I get it now, considering the target of fusio, I think fusio might not provide a lock abstraction, because fs lock does not make sense to other fs implementation than local fs. Just use std fs lock (or libc) should be fine.

[KKould]

It is a question whether OPFS supports fslock as planned under WASM (even if tonbo forces base_fs to be local, does all local storage support fslock ?)

[ethe]

does all local storage support fslock ?

It depends on target OS, I think in the most of cases it is guaranteed, OPFS does not support parallel read write on default.