You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
These lines fire a concurrent db insert task for each new alert. This is not a problem in and of itself, since version 3+ of SQLite implements a multi-read/single-write concurrency model, but there's a catch.
The process of writing to the database requires the following logical steps
obtain a SHARED lock -> still readable, new readers allowed
obtain a RESERVED lock -> still readable, new readers allowed
write some stuff durably to the transaction log -> still readable, new readers allowed
obtain a PENDING lock -> still readable, no new readers allowed
Each of the statements goes through this pipe, which means we have a bunch of (green) threads who likely all acquired a PENDING lock, so no new readers are allowed.
To avoid issues as the ones seen in #924 we should implement batch inserts into alerts table.
Handling all of the recording logic in a single transaction would also be ideal. Right now, we're inserting independently prompts, outputs and alerts. We could write into the 3 tables acquiring a lock for the transaction. Docs
BEGIN IMMEDIATE
-- write to prompts, output, and alertsCOMMIT;
Additional Context
No response
The text was updated successfully, but these errors were encountered:
Description
These lines fire a concurrent db insert task for each new alert. This is not a problem in and of itself, since version 3+ of SQLite implements a multi-read/single-write concurrency model, but there's a catch.
The process of writing to the database requires the following logical steps
Here's SQLite docs page.
Each of the statements goes through this pipe, which means we have a bunch of (green) threads who likely all acquired a PENDING lock, so no new readers are allowed.
Thanks to @blkt for the above investigation
To avoid issues as the ones seen in #924 we should implement batch inserts into
alerts
table.Handling all of the recording logic in a single transaction would also be ideal. Right now, we're inserting independently
prompts
,outputs
andalerts
. We could write into the 3 tables acquiring a lock for the transaction. DocsAdditional Context
No response
The text was updated successfully, but these errors were encountered: