fix: Database is locked using RWLock

[RisingOrange]

Sometimes when the add-on tries to access its database, it fails with OperationalError database is locked. An error message is shown to the user in this case and the current operation fails when this happens.

This is one of most frequent errors on Sentry.

OperationalError: database is locked: https://ankihub.sentry.io/issues/4632925659/?project=6546414

User error report: https://community.ankihub.net/t/freeze-while-syncing/105528

The reason for the error is that a thread tries to access the AnkiHub DB through the _AnkiHubDB class while the database is attached to the Anki's DB connection. This mostly works on MacOs and Linux, but not on Windows, where the error is raised.

This error is related:
DBError: Anki already open, or media currently syncing.: https://ankihub.sentry.io/issues/4574509733/?project=6546414
It is not shown to the user and handled silently instead (but when this happens we don't send user review data to the server). It happens when trying to attach the ankihub database to Anki's db connection while a connection to the ankihub database is open. This also seems to nearly exclusively happen on Windows. This error is also one of the most frequent ones on Sentry.

Related issues

Fix “database is locked” error

Proposed changes

We want a thread to have exclusive access to the AnkiHub database while it is attached to the Anki database. We also want threads to be able to concurrently access the AnkiHub database otherwise.

This can be solved using thread locks:

• We already have an attached_ankihub_db context manager. This PR modifies it and adds an an detached_ankihub_db context manager.
  • Queries to the AnkiHub DB now use the detached_ankihub_db context manager
  • Other code that needs the database to be detached (e.g. code that copies the DB files) now also uses the detached_ankihub_db context manager
• Multiple threads can use the detached_ankihub_db context concurrently
• While one thread is in the attached_ankihub_db context, no other thread can enter an attached_ankihub_db or detached_ankihub_db context
• This is implemented using the readerwriterlock library.

To make sure that the locks are used correctly, the db module no longer exposes attach_ankihub_db_to_anki_db_connection and detach_ankihub_db_from_anki_db_connection. Instead code outside the module can use the attached_ankihub_db and detached_ankihub_db context managers.

There was one feature which depended on the usage of attach_ankihub_db_to_anki_db_connection and detach_ankihub_db_from_anki_db_connection which can't be implemented using the context managers. This was the sorting capability of the EditedAfterSyncColumn. It can't be implemented using the context manager because for the query used for sorting we are passing the query as a string to Anki and don't control in which context the query is called. However this is not an important feature and probably very few people have used it.

How to reproduce the errors

I reproduced the errors on Windows by running the following snippets from Anki’s debug console:

database is locked

import ankihub
ankihub_db = ankihub.db.ankihub_db

ankihub.db.attach_ankihub_db_to_anki_db_connection()
# ankihub.db.detach_ankihub_db_from_anki_db_connection()
print(ankihub_db.ankihub_dids())

It fails after 5 seconds with sqlite3.OperationalError: database is locked.

When I remove the # in front of the line that detaches the db, the error doesn’t occur.

Anki already open, or media currently syncing

from ankihub import db

# Detach the db to reset the state if running the snippet multiple times
print("Detaching...")
db.db._detach_ankihub_db_from_anki_db_connection()
print(f"{db.is_ankihub_db_attached_to_anki_db()=}")
  
with db.ankihub_db.connection() as conn:
    ah_dids = conn.list("SELECT DISTINCT ankihub_deck_id FROM notes")
    print(f"{ah_dids=}")

    # This raises "anki.errors.DBError: Anki already open, or media currently syncing."
    print("Attaching...")
    db.db._attach_ankihub_db_to_anki_db_connection()

Resources

https://github.com/elarivie/pyReaderWriterLock

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (f95b9fb) 84.89% compared to head (f0e4d19) 85.12%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #817      +/-   ##
==========================================
+ Coverage   84.89%   85.12%   +0.23%     
==========================================
  Files          64       65       +1     
  Lines        5913     5931      +18     
==========================================
+ Hits         5020     5049      +29     
+ Misses        893      882      -11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pedroven]

LGTM

The base branch was changed.