Database upgrades should not rely on current Python classes

[kozlovsky]

Tribler has a database upgrade logic, which applies during the Tribler start when necessary. The database upgrader component goes through the list of upgrades and applies them sequentially:
7.2 -> 7.3 -> 7.3.1 -> 7.5 -> 7.6 -> 7.7

In this list, we have two types of upgrades:

• those which modify the existing database by adding some columns/indexes
• those which create a new database with a slightly different structure and copy data to the new database.

The second group currently consists of two migrations: 7.2 -> 7.3 and 7.5 -> 7.6. Implementation of these migrations uses current Python classes to define database schema. Generally speaking, this is incorrect. For example, an upgrade from 7.2 to 7.3, when applied from Tribler 7.6, actually creates a database with a 7.6 structure. (We still need to apply further migrations, as some of them change not structure, but the database content: for example, 7.3 -> 7.3.1 upgrade applies content filters to the database).

Now it works, because current database changes are compatible. But it may break in the future: some changes in 7.8 may break previous migrations 7.2 -> 7.3 or 7.5 -> 7.6.

So, with current logic, when writing a new migration, it is necessary to check that the previous migrations work correctly.

It is better to rewrite previous database upgrades to use SQL only and not rely on current Python classes. DDL commands should be serialized in string form and not be calculated dynamically.

Later, when PonyORM adds support of migrations, it would be possible to use them, but previous upgrades should use plain SQL.

[drew2a]

Related to:

• Current upgrader's implementation is a time bomb #7382

[qstokkink]

Migrations have now been removed.