How to configure unit tests for parallel testing with tenant databases setting?

[netdjw]

So, this spatie/laravel-multitenancy package (v3.0.2) is configured like this:

config/multitenancy.php

return [
    'tenant_finder' => Spatie\Multitenancy\TenantFinder\DomainTenantFinder::class,
    'tenant_artisan_search_fields' => [
        'id',
    ],
    'switch_tenant_tasks' => [
        \Spatie\Multitenancy\Tasks\SwitchTenantDatabaseTask::class,
        \Spatie\Multitenancy\Tasks\PrefixCacheTask::class,
    ],
    'tenant_model' => Domain::class,
    'queues_are_tenant_aware_by_default' => true,
    'tenant_database_connection_name' => 'tenant',
    'landlord_database_connection_name' => 'landlord',
    'current_tenant_container_key' => 'currentTenant',
    'actions' => [
        'make_tenant_current_action' => MakeTenantCurrentAction::class,
        'forget_current_tenant_action' => ForgetCurrentTenantAction::class,
        'make_queue_tenant_aware_action' => MakeQueueTenantAwareAction::class,
        'migrate_tenant' => MigrateTenantAction::class,
    ],
];

config/database.php:

return [
    'default' => env('DB_CONNECTION', 'mysql'),
    'connections' => [
        'mysql' => [
            // ...
        ],
        'landlord' => [
            'driver' => env('DB_DRIVER', 'mysql'),
            'url' => env('DATABASE_URL'),
            'host' => env('DB_HOST', '127.0.0.1'),
            'port' => env('DB_PORT', '3306'),
            'database' => env('DB_DATABASE', 'forge'),
            'username' => env('DB_USERNAME', 'forge'),
            'password' => env('DB_PASSWORD', ''),
            // ...
        ],
        'tenant' => [
            'driver' => env('DB_DRIVER', 'mysql'),
            'url' => env('DATABASE_URL'),
            'host' => env('DB_HOST', '127.0.0.1'),
            'port' => env('DB_PORT', '3306'),
            'database' => env('TENANT_DB_NAME', 'forge'),
            'username' => env('TENANT_DB_USER', 'forge'),
            'password' => env('TENANT_DB_PASSWORD', ''),
            // ...
        ],
        'test_landlord' => [
            'driver' => env('DB_DRIVER', 'mysql'),
            'url' => env('DATABASE_URL'),
            'host' => env('DB_HOST', '127.0.0.1'),
            'port' => env('DB_PORT', '3306'),
            'database' => env('DB_DATABASE', 'test_landlord'),
            'username' => env('DB_USERNAME', 'forge'),
            'password' => env('DB_PASSWORD', ''),
            // ...
        ],
        'test_tenant' => [
            'driver' => env('DB_DRIVER', 'mysql'),
            'url' => env('DATABASE_URL'),
            'host' => env('DB_HOST', '127.0.0.1'),
            'port' => env('DB_PORT', '3306'),
            'database' => env('TENANT_DB_NAME', 'test_tenant'),
            'username' => env('TENANT_DB_USER', 'forge'),
            'password' => env('TENANT_DB_PASSWORD', ''),
            // ...
        ],
    ]
];

I've created a tests/MultitenancyTestCase.php file for extend the Laravel's base TestCase class:

use Illuminate\Foundation\Testing\RefreshDatabase;
use Spatie\Multitenancy\Concerns\UsesMultitenancyConfig;
use Tests\TestCase;

class MultitenancyTestCase extends TestCase
{
    use CreatesApplication;
    use RefreshDatabase;

    protected function setUp(): void
    {
        parent::setUp();
        // ...
        Event::listen(MadeTenantCurrentEvent::class, function () {
            $this->beginDatabaseTransaction();
        });

        $this->setUpLandlord();
        $this->setUpTenant();
        // ...
    }

    private function setUpLandlord(): void
    {
        Artisan::call('migrate --path=database/migrations/landlord --database=test_landlord');
        Artisan::call('db:seed --class=RoleSeeder');
        // ... some other seeding stuff for testing, eg. creating users, etc.
    }

    private function setUpTenant(): void
    {
        Artisan::call('tenants:artisan "migrate --database=test_tenant"');
    }

    protected function connectionsToTransact()
    {
        return [
            $this->landlordDatabaseConnectionName(),
            $this->tenantDatabaseConnectionName(),
        ];
    }
}

Now I run the command:

php artisan test --parallel --stop-on-error

And I see in the MySQL these databases:

| test_landlord               |
| test_landlord_test_1        |
| test_landlord_test_10       |
| test_landlord_test_11       |
| test_landlord_test_12       |
| test_landlord_test_2        |
| test_landlord_test_3        |
| test_landlord_test_4        |
| test_landlord_test_5        |
| test_landlord_test_6        |
| test_landlord_test_7        |
| test_landlord_test_8        |
| test_landlord_test_9        |

And it sounds goog, but I don't see any test_tenant_test_1 or similar databases where the parallel testing create databases for tenants. And I see this error in console:

 SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'hu' for key 'langs.langs_code_unique' (Connection: test_tenant, SQL: insert into `langs` (`name`, `code`, `updated_at`, `created_at`) values (Trinidad and Tobago, hu, 2023-05-18 11:13:21, 2023-05-18 11:13:21))

Additional info: if I run tests without --parallel option, it works fine. The reason why I need to use parallel testing is the number of tests. We have 2600+ test cases and the run time is over 25 minutes with single thread. With parallel option the run time is under 10 minutes. So it's crutial bot in developer experience, both in CI/CD pipeline cost.

The question is:

How should I configure tests to create tenant databases for parallel testing?

[netdjw]

Meanwhile I debugged some details.

My connectionsToTransact() gives back these two database connection names:

  array:2 [
    0 => "test_landlord"
    1 => "test_tenant"
  ]

But it seems to me the paratest not use this function for detecting database connection's name.

So I try to make a workaround to set the right tenant database name in the Config and create the test database for tenant:

$tenant_db_name =  $tenantConfig->getDbName() . '_test_' . $token;
DB::statement("CREATE DATABASE IF NOT EXISTS $tenant_db_name");
Config::set('database.connections.test_tenant.database', $tenant_db_name);

Later I run into a wierd behaviour of multitenancy package (maybe just for me wierd). When I call makeCurrent() function it's reset the tenant database name in config (database.connections.test_tenant.database). So I workaround this too.

// save tenant database name
$preset_tenant_db_name = Config::get('database.connections.test_tenant.database');

// set the first domain as current tenant
Domain::first()->makeCurrent();

// reset tenant database name
Config::set('database.connections.test_tenant.database', $preset_tenant_db_name);

And then I ran out from ideas at this point:

Artisan::call('tenants:artisan "migrate --database=test_tenant"');

At this point the multitenancy package (I guess) not use the loaded config, but reset again from config/database.php.

I would be happy with any idea...

Thanks!

[salvisb]

Hi @netdjw. Have you solved the above? If so, any chance you could share the solution?

[netdjw]

Unfortunately no, not yet.

[pratikkuikel]

@netdjw @salvisb Any update on this ? Testing takes lot of timeee.

[pratikkuikel]

I mean did you got parallel testing figured out or any workaround? Looking for help :-)

[netdjw]

No, I just marked incomplete our related tests.

[pratikkuikel]

This Article from Sarah Ting could be of help. Extending the ParallelTestingServiceProvider with the logic of your own.

I took another approach to enable parallel testing though.
I create fresh landlord and tenant test database everytime while running tests and seed that database.
In the boot method of my AppServiceProvider I hooked into parallelTesting.

public function boot(): void
    {
        ParallelTesting::setUpProcess(function (int $token) {
            if ($token === 1) {
                $currentConnection = config('database.default');

                Artisan::call('test:migrate-landlord --fresh');

                config(['database.default' => $currentConnection]);
                Artisan::call('db:seed');
            }
        });

        ParallelTesting::setUpTestCase(function (int $token) {
            Context::add('case', 'parallel');
        });
    }

I leveraged Context class to determine if the currently running test is parallel. And validated the existence of context to not run the setup from TestCase if the test is parallel as the database setup is already being handled by setUpProcess of parallelTesting. $initialized makes sure that the TestCase runs only one time and not recreate database for every test. And with this configuration I can run both sequential and parallel tests.

// TESTCASE
    protected static $initialized = false;

    protected function setUp(): void
    {
        parent::setUp();


        // This shouldn't run when running parallel tests, context is set on AppServiceProvider

        if (!Context::has('case')) {

            if (!self::$initialized) {

                $this->migrateDb();

                $this->seedDb();

                self::$initialized = true;
            }
        }

    }

    protected function migrateDb(): self
    {
        $this
            ->artisan("test:migrate-landlord --fresh");
        return $this;
    }

    protected function seedDb(): self
    {
        $this
            ->artisan("db:seed");
        return $this;
    }

[salvisb]

@pratikkuikel we are implementing something very similar to what you've just described. We utilize this trait to extend the base TestCase. However, I believe this setup only facilitates testing for a single tenant; it does not support switching between tenants or testing interactions between them, should that be necessary.

trait ConfigTestCaseTrait
{
    use RefreshDatabase, CreatesApplication;

    protected array $connectionsToTransact = ['landlord', 'tenant'];

    protected function beforeRefreshingDatabase(): void
    {
        $this->setupEnvironment();
    }

    protected function refreshTestDatabase(): void
    {
        if (! RefreshDatabaseState::$migrated) {
            $this->migrate();

            /** @var Application $application */
            $application = null;
            $this->app[Kernel::class]->setArtisan($application);

            RefreshDatabaseState::$migrated = true;
        }

        $this->beginDatabaseTransaction();
    }

    private function migrate(): void
    {
        $this->createLandlordDatabase();

        $this->createTenantDatabase();

        $this->artisan('migrate:fresh --database=landlord --path=database/migrations/landlord --drop-views');

        $this->artisan('migrate:fresh --database=tenant --path=database/migrations --drop-views');
    }

    private function setupEnvironment(): void
    {
        config(['database.default' => 'landlord']);

        config()->set('multitenancy.tenant_database_connection_name', 'tenant');

        config()->set('multitenancy.landlord_database_connection_name', 'landlord');

        config()->set('multitenancy.switch_tenant_tasks', []);

        $suffix = ParallelTesting::token() ? '_'.ParallelTesting::token() : '';

        config([
            'database.connections.landlord' => [
                'driver' => 'mysql',
                'username' => env('DB_USERNAME', 'root'),
                'host' => env('DB_HOST', '127.0.0.1'),
                'port' => env('DB_PORT', '3306'),
                'password' => env('DB_PASSWORD'),
                'database' => env('DB_DATABASE_LANDLORD').$suffix,
            ],

            'database.connections.tenant' => [
                'driver' => 'mysql',
                'username' => env('DB_USERNAME', 'root'),
                'host' => env('DB_HOST', '127.0.0.1'),
                'port' => env('DB_PORT', '3306'),
                'password' => env('DB_PASSWORD'),
                'database' => env('DB_DATABASE_TENANT').$suffix,
            ],

        ]);
    }

    private function createLandlordDatabase(): void
    {
        $database = config('database.connections.landlord.database');
        $charset = config('database.connections.landlord.charset', 'utf8mb4');
        $collation = config('database.connections.landlord.collation', 'utf8mb4_unicode_ci');

        DB::connection('mysql')->unprepared("CREATE DATABASE IF NOT EXISTS $database CHARACTER SET $charset COLLATE $collation;");
    }

    private function createTenantDatabase(): void
    {
        $database = config('database.connections.tenant.database');
        $charset = config('database.connections.tenant.charset', 'utf8mb4');
        $collation = config('database.connections.tenant.collation', 'utf8mb4_unicode_ci');

        DB::connection('mysql')->unprepared("CREATE DATABASE IF NOT EXISTS $database CHARACTER SET $charset COLLATE $collation;");
    }
}

[pratikkuikel]

@salvisb Great, your setup is cleaner than mine. I think in my setup, juggling between tenants is possible; I need to test it out first. What I am doing rn is when I am seeding landlord database, I create a tenant and tenant database. Let me try with creating multiple tenants.