DriftIsolate.connect waits forever occasionally

[laniakea33]

Hi!
I'm working on app which needs to access database in UI isolate and background service also. When app is opened, on main function, I launch a drift isolate and connect main isolate to it, register Sendport to IsolateNameServer also. After that, user start background service which collects location information into drift database. So when the background service is launched (other isolate), on onStart() callback function, service searches for the sendport on the IsolateNameServer and call connect function. So two client isolates (main and background) are connected to the drift isolate, location data is collected to local db in background service and presented to ui in main isolate through query stream. Below is the core code.

drift_database.dart

class LocalDatabase extends _$LocalDatabase {
  LocalDatabase(super.executor);

  ...

  static Future<LocalDatabase> instance() async {
    final isolate = await createDatabaseIsolate();
    final connection = await isolate.connect();
    return LocalDatabase(connection);
  }
}

Future<DriftIsolate> createDatabaseIsolate() async {
  final port = IsolateNameServer.lookupPortByName(dbPortName);
  if (port != null) {
    return DriftIsolate.fromConnectPort(port);
  } else {
    final token = RootIsolateToken.instance;
    return DriftIsolate.spawn(() {
      BackgroundIsolateBinaryMessenger.ensureInitialized(token!);

      return LazyDatabase(() async {
        final dbFolder = await getDatabaseDirectory();
        final file = File(p.join(dbFolder.path, dbFileName));
        return NativeDatabase(file);
      });
    }).then((value) {
      IsolateNameServer.registerPortWithName(value.connectPort, dbPortName);
      return value;
    });
  }
}

main.dart

Future<void> main() async {
  WidgetsFlutterBinding.ensureInitialized();

  ...

  final localDatabase = await LocalDatabase.instance();

  ...

  runApp(ProviderScope(
    overrides: [
      localDatabaseProvider.overrideWithValue(localDatabase),
    ],
    child: configuredApp,
  ));
}

position_service.dart

@pragma('vm:entry-point')
Future<void> onStart(ServiceInstance service) async {
  
  ...
  
  LocalDatabase? db;

  ...

  service.on('startSavingPosition').listen((event) {
    ...

    positionSubscription = g.Geolocator.getPositionStream(
      locationSettings: const g.LocationSettings(
        accuracy: g.LocationAccuracy.best,
        distanceFilter: positionDistanceFilter,
      ),
    ).listen((event) {
      runZonedGuarded<Future<void>>(() async {
        double distance = double.parse((lastPosition != null
            ? g.Geolocator.distanceBetween(
          lastPosition!.latitude,
          lastPosition!.longitude,
          event.latitude,
          event.longitude,
        )
            : 0)
            .toStringAsFixed(2));

        await db?.createPosition(
          PositionsCompanion(
            courseId: Value(courseId!),
            latitude: Value(event.latitude),
            longitude: Value(event.longitude),
            timestamp: Value(DateTime.now().millisecondsSinceEpoch),
            distance: Value(distance),
          ),
        );

        ...

      }, (error, stack) {
        ...
      });
    });
  });

  ...

  db = await LocalDatabase.instance();
}

This works well but occasionally 'await LocalDatabase.instance();' is never finished and when I look deep into the code, I figured out that 'await isolate.connect();' is never finished because 'await client.serverInfo;' in 'connectToRemoteAndInitialize()' function in the 'remote.dart' file is never return some value.

I guess this happens when this app tries to connect to the drift isolate which has some problem, but I couldn't know how to reproduce this issue. Should I ping with timeout to this drift isolate before every database access? Is there any solution? Sorry for my bad English and thank you!

[simolus3]

Do you have a way to reproduce this reliably if you can hit this in the debugger? In drift version 2.20.0, you can pass a timeout parameter in .connect() via connectTimeout. If you run into a timeout then, the isolate is likely unable to accept new connections for some reason. This is still worth figuring out because it shouldn't happen, but at least you can likely work around it by starting another isolate.

[laniakea33]

I also tried to find a stable way to reproduce it, but unfortunately I couldn't find it. However, it does not occur when connecting the isolate for the first time from the UI while opening the app, and it occurs when connecting the second time from a background service. Currently, I set a timeout in the connect() function, and in case of timeout, I work around by regenerating the drift isolate and then reconnecting to it on both the UI and background service. However, if there is a problem with drift isolate, not only connect() but also other general query functions will be in a waiting state forever. So I'm worried that I'll have to set timeout and regeneration logic in every query function.

[simolus3]

Yeah that doesn't sound good... To me, it sounds like there might be a fatal error in the background isolate that we're somehow missing... DriftIsolate.spawn has a callback which you can use to customize how we setup the isolate, can you try this?

isolateSpawn: <T>(function, arg) async {
  final errorPort = ReceivePort();
  final exitPort = ReceivePort();

  errorPort.listen((error) {
    print('Unhandled drift isolate error: $error');
  });

  exitPort.first.whenComplete(() {
    exitPort.close();
    errorPort.close();
  });

  return await Isolate.spawn(
    function,
    arg,
    errorsAreFatal: true,
    onError: errorPort.sendPort,
    onExit: exitPort.sendPort,
  );
},

Do you see any errors being printed before the timeout in that case?