I'm getting a "Packets out of order" exception using MySQL

[maxacarvalho]

Shlink version

3.6.3

PHP version

8.1

How do you serve Shlink

RoadRunner Docker image

Database engine

MySQL

Database version

8

Current behavior

After a while running the application throws the following exception:

nickdev-shlink-1  | 2023-07-28T17:42:44+0000	INFO	server      	[2023-07-28T17:42:44.730586+00:00] [c67aab06-9c69-4d8d-a59b-0196abb8766d] Shlink.ERROR - ErrorException: Packets out of order. Expected 1 received 0. Packet size=145 in /etc/shlink/vendor/doctrine/dbal/src/Driver/PDO/Statement.php:121
nickdev-shlink-1  | Stack trace:
nickdev-shlink-1  | #0 [internal function]: Mezzio\ProblemDetails\ProblemDetailsMiddleware::Mezzio\ProblemDetails\{closure}(2, 'Packets out of ...', '/etc/shlink/ven...', 121)
nickdev-shlink-1  | #1 /etc/shlink/vendor/doctrine/dbal/src/Driver/PDO/Statement.php(121): PDOStatement->execute(NULL)
nickdev-shlink-1  | #2 /etc/shlink/vendor/doctrine/dbal/src/Connection.php(1096): Doctrine\DBAL\Driver\PDO\Statement->execute()
nickdev-shlink-1  | #3 /etc/shlink/vendor/doctrine/orm/lib/Doctrine/ORM/Persisters/Entity/BasicEntityPersister.php(756): Doctrine\DBAL\Connection->executeQuery('SELECT t0.id AS...', Array, Array)
nickdev-shlink-1  | #4 /etc/shlink/vendor/doctrine/orm/lib/Doctrine/ORM/EntityRepository.php(241): Doctrine\ORM\Persisters\Entity\BasicEntityPersister->load(Array, NULL, NULL, Array, NULL, 1, NULL)
nickdev-shlink-1  | #5 /etc/shlink/module/Rest/src/Service/ApiKeyService.php(85): Doctrine\ORM\EntityRepository->findOneBy(Array)
nickdev-shlink-1  | #6 /etc/shlink/module/Rest/src/Service/ApiKeyService.php(52): Shlinkio\Shlink\Rest\Service\ApiKeyService->getByKey('fc4398f7-762a-4...')
nickdev-shlink-1  | #7 /etc/shlink/module/Rest/src/Middleware/AuthenticationMiddleware.php(47): Shlinkio\Shlink\Rest\Service\ApiKeyService->check('fc4398f7-762a-4...')
nickdev-shlink-1  | #8 /etc/shlink/vendor/mezzio/mezzio/src/Middleware/LazyLoadingMiddleware.php(26): Shlinkio\Shlink\Rest\Middleware\AuthenticationMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #9 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(49): Mezzio\Middleware\LazyLoadingMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #10 /etc/shlink/module/Rest/src/Middleware/BodyParserMiddleware.php(34): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #11 /etc/shlink/vendor/mezzio/mezzio/src/Middleware/LazyLoadingMiddleware.php(26): Shlinkio\Shlink\Rest\Middleware\BodyParserMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #12 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(49): Mezzio\Middleware\LazyLoadingMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #13 /etc/shlink/vendor/mezzio/mezzio-router/src/Middleware/ImplicitOptionsMiddleware.php(75): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #14 /etc/shlink/vendor/mezzio/mezzio/src/Middleware/LazyLoadingMiddleware.php(26): Mezzio\Router\Middleware\ImplicitOptionsMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #15 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(49): Mezzio\Middleware\LazyLoadingMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #16 /etc/shlink/module/Rest/src/Middleware/ErrorHandler/BackwardsCompatibleProblemDetailsHandler.php(22): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #17 /etc/shlink/vendor/mezzio/mezzio/src/Middleware/LazyLoadingMiddleware.php(26): Shlinkio\Shlink\Rest\Middleware\ErrorHandler\BackwardsCompatibleProblemDetailsHandler->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #18 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(49): Mezzio\Middleware\LazyLoadingMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #19 /etc/shlink/vendor/laminas/laminas-stratigility/src/MiddlewarePipe.php(75): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #20 /etc/shlink/vendor/laminas/laminas-stratigility/src/Middleware/PathMiddlewareDecorator.php(60): Laminas\Stratigility\MiddlewarePipe->process(Object(Laminas\Diactoros\ServerRequest), Object(Psr\Http\Server\RequestHandlerInterface@anonymous))
nickdev-shlink-1  | #21 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(49): Laminas\Stratigility\Middleware\PathMiddlewareDecorator->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #22 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(42): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #23 /etc/shlink/vendor/mezzio/mezzio-router/src/Middleware/ImplicitHeadMiddleware.php(77): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #24 /etc/shlink/vendor/mezzio/mezzio/src/Middleware/LazyLoadingMiddleware.php(26): Mezzio\Router\Middleware\ImplicitHeadMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #25 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(49): Mezzio\Middleware\LazyLoadingMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #26 /etc/shlink/vendor/mezzio/mezzio-router/src/Middleware/RouteMiddleware.php(50): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #27 /etc/shlink/vendor/mezzio/mezzio/src/Middleware/LazyLoadingMiddleware.php(26): Mezzio\Router\Middleware\RouteMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #28 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(49): Mezzio\Middleware\LazyLoadingMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #29 /etc/shlink/vendor/laminas/laminas-stratigility/src/MiddlewarePipe.php(75): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #30 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(49): Laminas\Stratigility\MiddlewarePipe->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #31 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(42): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #32 /etc/shlink/vendor/shlinkio/shlink-common/src/Middleware/CloseDbConnectionMiddleware.php(24): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #33 /etc/shlink/vendor/mezzio/mezzio/src/Middleware/LazyLoadingMiddleware.php(26): Shlinkio\Shlink\Common\Middleware\CloseDbConnectionMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #34 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(49): Mezzio\Middleware\LazyLoadingMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #35 /etc/shlink/vendor/laminas/laminas-stratigility/src/MiddlewarePipe.php(75): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #36 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(49): Laminas\Stratigility\MiddlewarePipe->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #37 /etc/shlink/vendor/laminas/laminas-stratigility/src/Middleware/PathMiddlewareDecorator.php(122): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #38 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(42): Psr\Http\Server\RequestHandlerInterface@anonymous->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #39 /etc/shlink/vendor/mezzio/mezzio-problem-details/src/ProblemDetailsMiddleware.php(46): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #40 /etc/shlink/vendor/mezzio/mezzio/src/Middleware/LazyLoadingMiddleware.php(26): Mezzio\ProblemDetails\ProblemDetailsMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #41 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(49): Mezzio\Middleware\LazyLoadingMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #42 /etc/shlink/vendor/laminas/laminas-stratigility/src/MiddlewarePipe.php(75): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #43 /etc/shlink/vendor/laminas/laminas-stratigility/src/Middleware/PathMiddlewareDecorator.php(60): Laminas\Stratigility\MiddlewarePipe->process(Object(Laminas\Diactoros\ServerRequest), Object(Psr\Http\Server\RequestHandlerInterface@anonymous))
nickdev-shlink-1  | #44 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(49): Laminas\Stratigility\Middleware\PathMiddlewareDecorator->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #45 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(42): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #46 /etc/shlink/module/Rest/src/Middleware/CrossDomainMiddleware.php(24): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #47 /etc/shlink/vendor/mezzio/mezzio/src/Middleware/LazyLoadingMiddleware.php(26): Shlinkio\Shlink\Rest\Middleware\CrossDomainMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #48 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(49): Mezzio\Middleware\LazyLoadingMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #49 /etc/shlink/vendor/laminas/laminas-stratigility/src/Middleware/ErrorHandler.php(129): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #50 /etc/shlink/vendor/mezzio/mezzio/src/Middleware/LazyLoadingMiddleware.php(26): Laminas\Stratigility\Middleware\ErrorHandler->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #51 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(49): Mezzio\Middleware\LazyLoadingMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #52 /etc/shlink/vendor/php-middleware/request-id/src/RequestIdMiddleware.php(32): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #53 /etc/shlink/vendor/mezzio/mezzio/src/Middleware/LazyLoadingMiddleware.php(26): PhpMiddleware\RequestId\RequestIdMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #54 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(49): Mezzio\Middleware\LazyLoadingMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #55 /etc/shlink/vendor/shlinkio/shlink-common/src/Middleware/ContentLengthMiddleware.php(24): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #56 /etc/shlink/vendor/mezzio/mezzio/src/Middleware/LazyLoadingMiddleware.php(26): Shlinkio\Shlink\Common\Middleware\ContentLengthMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #57 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(49): Mezzio\Middleware\LazyLoadingMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #58 /etc/shlink/vendor/shlinkio/shlink-common/src/Middleware/AccessLogMiddleware.php(26): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #59 /etc/shlink/vendor/mezzio/mezzio/src/Middleware/LazyLoadingMiddleware.php(26): Shlinkio\Shlink\Common\Middleware\AccessLogMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #60 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(49): Mezzio\Middleware\LazyLoadingMiddleware->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #61 /etc/shlink/vendor/laminas/laminas-stratigility/src/MiddlewarePipe.php(75): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #62 /etc/shlink/vendor/laminas/laminas-stratigility/src/Next.php(49): Laminas\Stratigility\MiddlewarePipe->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\Next))
nickdev-shlink-1  | #63 /etc/shlink/vendor/laminas/laminas-stratigility/src/MiddlewarePipe.php(75): Laminas\Stratigility\Next->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #64 /etc/shlink/vendor/laminas/laminas-stratigility/src/MiddlewarePipe.php(64): Laminas\Stratigility\MiddlewarePipe->process(Object(Laminas\Diactoros\ServerRequest), Object(Laminas\Stratigility\EmptyPipelineHandler))
nickdev-shlink-1  | #65 /etc/shlink/vendor/mezzio/mezzio/src/Application.php(34): Laminas\Stratigility\MiddlewarePipe->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #66 /etc/shlink/bin/roadrunner-worker.php(24): Mezzio\Application->handle(Object(Laminas\Diactoros\ServerRequest))
nickdev-shlink-1  | #67 /etc/shlink/bin/roadrunner-worker.php(32): {closure}()
nickdev-shlink-1  | #68 {main}

Expected behavior

I'd expect no exception.

How to reproduce

I just leave the application running on my development environment.

[acelaya]

What is the behavior you get when this exception happens. Full stop of the service? Only some requests fail? Requests continue working regardless of the error?

Have you tried any of the solutions suggested here https://stackoverflow.com/questions/63301495/pdoexception-packets-out-of-order-expected-0-received-1-packet-size-23?

[acelaya]

I'm also seeing it might be a bug on some PHP versions (see https://bugs.php.net/bug.php?id=81335), so I was going to suggest updating to the latest 8.1.x

However, I see you mentioned you use the docker image, but the docker image is based on PHP 8.2, so which one is true?

[maxacarvalho]

Sorry for the lack of response @acelaya
You are right, my mistake above, I'm using the Docker image so PHP version is 8.2.7.
I tried to set the MySQL max_allowed_packet to 130MB and I'm still getting the error above after the container is running for a while. I'm not sure what to do next, what's the recommended setting for that config variable?
Once I restart the container everything works as expected.

[acelaya]

I guess there's not much to do, as it's a PHP bug.

One of the latest comments there https://bugs.php.net/bug.php?id=81335, from 1 month ago, says it's still reproducible on PHP 8.2.7

I will try to update the base image (I think the most recent version is 8.2.9), but it doesn't seem it's confirmed to be fixed yet.

[acelaya]

Update on this. The last message in that PHP bug tracker thread mentions the fix has been merged.

Linked PR is tagged as PHP 8.2, so I would say this should be fixed now.

Shlink 4.1.1 was released a few days ago, so the corresponding docker image should include that fix.