Any issues with uploading to bunny.cdn?

[tacman]

I have a script that used to work that uploads files to bunny. It now gives an "idle timeout" error. I don't think I've changed anything, the script itself it quite simply, pretty much cut and paste from the upload documentation here.

It fails on line 106 of BunnyClient.php. Downloading works fine, and I can upload manually via the dashboard.

        $response = $this->client->sendRequest(
            request: $request,
        );

It isn't authentication, as uploading with an incorrect password fails with authentication error.

Any ideas how to debug this?

Thx.

[ToshY]

Hey @tacman 👋

I've seen someone mentioning a similar "idle timeout" issue a while back: #46 (comment). This was before I implemented PSR changes (to use your own HTTP client). I suspect it might have to due with the (large) file size. How large are the files you are trying to upload? If you have a small snippet of a reproducable example, I can try to reproduce it myself.

Anyway, as mentioned in the before-mentioned issue, you could try adding "timeout" options to your HTTP client. Specifically for Symfony, if you are using the Psr18Client for the BunnyClient, you could try and play around with the the timeout or max_duration options.

./config/packages/framework.yaml

framework:
  http_client:
    default_options:
      retry_failed: true
      http_version: '2.0'
      timeout: 10
      max_duration: 150

https://symfony.com/doc/current/http_client.html#dealing-with-network-timeouts

[tacman]

Makes sense. Currently I have

        $this->bunnyClient = new BunnyClient(
            client: new \Symfony\Component\HttpClient\Psr18Client(),
        );

What I want to have is

    public function __construct(
        private readonly HttpClientInterface $client,
    ) {

        $this->bunnyClient = new BunnyClient(
            client: $this->client,
        );

Obviously, the second one throws an error:

#message: "ToshY\BunnyNet\Client\BunnyClient::__construct(): Argument #1 ($client) must be of type Psr\Http\Client\ClientInterface, Symfony\Component\HttpClient\TraceableHttpClient given, called in /home/tac/g/survos/survos/packages/bunny-bundle/src/Service/BunnyService.php on line 37"
#code: 0
#file: "./vendor/toshy/bunnynet-php/src/Client/BunnyClient.php"

The Symfony docs from https://symfony.com/doc/current/http_client.html#interoperability say

The component is interoperable with four different abstractions for HTTP clients: Symfony Contracts, PSR-18, HTTPlug v1/v2 and native PHP streams. If your application uses libraries that need any of them, the component is compatible with all of them. They also benefit from autowiring aliases when the framework bundle is used.

If you are writing or maintaining a library that makes HTTP requests, you can decouple it from any specific HTTP client implementations by coding against either Symfony Contracts (recommended), PSR-18 or HTTPlug v2.

---

Symfony contracts are recommended, I think that encompasses everything, but if not, there must be a way to say "pass my Symfony Http client as a PSR-18". Or something like that, my terminology may certainly be wrong.

[ToshY]

If you are writing or maintaining a library that makes HTTP requests, you can decouple it from any specific HTTP client implementations by coding against either Symfony Contracts (recommended), PSR-18 or HTTPlug v2.

It's understandable that for a Symfony application the recommended way would be contracts (same as "Cache Contracts versus PSR-6"). While the contract implementations are (most of the time) easier to use, the PSR way is more generic and framework agnostic. Thus choosing the PSR-18 standard for this library.

Symfony contracts are recommended, I think that encompasses everything, but if not, there must be a way to say "pass my Symfony Http client as a PSR-18". Or something like that, my terminology may certainly be wrong.

I think I've mentioned this before in #121 (comment), which is due to the TraceableHttpClient implementing Symfony\Contracts\HttpClient\HttpClientInterface and not Psr\Http\Client\ClientInterface. From looking at the Symfony\Component\HttpClient\Psr18Client implementation, you can pass a HttpClientInterface to it.

So in your case, I think you can do the following:

    public function __construct(
        private readonly HttpClientInterface $client,
    ) {
        $this->bunnyClient = new BunnyClient(
            client: new \Symfony\Component\HttpClient\Psr18Client(
                client: $this->client
            ),
        );
    }

[tacman]

thanks, but the part I'm confused about is how to add the parameters to the ->request call, to get the callback.

[ToshY]

thanks, but the part I'm confused about is how to add the parameters to the ->request call, to get the callback.

Instead of adding the options to the request call itself, consider adding the options (like on_progress) when constructing your HTTP client. Here's an example with adding a logger to the on_progress example (instead of an event) to show you the gist of it.

src/Factory/Http/Psr18ClientFactory.php

<?php

declare(strict_types=1);

namespace App\Factory\Http;

use Psr\Log\LoggerInterface;
use Symfony\Component\DependencyInjection\Attribute\Autowire;
use Symfony\Component\HttpClient\HttpClient;
use Symfony\Component\HttpClient\Psr18Client;

final class Psr18ClientFactory
{
    public function __construct(
        #[Autowire(service: 'monolog.logger.http_client')]
        private LoggerInterface $logger,
    ) {
    }

    public function __invoke(): Psr18Client
    {
        $httpClient = HttpClient::create(
            defaultOptions: [
                'on_progress' => function (int $dlNow, int $dlSize): void {
                    $this->logger->info(
                        sprintf(
                            'Number of bytes: %s; Total size: %d',
                            $dlNow,
                            $dlSize,
                        ),
                    );
                },
            ],
        );

        return new Psr18Client($httpClient);
    }
}

config/services.yaml

  custom.http_client:
    class: Symfony\Component\HttpClient\Psr18Client
    factory: '@App\Factory\Http\Psr18ClientFactory'

  bunny.client:
    class: ToshY\BunnyNet\Client\BunnyClient
    bind:
      $client: '@custom.http_client'

  bunny.edge_storage.api:
    class: ToshY\BunnyNet\EdgeStorageAPI
    bind:
      $apiKey: '%env(STORAGE_ZONE_API_KEY)%'
      $client: '@bunny.client'
      $region: !php/const \ToshY\BunnyNet\Enum\Region::FS

  ToshY\BunnyNet\EdgeStorageAPI $bunnyEdgeStorageAPI: '@bunny.edge_storage.api'

Now using it in your service/controller:

    public function __construct(
        private readonly EdgeStorageAPI $bunnyEdgeStorageAPI,
    ) {
    }

   public function download(): void
   {
        $response = $this->bunnyEdgeStorageAPI->downloadFile(
            storageZoneName: 'my-storage-zone-1',
            fileName: 'bunny.jpg',
        );
   }

The above example will log the following:

phpfpm-1  | 2024-09-30T18:36:28.380106350Z {"message":"Number of bytes: 0; Total size: 0","context":{},"level":200,"level_name":"INFO","channel":"http_client","datetime":"2024-09-30T20:36:28.379933+02:00","extra":{}}
phpfpm-1  | 2024-09-30T18:36:28.380316052Z {"message":"Number of bytes: 0; Total size: 0","context":{},"level":200,"level_name":"INFO","channel":"http_client","datetime":"2024-09-30T20:36:28.380118+02:00","extra":{}}
phpfpm-1  | 2024-09-30T18:36:28.383697556Z {"message":"Number of bytes: 0; Total size: 0","context":{},"level":200,"level_name":"INFO","channel":"http_client","datetime":"2024-09-30T20:36:28.383521+02:00","extra":{}}
phpfpm-1  | 2024-09-30T18:36:28.383940361Z {"message":"Number of bytes: 0; Total size: 0","context":{},"level":200,"level_name":"INFO","channel":"http_client","datetime":"2024-09-30T20:36:28.383707+02:00","extra":{}}
phpfpm-1  | 2024-09-30T18:36:28.391529697Z {"message":"Number of bytes: 16238; Total size: 41141","context":{},"level":200,"level_name":"INFO","channel":"http_client","datetime":"2024-09-30T20:36:28.391345+02:00","extra":{}}
phpfpm-1  | 2024-09-30T18:36:28.391797619Z {"message":"Number of bytes: 16238; Total size: 41141","context":{},"level":200,"level_name":"INFO","channel":"http_client","datetime":"2024-09-30T20:36:28.391551+02:00","extra":{}}
phpfpm-1  | 2024-09-30T18:36:28.400298474Z {"message":"Number of bytes: 16238; Total size: 41141","context":{},"level":200,"level_name":"INFO","channel":"http_client","datetime":"2024-09-30T20:36:28.400111+02:00","extra":{}}
phpfpm-1  | 2024-09-30T18:36:28.400407307Z {"message":"Number of bytes: 16238; Total size: 41141","context":{},"level":200,"level_name":"INFO","channel":"http_client","datetime":"2024-09-30T20:36:28.400298+02:00","extra":{}}
phpfpm-1  | 2024-09-30T18:36:28.402074380Z {"message":"Number of bytes: 16238; Total size: 41141","context":{},"level":200,"level_name":"INFO","channel":"http_client","datetime":"2024-09-30T20:36:28.401911+02:00","extra":{}}
phpfpm-1  | 2024-09-30T18:36:28.402274856Z {"message":"Number of bytes: 16238; Total size: 41141","context":{},"level":200,"level_name":"INFO","channel":"http_client","datetime":"2024-09-30T20:36:28.402099+02:00","extra":{}}
phpfpm-1  | 2024-09-30T18:36:28.405500690Z {"message":"Number of bytes: 32530; Total size: 41141","context":{},"level":200,"level_name":"INFO","channel":"http_client","datetime":"2024-09-30T20:36:28.405305+02:00","extra":{}}
phpfpm-1  | 2024-09-30T18:36:28.405688721Z {"message":"Number of bytes: 32530; Total size: 41141","context":{},"level":200,"level_name":"INFO","channel":"http_client","datetime":"2024-09-30T20:36:28.405494+02:00","extra":{}}
phpfpm-1  | 2024-09-30T18:36:28.405782737Z {"message":"Number of bytes: 32530; Total size: 41141","context":{},"level":200,"level_name":"INFO","channel":"http_client","datetime":"2024-09-30T20:36:28.405654+02:00","extra":{}}
phpfpm-1  | 2024-09-30T18:36:28.406466899Z {"message":"Number of bytes: 32530; Total size: 41141","context":{},"level":200,"level_name":"INFO","channel":"http_client","datetime":"2024-09-30T20:36:28.406292+02:00","extra":{}}
phpfpm-1  | 2024-09-30T18:36:28.411526556Z {"message":"Number of bytes: 41141; Total size: 41141","context":{},"level":200,"level_name":"INFO","channel":"http_client","datetime":"2024-09-30T20:36:28.411425+02:00","extra":{}}
phpfpm-1  | 2024-09-30T18:36:28.411763890Z {"message":"Number of bytes: 41141; Total size: 41141","context":{},"level":200,"level_name":"INFO","channel":"http_client","datetime":"2024-09-30T20:36:28.411602+02:00","extra":{}}
phpfpm-1  | 2024-09-30T18:36:28.411966620Z {"message":"Number of bytes: 41141; Total size: 41141","context":{},"level":200,"level_name":"INFO","channel":"http_client","datetime":"2024-09-30T20:36:28.411785+02:00","extra":{}}

[tacman]

Thanks you for this detailed response. I still need to wrap my head around this, my understanding of how all these fit together is still evolving. In particular, I think the custom factory is the piece I was missing. Regardless, with this complete example I should be able to figure out how to integrate it with my bundle. Again, muchas gracias!

[tacman]

Thanks. Indeed, today it's working better. I think bunny itself may have been having a problem.

In the debugging process though, it got me wondering about the zone name, path, filename separation. For my bundle, I'm refactoring it so that it's more natural to put the path and filename into one.

bin/console bunny:download data/myfile.zip
# downloads to data/myfile.zip, default zone is configured in the environment, along with the API keys
bin/console bunny:download storage/myfile.sav data/myfile.zip --zone=backups

Since your library is a wrapper to the bunny API, which requires the path and filename to be separate, I thought I'd ask if you already had utilities (or an interest in having them) that splits a path into the components.

Someday I'd like to support URL to bunny, e.g.

bin/console bunny:upload https://example.com/data.zip example.com/ --zone=websites

[ToshY]

Since your library is a wrapper to the bunny API, which requires the path and filename to be separate, I thought I'd ask if you already had utilities (or an interest in having them) that splits a path into the components.

Could you elaborate on this? Are you looking for a utility/snippet to split <directory>/<file> into <directory> and <file> parts?

[tacman]

yes, exactly. My code is functional now but not that elegant. I previous passed the target filename and path as separate parameters, now they're combined, so I can say

 bin/console bunny:download data/x.zip 
 bin/console bunny:upload data/x.zip 

if there's no second argument, the command assumes you want to mirror the directory structure. The zone is passed separately (and can be set), since zones all have their own passwords. And it makes it easier for different projects to share the one bunny account, all with separate zones.

https://github.com/survos/SurvosBunnyBundle/blob/main/src/Command/BunnyDownloadCommand.php#L43-L60

I wish the underlying bunny API allowed passing a filename with a path attached, that seems pretty natural.