[🧹 CHORE]: The pool.allocate_timeout parameter serves two functions

[GityaMan]

No duplicates 🥲.

• I have searched for a similar issue.

What should be improved or cleaned up?

Now pool.allocate_timeout is responsible for the timeout of creating a new worker. The worker has a long initialization of several seconds.
worker_watcher/worker_watcher.go::allocate

The parameter is also a timeout before adding workers for a dynamic pool.
pool/static_pool/pool.go::exec (line 196)

I tried version 2024.3.2 and saw a performance drop of several times compared to the static pool.
The dynamic pool waits for an unacceptably large queue of requests before creating a new worker.

ab -n 5000 -c5 http://localhost:81

    num_workers: 6
    allocate_timeout: 1s
    destroy_timeout: 10s

Requests per second:    322.10 [#/sec] (mean)
Percentage of the requests served within a certain time (ms)
  50%     15
  66%     16
  75%     17
  80%     18
  90%     20
  95%     20
  98%     25
  99%     29
 100%     31 (longest request)

static pool

    num_workers: 1
    allocate_timeout: 1s
    destroy_timeout: 10s
    dynamic_allocator:
      max_workers: 6
      spawn_rate: 1
      idle_timeout: 30s

Requests per second:    74.32 [#/sec] (mean)
Percentage of the requests served within a certain time (ms)
  50%     67
  66%     68
  75%     69
  80%     70
  90%     74
  95%     77
  98%     80
  99%     82
 100%     86 (longest request)

dynamic pool

Increasing allocate_timeout only makes things worse.

I suggest adding the parameter "increase_threshold_timeout" (the name is not important to me) to separate the behavior in two cases.

[rustatian]

Hey @GityaMan 👋🏻
Added to a dyn allocator improvements list.