Fallback to Set Thread Num by Cgroups when Users Did Not Do So

[PwzXxm]

issue: #187
/kind enhancement

Read system cgroups to find out the CPU quota and period to get the correct limited CPU num under the K8s environment when running Knowhere alone, i.e., without Milvus.

For instance, if an instance has 64 CPUs, k8s limits it to 16c,

• Before the change, Knowhere initializes the thread pool to 64 and uses 25% of each CPU.
• After the change, Knowhere initializes the thread pool to 16 and uses 100% of each CPU.

If it fails to read from groups, fallback hardware_concurrency().
For systems other than Linux, fallback as well.

Support cgroups V1 only for now. Maybe support V2 later.

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: PwzXxm

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [PwzXxm]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[xiaofan-luan]

The easiest way is simply passed in from golang?

[xiaofan-luan]

as a library, I think it make sense to receive a params of concurrency?
If user don't pass in then we fallback to current logic

[PwzXxm]

The easiest way is simply passed in from golang?

Yes, if you run Milvus + Knowhere, Milvus will use a hook to pass it to Knowhere. This PR fixs when Knowhere is running alone, e.g., VectorDBBench now supports benchmark Knowhere/Cardinal alone and we encountered this issue.

[PwzXxm]

as a library, I think it make sense to receive a params of concurrency? If user don't pass in then we fallback to current logic

Right. This PR will not change this logic. Use the Milvus passed params first; if not received anything, fallback to read from cgroups; fails to read from cgroups, fallback to system thread num.

[zhengbuqian]

LGTM overall but need more comments to explain how it works so the code reader don't need to google.

[liliu-z]

Why we cannot simply call the init method to set the number from lib caller side?

[PwzXxm]

Why we cannot simply call the init method to set the number from lib caller side?

This requires an additional pyknowhere interface and a manual setup every time. Then the question is, should Knowhere use ALL of the CPUs BY DEFAULT even if it was limited to a smaller number of them? Or should Knowhere care about the environment at all when the user didn't set any value?

[alexanderguzhva]

Just to confirm. What is going to happen if I run under taskset or numactl?

[zhengbuqian]

Just to confirm. What is going to happen if I run under taskset or numactl?

taskset or numactl doesn't rely on cgroups on controlling the cpu usage of processes, they do that by setting the process's CPU affinity mask using sched_setaffinity syscall. So this won't detect the limitation set by taskset or numactl.

[PwzXxm]

Just to confirm. What is going to happen if I run under taskset or numactl?

Good question. I just tested both.

Buqian is correct that they do not interact with each other. So for this PR, you can view it as an additional shield protection under k8s env. If you run taskset or numactl and bind to a single CPU, under an 8c K8s pod, on an instance with 64 CPUs:

• Before the change, you run the process on a single CPU with 64 threads.
• After this change, you run the process on a single CPU with 8 threads.

If it is not under K8s env, no limit on cgroups, and there is no change after this PR, it will create 64 threads.

In this case, it might be better to call the setters or we might create a separate PR for this.
This is an enchantment for the Knowhere default behavior and the initiative is to be less error-prone on testing.