Proper way to stop nsys profiling for SGLang serving

[chenzhengda]

When using nsys profile to profile SGLang serving, I often encounter issues where the profile data is not properly saved when terminating the server with Ctrl+C. According to the documentation https://docs.sglang.ai/references/benchmark_and_profiling.html#profile-with-nsight , the recommended command for profiling a server is:

nsys profile --trace-fork-before-exec=true --cuda-graph-trace=node -o sglang.out --delay 60 --duration 70 python3 -m sglang.launch_server --model-path meta-llama/Llama-3.1-8B-Instruct --disable-radix-cache

Could you please clarify:
What is the recommended way to stop the profiling process?

[jhinpan]

cc @Fridge003 . Could you help take a look? Thx!

[Fridge003]

nsys profile --trace-fork-before-exec=true --cuda-graph-trace=node -o sglang.out --delay 60 --duration 70 python3 -m sglang.launch_server --model-path meta-llama/Llama-3.1-8B-Instruct --disable-radix-cache

Hi @chenzhengda, by setting the --delay and --duration parameters properly, the nsys will automatically stop the profiling. --delay marks the starting time of profiling in seconds, and --duration marks the time for profiling, which should be a value greater than model running time.

This feature is also discussed in #3049.