[Bug] The script "profile_generation.py" went haywire and crashed

[yuchiwang]

Checklist

• 1. I have searched related issues but cannot get the expected help.
• 2. The bug has not been fixed in the latest version.
• 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

Describe the bug

if set c(concurrency) large, crashed, maybe kv_cache is not enough, but there is no clear indication.
Run script as below:

Traceback:

Reproduction

python benchmark/profile_generation.py /data/models/Qwen2.5-14B-Instruct --backend pytorch -pt 2048 -ct 512 -c 128 --tp 1

Environment

maca C500 GPU.

Error traceback

No response

[yuchiwang]

应该是kv_cache不够导致，如果是kv_cache不够最好能给一个明确提示，这样比较用户友好。 @Reinerzhou