Improve default llm retry logic to be more optimized

Improve default llm retry logic to be more optimized #3567