Repository navigation
fix(litellm): log and back off on non-filter BadRequestError - #1472
Open
pcbeingused333 wants to merge 1 commit into
Open
pcbeingused333 wants to merge 1 commit into
pcbeingused333 wants to merge 1 commit into
Conversation
The BadRequestError handler only acted on the Azure content-filter message; any other BadRequestError (e.g. an unsupported parameter) was swallowed silently with no log and no backoff, so each prompt hammered the API max_retry times and ended with an empty response and no trace of the cause. Route those errors through the generic handler so they are logged and backed off like every other API failure. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix silently swallowed
BadRequestErrorin the LiteLLM endpoint retry loop.The
LiteLLMClientretry loop had a dedicatedexcept litellm.BadRequestErrorbranch that only acted when the message was the Azure content-filter one. Any otherBadRequestError(for example an unsupported parameter) was swallowed: no log, no backoff, and because the laterexcept Exceptionis a sibling clause it never ran. Each prompt therefore hit the APIapi_max_retrytimes back to back and ended with an emptyLitellmModelResponse, with nothing in the logs about the cause.The fix folds the content-filter check into the generic handler, so the filter case still returns an empty response immediately and every other error is logged and backed off like any other API failure.
Tests:
tests/unit/models/endpoints/test_litellm_model.pymockslitellm.completionand checks that a non-filterBadRequestErroris logged and backed off for every attempt, and that the content-filter case still returns after a single call.This pull request was prepared with the assistance of an AI coding agent (Claude Code) and submitted by Alex Castillo, who is responsible for its content.