feat: add safe repr implementations for watsonx and litellm backends#1359
feat: add safe repr implementations for watsonx and litellm backends#1359cptnm3 wants to merge 2 commits into
Conversation
Added __repr__ and __str__ to show useful information with masked api_key Closes generative-computing#1317 Signed-off-by: Vishal V <VishalV@ibm.com>
|
@cptnm3 thanks for the PR! If you don't mind, could you leave a comment on the issue so we can assign it to you? |
|
Done @psschwei . Sorry, in the contribution guidelines I did not see that I have to be assigned to an issue before working on it. Will keep in mind next time. Thanks! |
|
no worries! |
AngeloDanducci
left a comment
There was a problem hiding this comment.
Thanks for the contribution, @cptnm3! The implementation looks good overall and follows the existing OpenAIBackend pattern nicely.
I've left a few inline comments with some requested changes — mostly moving and mocking some of the tests, and a couple of small test improvements. Once those are addressed this should be good to go.
| def __repr__(self) -> str: | ||
| """Return a useful string representation for debugging.""" | ||
| return ( | ||
| f"{self.__class__.__name__}(" | ||
| f"model_id={self._model_id!r}, " | ||
| f"base_url={self._base_url!r})" | ||
| ) |
There was a problem hiding this comment.
| def __repr__(self) -> str: | |
| """Return a useful string representation for debugging.""" | |
| return ( | |
| f"{self.__class__.__name__}(" | |
| f"model_id={self._model_id!r}, " | |
| f"base_url={self._base_url!r})" | |
| ) | |
| def __repr__(self) -> str: | |
| """Return a useful string representation for debugging.""" | |
| base_url_repr = self._base_url if self._explicit_base_url else None | |
| return ( | |
| f"{self.__class__.__name__}(" | |
| f"model_id={self._model_id!r}, " | |
| f"base_url={base_url_repr!r})" | |
| ) |
self._base_url always holds the fallback value http://localhost:11434/v1 when base_url is not passed by the caller.
Recommend adding this to catch that case.
There was a problem hiding this comment.
If you do the above also recommend adding another test case to verify
def test_repr_no_base_url_shows_none():
backend = LiteLLMBackend(model_id="ollama_chat/test-model")
r = repr(backend)
assert "base_url=None" in r
assert "http://localhost:11434" not in r
| session.reset() | ||
|
|
||
|
|
||
| def test_repr_shows_model_id_and_base_url(): |
There was a problem hiding this comment.
Recommend adding a test_str_* requivalent.
def test_str_shows_model_id_and_base_url():
backend = LiteLLMBackend(
model_id="ollama_chat/test-model", base_url="http://localhost:11434/v1"
)
assert "ollama_chat/test-model" in str(backend)
assert "object at 0x" not in str(backend)
or similar.
| from mellea.stdlib.context import ChatContext, SimpleContext | ||
|
|
||
|
|
||
| def test_repr_masks_api_key(): |
There was a problem hiding this comment.
Recommend moving test_repr_masks_api_key, test_repr_no_key_shows_none, and test_str_masks_api_key to a separate file (e.g. test/backends/test_watsonx_repr.py) that uses pytest.importorskip only for the import itself at the top but carries no require_api_key or e2e marker — making them plain unit tests that run by default.
Same with the test_litellm_ollama.py addition.
There was a problem hiding this comment.
Moved to mellea/backends/test_litellm_repr.py and test/backends/test_watsonx_repr.py
| assert "test-project" not in r | ||
|
|
||
|
|
||
| def test_repr_no_key_shows_none(monkeypatch: pytest.MonkeyPatch): |
There was a problem hiding this comment.
Inverted assertions here — the test passes api_key=None with WATSONX_API_KEY set in the environment. The constructor resolves None → "env-key" before storing it (watsonx.py lines 132–133), so self._api_key is "env-key" and repr shows '***'. The assertions assert "***" not in r and assert "_api_key=None" in r are both inverted and will always fail.
There was a problem hiding this comment.
Corrected this. Please take a look
| assert "test-project" not in r | ||
|
|
||
|
|
||
| def test_str_masks_api_key(): |
There was a problem hiding this comment.
When moving these tests over, recommend using a helper function
def clear_watsonx_env(monkeypatch: pytest.MonkeyPatch):
monkeypatch.delenv("WATSONX_API_KEY", raising=False)
monkeypatch.delenv("WATSONX_URL", raising=False)
monkeypatch.delenv("WATSONX_PROJECT_ID", raising=False)
and then also using monkeypatch for some of the setup
def test_str_masks_api_key(clear_watsonx_env):
with patch("mellea.backends.watsonx.APIClient"), patch(
"mellea.backends.watsonx.ModelInference"
):
This avoids the issue of the init validating the url with fake credentials before the assertions.
| assert "***" in str(backend) | ||
| assert "test-project" not in str(backend) | ||
|
|
||
|
|
There was a problem hiding this comment.
Recommend adding an additional test test_repr_env_key_is_masked or similar.
Watsonx repr tests moved to test_watsonx_repr.py — plain unit tests, no e2e/require_api_key markers. Fix inverted assertions in test_repr_no_key_shows_none; add test_repr_env_key_is_masked for env-resolved key path. Use _make_backend helper + mock Credentials to avoid real SDK init. LiteLLM repr test moved to test_litellm_repr.py. Add test_repr_omits_default_base_url and test_str_shows_model_id_and_base_url. Fix LiteLLMBackend.__repr__ to show base_url=None when caller omitted it. Signed-off-by: Vishal V <VishalV@ibm.com>
|
Hi @AngeloDanducci. thanks for the review. I have made the suggested changes. Please take a look and let me know in case I have missed anything else |
Pull Request
Issue
Fixes #1317
Description
Added
__repr__and__str__to show useful information with masked api_keyTesting
Attribution
Adding a new component, requirement, sampling strategy, or tool?
If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.
NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.