Phase 8: Logging & Documentation - Complete

Logging Implementation:
- Add logging.getLogger(__name__) to all core modules (currency, sgs, odata/framework)
- Log HTTP requests with debug level: URL, params (no secrets), status, response length
- Log retry attempts with warning level for connection errors
- Log date rollback attempts in currency module

Example Scripts:
- examples/sgs_time_series.py - Basic SGS usage, multiple series, text output
- examples/currency_exchange.py - USD/EUR rates, bid/ask filtering, cache management
- examples/odata_query.py - OData filtering, sorting, column selection with examples
- examples/async_usage.py - Async API usage, concurrent operations with asyncio

Documentation Updates:
- Add "Which Module Should I Use?" decision table to README
  - Guides users to right module based on use case
  - Compares SGS vs PTAX currency, OData services
  - Shows async API option
- Add "Quick Start" section with code examples for each module
- Add comprehensive FAQ covering:
  - Module differences and data coverage
  - Historical data ranges
  - Async API usage
  - Error handling and specific exceptions
  - Logging setup for debugging
  - Caching behavior and management
  - Long-running application considerations
  - Contributing and documentation resources

Definition of Done:
- ✅ 103 unit tests pass (67 original + 12 currency negative + 15 SGS negative + 9 async)
- ✅ ruff check: All checks passed
- ✅ ruff format: All files properly formatted
- ✅ mypy: No type errors (strict mode)

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

Author: wilsonfreitas

README.md

@@ -59,3 +59,134 @@ macroeconômicas fornecidos por um conjuto de instituições do mercado
 financeiro.
 A classe `bcb.Expectativas` implementa essa interface no
 padrão OData.
+
+# Which Module Should I Use?
+
+Use this table to choose the right module for your use case:
+
+| Use Case | Module | Key Features |
+|----------|--------|--------------|
+| Daily time series (inflation, interest rates) | `bcb.sgs` | Largest historical dataset, granular frequency control, multiple pre-defined series |
+| Daily foreign exchange rates (PTAX) | `bcb.currency` | Bid/ask spreads, quick implementation, daily frequency |
+| Market expectations (FOCUS survey) | `bcb.odata` (Expectativas) | Forward-looking economic indicators, consensus forecasts |
+| Interest rates (various types) | `bcb.odata` (TaxaJuros) | Detailed rate curves, real estate lending rates |
+| Real estate financing data | `bcb.odata` (MercadoImobiliario) | Mortgage originations, average rates, volumes |
+| Financial institution information | `bcb.odata` (IFDATA) | Bank balance sheet data, regulatory information |
+| Advanced data analysis with filters | `bcb.odata` (any service) | Chainable API, SQL-like filtering, sorting, selection |
+| Concurrent data fetching | Any module with `async_get()` | Non-blocking requests, improved performance for bulk operations |
+
+# Quick Start
+
+## Time Series with SGS
+
+```python
+from bcb import sgs
+
+# Fetch SELIC rate (code 1)
+df = sgs.get(1, start="2023-01-01", end="2024-12-31")
+```
+
+## Exchange Rates with Currency
+
+```python
+from bcb import currency
+
+# Fetch USD bid/ask prices
+usd = currency.get("USD", start="2023-01-01", end="2024-12-31")
+```
+
+## Market Expectations with OData
+
+```python
+from bcb import Expectativas
+
+api = Expectativas()
+endpoint = api.get_endpoint("ExpectativasMercadoAnuais")
+
+# Get IPCA forecasts
+df = endpoint.query().filter(endpoint.Indicador == "IPCA").limit(100).collect()
+```
+
+# FAQ
+
+## Q: What's the difference between SGS and PTAX currency data?
+**A:** SGS contains mostly economic indicators. For currency exchange rates, use `bcb.currency` (PTAX data) for daily rates or `bcb.odata` PTAX service for detailed institutional data. The currency module is simpler for common use cases.
+
+## Q: How far back does historical data go?
+**A:** It varies by series:
+- SGS: Most series go back to 1980s or 1990s (check specific code documentation)
+- Currency: Daily rates available from approximately 1980
+- OData services: Varies; check BCB documentation for specific endpoints
+
+## Q: Can I fetch data asynchronously?
+**A:** Yes! All modules have `async_get()` or similar async methods. Use them for concurrent requests:
+```python
+import asyncio
+from bcb import sgs
+
+async def main():
+    results = await asyncio.gather(
+        sgs.async_get(1),  # SELIC
+        sgs.async_get(433),  # IPCA
+    )
+
+asyncio.run(main())
+```
+
+## Q: How do I handle errors/missing data?
+**A:** The library raises specific exceptions:
+- `CurrencyNotFoundError`: Currency symbol not found
+- `SGSError`: SGS service error
+- `BCBRateLimitError`: Rate limit exceeded (HTTP 429)
+- `BCBAPIError`: Other API errors
+
+```python
+from bcb import sgs
+from bcb.exceptions import SGSError, BCBRateLimitError
+
+try:
+    df = sgs.get(99999)  # Invalid code
+except SGSError as e:
+    print(f"Data error: {e}")
+except BCBRateLimitError:
+    print("Rate limited - please try again later")
+```
+
+## Q: How do I enable logging to debug requests?
+**A:** The library uses Python's standard logging module:
+```python
+import logging
+
+logging.basicConfig(level=logging.DEBUG)
+logger = logging.getLogger("bcb")
+logger.setLevel(logging.DEBUG)
+
+# Now all HTTP requests/responses will be logged
+```
+
+## Q: Is there caching to avoid redundant requests?
+**A:** Yes:
+- `bcb.currency`: Automatic in-memory cache of currency lists
+- `bcb.odata`: OData metadata cached per service URL
+- Call `currency.clear_cache()` to reset if data changes
+
+## Q: Can I use this in a long-running application?
+**A:** Yes, but be mindful of:
+- Rate limits: BCB APIs may have limits; implement backoff if needed
+- Caching: Currency cache persists in memory; clear it if data updates matter
+- Connection pooling: Uses httpx with connection pooling by default
+- Async API: Use async methods for truly non-blocking behavior
+
+## Q: How do I contribute or report issues?
+**A:** Visit the [GitHub repository](https://github.com/wilsonfreitas/python-bcb) to:
+- Report bugs
+- Request features
+- Submit pull requests
+- View documentation
+
+## Q: Where can I find more detailed documentation?
+**A:**
+- [API Documentation](https://bcb-python.readthedocs.io/)
+- [Examples Directory](./examples/)
+- Inline docstrings: `help(bcb.sgs.get)`, `help(bcb.currency.get)`, etc.
+- [BCB Open Data Portal](https://dadosabertos.bcb.gov.br/)

bcb/currency.py

@@ -1,6 +1,7 @@
 from __future__ import annotations
 
 import asyncio
+import logging
 import re
 import threading
 from datetime import date, timedelta
@@ -24,6 +25,8 @@
 if TYPE_CHECKING:
     import httpx
 
+logger = logging.getLogger(__name__)
+
 """
 O módulo :py:mod:`bcb.currency` tem como objetivo fazer consultas no site do conversor de moedas do BCB.
 """
@@ -161,7 +164,11 @@ def _currency_id_list(
         "https://ptax.bcb.gov.br/ptax_internet/consultaBoletim.do?"
         "method=exibeFormularioConsultaBoletim"
     )
+    logger.debug(f"Fetching currency ID list from {url1}")
     res = _CLIENT.get(url1)
+    logger.debug(
+        f"Currency ID list response: status={res.status_code}, length={len(res.content)}"
+    )
     if res.status_code == 429:
         raise BCBRateLimitError(
             "BCB API rate limit exceeded. Please try again later.",
@@ -228,18 +235,28 @@ def _get_valid_currency_list(
         )
 
     url2 = f"https://www4.bcb.gov.br/Download/fechamento/M{_date:%Y%m%d}.csv"
+    logger.debug(f"Fetching currency list from {url2}")
     try:
         res = _CLIENT.get(url2)
     except Exception as ex:
         # Connection error: retry same date up to 3 times
         if n >= 3:
             raise ex
+        logger.warning(
+            f"Connection error fetching {url2}, retrying (attempt {n + 1}/3)"
+        )
         return _get_valid_currency_list(_date, n + 1, max_rollback)
 
+    logger.debug(
+        f"Currency list response: status={res.status_code}, length={len(res.content)}"
+    )
     if res.status_code == 200:
         return res
     else:
         # Non-200 response (file not found for date): roll back to previous day
+        logger.debug(
+            f"Currency list not found for {_date}, rolling back to previous day"
+        )
         return _get_valid_currency_list(_date - timedelta(1), 0, max_rollback)
 
 
@@ -329,7 +346,11 @@ def _fetch_symbol_response(
     """
     cid = _get_currency_id(symbol)  # Raises CurrencyNotFoundError if not found
     url = _currency_url(cid, start_date, end_date)
+    logger.debug(f"Fetching currency data for {symbol} from {url.split('?')[0]}")
     res = _CLIENT.get(url)
+    logger.debug(
+        f"Currency data response: status={res.status_code}, length={len(res.content)}"
+    )
 
     # Handle HTML error response (e.g., no data for date range)
     if res.headers["Content-Type"].startswith("text/html"):

bcb/odata/framework.py

@@ -1,5 +1,6 @@
 from __future__ import annotations
 
+import logging
 import threading
 from io import BytesIO
 from typing import Any, Optional, Union
@@ -12,6 +13,8 @@
 from bcb.http import _CLIENT, _ASYNC_CLIENT
 from bcb.exceptions import ODataError
 
+logger = logging.getLogger(__name__)
+
 # Module-level metadata cache for OData services
 # Maps service URL → ODataMetadata instance
 _METADATA_CACHE: dict[str, "ODataMetadata"] = {}
@@ -279,7 +282,11 @@ def __init__(self, url: str) -> None:
         self._parse_function_imports(schema)
 
     def _load_document(self) -> None:
+        logger.debug(f"Fetching OData metadata from {self.url}")
         res = _CLIENT.get(self.url)
+        logger.debug(
+            f"OData metadata response: status={res.status_code}, length={len(res.content)}"
+        )
         self.doc = etree.parse(BytesIO(res.content))
 
     def _parse_entity(self, entity_element: Any, namespace: str) -> ODataEntity:
@@ -551,7 +558,12 @@ def text(self) -> str:
                 params["@" + (p.name or "")] = p.format(val)
         qs = "&".join([f"{quote(k)}={quote(str(v))}" for k, v in params.items()])
         headers = {"OData-Version": "4.0", "OData-MaxVersion": "4.0"}
-        res = _CLIENT.get(self.odata_url() + "?" + qs, headers=headers)
+        url = self.odata_url()
+        logger.debug(f"Fetching OData query from {url}")
+        res = _CLIENT.get(url + "?" + qs, headers=headers)
+        logger.debug(
+            f"OData query response: status={res.status_code}, length={len(res.text)}"
+        )
         return res.text
 
     def show(self) -> None:

bcb/sgs/__init__.py

@@ -2,6 +2,7 @@
 
 import asyncio
 import json
+import logging
 from dataclasses import dataclass
 from io import StringIO
 from typing import (
@@ -23,6 +24,8 @@
 from bcb.exceptions import BCBRateLimitError, SGSError
 from bcb.utils import Date, DateInput
 
+logger = logging.getLogger(__name__)
+
 """
 Sistema Gerenciador de Séries Temporais (SGS)
 
@@ -332,7 +335,9 @@ def get_json(
         série temporal univariada em formato JSON.
     """
     url, payload = _get_url_and_payload(code, start, end, last)
+    logger.debug(f"Fetching SGS time series code={code} from {url.split('/dados')[0]}")
     res = _CLIENT.get(url, params=payload)
+    logger.debug(f"SGS response: status={res.status_code}, length={len(res.text)}")
 
     # Check for rate limiting first
     if res.status_code == 429:
@@ -388,7 +393,13 @@ async def async_get_json(
         Se a API retorna um erro
     """
     url, payload = _get_url_and_payload(code, start, end, last)
+    logger.debug(
+        f"Fetching SGS time series (async) code={code} from {url.split('/dados')[0]}"
+    )
     res = await _ASYNC_CLIENT.get(url, params=payload)
+    logger.debug(
+        f"SGS (async) response: status={res.status_code}, length={len(res.text)}"
+    )
 
     # Check for rate limiting first
     if res.status_code == 429:

examples/async_usage.py

@@ -0,0 +1,87 @@
+"""
+Example: Using Async APIs
+
+This example demonstrates how to use the async versions of the APIs
+for concurrent data fetching (useful for fetching multiple series).
+"""
+
+import asyncio
+from datetime import datetime
+from bcb import sgs, currency
+from bcb.odata.api import Expectativas
+
+
+async def fetch_multiple_sgs_series():
+    """Fetch multiple SGS time series concurrently."""
+    print("Example 1: Fetching multiple SGS series concurrently")
+
+    # Fetch SELIC, CDI, and IPCA concurrently
+    codes = [1, 12, 433]  # SELIC, CDI, IPCA
+
+    df = await sgs.async_get(codes, start="2023-01-01", end="2024-12-31", multi=True)
+    print("Concurrent SGS fetch completed")
+    print(df.head())
+    print()
+
+
+async def fetch_multiple_currencies():
+    """Fetch currency rates concurrently."""
+    print("Example 2: Fetching currency rates concurrently")
+
+    # Fetch USD rate (note: you'd need to implement multi-symbol async
+    # for this to truly be concurrent for different symbols)
+    df = await currency.async_get("USD", start="2024-01-01", end="2024-12-31")
+    print("Async currency fetch completed")
+    print(df.head())
+    print()
+
+
+async def fetch_odata_async():
+    """Fetch OData results asynchronously."""
+    print("Example 3: Fetching OData results asynchronously")
+
+    api = Expectativas()
+    endpoint = api.get_endpoint("ExpectativasMercadoAnuais")
+
+    # Build and execute query asynchronously
+    query = endpoint.query().filter(endpoint.Indicador == "IPCA").limit(5)
+    df = await query.async_collect()
+    print("Async OData fetch completed")
+    print(df)
+    print()
+
+
+async def concurrent_operations():
+    """Execute multiple async operations concurrently."""
+    print("Example 4: Multiple concurrent operations")
+
+    # Create tasks for concurrent execution
+    tasks = [
+        sgs.async_get(1, start="2024-01-01", end="2024-12-31"),  # SELIC
+        sgs.async_get(11, start="2024-01-01", end="2024-12-31"),  # CDI
+        sgs.async_get(433, start="2024-01-01", end="2024-12-31"),  # IPCA
+    ]
+
+    # Wait for all tasks to complete
+    results = await asyncio.gather(*tasks)
+    print(f"Fetched {len(results)} concurrent series")
+    print("First series sample:")
+    print(results[0].head())
+    print()
+
+
+async def main():
+    """Run all async examples."""
+    try:
+        await fetch_multiple_sgs_series()
+        await fetch_multiple_currencies()
+        await fetch_odata_async()
+        await concurrent_operations()
+    except Exception as e:
+        print(f"Error: {type(e).__name__}: {e}")
+
+
+if __name__ == "__main__":
+    # Run the async examples
+    # Note: This requires Python 3.7+ with asyncio
+    asyncio.run(main())