Skip to content

ArizeClient().ml.log fails to upload dataframe with a pyarrow warning #128

@mareklabonarski

Description

@mareklabonarski

When using ArizeClient().ml.log to upload a pandas dataframe to Arize AX unsing v8.30.1, the response is 200, but nothing is visible in Arize AX ML Models tab.
The old v7 version of SDK uploaded the same pandas dataframe sucessfully and model was created in Arize AX ML Models tab.

Old vs new method call:

# old
client = arize.pandas.logger.Client(
    space_id=space_id, space_key=space_key, api_key=api_key, uri=uri
)
res = client.log(
    dataframe=data_to_send,
    model_id=model_id,
    model_version=model_version,
    model_type=model_types_mapper[self.model_type],
    environment=environment,
    schema=schema,
    surrogate_explainability=surrogate_explainability,
    batch_id=batch_id
)

# new
client = arize.ArizeClient(
    api_key=api_key, api_host=HttpUrl(self.uri).host, api_scheme=HttpUrl(self.uri).scheme
    # same with single_host and single_port
)
client.ml.log(
    space_id=space_id,
    model_name=model_id,
    model_version=model_version,
    dataframe=data_to_send,
    model_type=model_types_mapper[self.model_type],
    environment=environment,
    schema=schema,
    surrogate_explainability=surrogate_explainability,
    batch_id=batch_id,
)

Logs from the upload attempt:

  arize._lazy | DEBUG | Lazily loading subclient 'ml'
  arize.ml.client | DEBUG | Performing required validation.
  arize.ml.client | DEBUG | Performing parameters validation.
  arize.ml.client | DEBUG | Removing unnecessary columns.
  arize.ml.client | DEBUG | Converting data to Arrow format
  arize.ml.client | DEBUG | Performing types validation.
  arize.ml.client | DEBUG | Performing values validation.
  arize.ml.batch_validation.validator | WARNING | Caution when sending predictions with future timestamps.Arize only stores 2 years worth of data. For example, if you sent predictions to Arize from 1.5 years ago, and now send predictions with timestamps of a year in the future, the oldest 0.5 years will be dropped to maintain the 2 years worth of data requirement.
  arize.utils.arrow | DEBUG | Preparing to log Arrow table via file upload | rows=40 cols=12
  arize.utils.arrow | DEBUG | Serializing schema
  arize.utils.arrow | DEBUG | Writing table to temporary file: /tmp/tmp8dcwg6fo/arize-qdv1_5xy.arrow
  arize.utils.arrow | DEBUG | Uploading file to Arize | path='/tmp/tmp8dcwg6fo/arize-qdv1_5xy.arrow' size_bytes=9624
  arize.utils.arrow | WARNING | Failed to get project URL: Expecting value: line 1 column 1 (char 0)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions