refactor: ♻️ download field metadata with pytask#153
Conversation
| env_key="REDC_CPH_API_KEY", url="https://redcap.regionh.dk/api/" | ||
| ) | ||
| Odense = APIConfig(env_key="REDC_ODN_API_KEY", url="https://redcap.sdu.dk/api/") | ||
| Test = APIConfig(env_key="TEST_API_KEY", url="https://redcap.au.dk/api/") |
There was a problem hiding this comment.
Can this live here for testing?
There was a problem hiding this comment.
If we don't want to keep passing the paths around that declare task dependencies, we can move them into a data catalog
| def task_download_field_metadata( | ||
| field_metadata_path: Annotated[Path, Product] = BLD_REDCAP / "field_metadata.json", | ||
| ) -> None: | ||
| """Download field metadata to `BLD_REDCAP`.""" | ||
| download_redcap_metadata(field_metadata_path, "metadata") |
There was a problem hiding this comment.
As we will be downloading other metadata as well, I'm calling this field metadata because it contains metadata about all fields in REDCap.
There was a problem hiding this comment.
Maybe this is a good place to discuss the main issue/question I've run into with pytask so far: download tasks declare that they produce a file or files. If that file is unchanged, they won't rerun automatically when the pipeline is run with uv run pytask. We can use the --force flag on the CLI to get around this. Is that a workable solution or should I look for another way?
There was a problem hiding this comment.
I also put up the changes for pulling out common logic separately as #154 because I wasn't sure how it was most helpful.
1 participant
Description
This PR sets up pytask and downloads the data dict from REDCap. As we will be downloading other metadata as well, I'm calling it field metadata because it contains metadata about all fields in REDCap.
Related to #97
Needs a thorough review.
Checklist
just run-all