Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 25 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,13 @@ The customer-facing CLI for the tracebloc declarative ingestion path. Wraps the

## Status

🚧 **Pre-alpha** — Phase 0 of the [v0.1 roadmap](https://github.com/tracebloc/client/issues/147). Today this binary implements only `version` and `completion`. Subsequent phases (#149–#153) land schema validation, cluster discovery, data staging, submission, and distribution.
**v0.1 — feature-complete on `develop`; first release pending.** Every phase of the [v0.1 roadmap](https://github.com/tracebloc/client/issues/147) (#148–#153) is merged. The binary implements `version`, `completion`, `ingest validate`, `cluster info`, and the full `dataset push` flow — local schema validation, cluster discovery, data staging, submission, and Job watching, end to end.

For the production ingestion flow today, use the Helm chart: `helm install tracebloc/ingestor --set-file ingestConfig=./ingest.yaml`. See the chart's [README](https://github.com/tracebloc/client/blob/develop/ingestor/README.md) for the full customer journey including the data-staging recipe.
`dataset push` covers **9 of 10 task categories**: `image_classification`, `object_detection`, `keypoint_detection`, `text_classification`, `masked_language_modeling`, `tabular_classification`, `tabular_regression`, `time_series_forecasting`, and `time_to_event_prediction`. `semantic_segmentation` is pending mask-sidecar support upstream ([data-ingestors#136](https://github.com/tracebloc/data-ingestors/issues/136)); `instance_segmentation` is not yet implemented.

No tagged release exists yet — [build from source](#building-from-source) for now. The release pipeline (GitHub Release + Homebrew tap + install scripts) is wired and fires on the first `v*` tag.

The Helm chart remains a sibling interface for the Kubernetes-native workflow: `helm install tracebloc/ingestor --set-file ingestConfig=./ingest.yaml` (see the chart's [README](https://github.com/tracebloc/client/blob/develop/ingestor/README.md)).

## Why a CLI in addition to the chart?

Expand Down Expand Up @@ -37,10 +41,12 @@ Customer interfaces (pick one or many):

The protocol — the v1 schema + the POST endpoint — is the stable point. Everything above is interchangeable.

## v0.1 design (target customer experience, once Phase 5 ships)
## Customer experience

> The install commands below light up once `v0.1.0` is tagged and the release pipeline publishes the artifacts. Until then, [build from source](#building-from-source).

```bash
# Install once
# Install once (after v0.1.0 is released)
brew install tracebloc/tap/tracebloc # macOS
curl -fsSL https://install.tracebloc.io | sh # Linux/macOS
irm https://install.tracebloc.io/install.ps1 | iex # Windows
Expand Down Expand Up @@ -73,7 +79,7 @@ go build -o tracebloc ./cmd/tracebloc
./tracebloc version
```

Requires Go 1.22 or newer. The binary self-reports its build metadata; a `go build` without `-ldflags` reports `dev / unknown / unknown` for version/git-sha/build-date so support can tell a local hack apart from a release build.
Requires Go 1.26 or newer (the `k8s.io/*` dependencies set the floor; see `go.mod`). The binary self-reports its build metadata; a `go build` without `-ldflags` reports `dev / unknown / unknown` for version/git-sha/build-date so support can tell a local hack apart from a release build.

```bash
# Release-style build with version metadata
Expand All @@ -86,14 +92,20 @@ go build -ldflags "\

## Roadmap

| Phase | Ticket | What |
|---|---|---|
| **0 (this PR)** | [#148](https://github.com/tracebloc/client/issues/148) | Repo bootstrap + Go module + CI + `tracebloc version` |
| 1 | [#149](https://github.com/tracebloc/client/issues/149) | Embed `ingest.v1.json` + `tracebloc ingest validate <path>` (local-only) |
| 2 | [#150](https://github.com/tracebloc/client/issues/150) | Cluster discovery + ingestor SA token via TokenRequest |
| 3 | [#151](https://github.com/tracebloc/client/issues/151) | Stage data into the shared PVC via ephemeral Pod |
| 4 | [#152](https://github.com/tracebloc/client/issues/152) | Submit to jobs-manager + watch ingestor Job + summary |
| 5 | [#153](https://github.com/tracebloc/client/issues/153) | GitHub Releases, Homebrew tap, install.sh distribution |
All v0.1 phases are merged:

| Phase | Ticket | What | Status |
|---|---|---|---|
| 0 | [#148](https://github.com/tracebloc/client/issues/148) | Repo bootstrap + Go module + CI + `tracebloc version` | ✅ |
| 1 | [#149](https://github.com/tracebloc/client/issues/149) | Embed `ingest.v1.json` + `tracebloc ingest validate <path>` (local-only) | ✅ |
| 2 | [#150](https://github.com/tracebloc/client/issues/150) | Cluster discovery + ingestor SA token via TokenRequest | ✅ |
| 3 | [#151](https://github.com/tracebloc/client/issues/151) | Stage data into the shared PVC via ephemeral Pod | ✅ |
| 4 | [#152](https://github.com/tracebloc/client/issues/152) | Submit to jobs-manager + watch ingestor Job + summary | ✅ |
| 5 | [#153](https://github.com/tracebloc/client/issues/153) | GitHub Releases, Homebrew tap, install.sh distribution | ✅ (awaiting first `v*` tag) |

Beyond the original phases, `dataset push` was widened from image-classification-only to 9 of 10 modalities, and the test suite gained unit-coverage wins plus a kind-based integration harness for the real-I/O seams.

**Next (v0.2):** cloud-source ingestion (S3/GCS/HTTPS) for datasets above the 1 GiB local cap; `semantic_segmentation` ([data-ingestors#136](https://github.com/tracebloc/data-ingestors/issues/136)) and `instance_segmentation`; more `dataset` verbs (`list`, `rm`). Smaller follow-ups are tracked as [open issues](https://github.com/tracebloc/cli/issues) (#3–#7).

Epic: [tracebloc/client#147](https://github.com/tracebloc/client/issues/147).

Expand Down
2 changes: 1 addition & 1 deletion internal/cli/cluster.go
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,6 @@ func runClusterInfo(
_, _ = fmt.Fprintf(out, " expires in: never (static-secret fallback)\n")
}
_, _ = fmt.Fprintln(out)
_, _ = fmt.Fprintln(out, "Ready for `tracebloc dataset push` (coming in Phase 3).")
_, _ = fmt.Fprintln(out, "Ready for `tracebloc dataset push`.")
return nil
}
16 changes: 9 additions & 7 deletions internal/cli/dataset.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,9 @@ func newDatasetCmd() *cobra.Command {
Long: `Commands for staging and managing datasets on the cluster's
shared PVC.

Today: ` + "`dataset push`" + ` stages a local directory to the cluster's
shared PVC. Submission to jobs-manager (so the ingestor Job actually
runs) lands in Phase 4 (` + "`tracebloc/client#152`" + `); for now the staged
files are picked up by the existing helm ` + "`tracebloc/ingestor`" + ` flow.
` + "`dataset push`" + ` stages a local dataset to the cluster's shared
PVC, submits the ingestion run to jobs-manager, and watches the
ingestor Job to completion (streaming logs + the final summary).

` + "`tracebloc cluster info`" + ` is the pre-flight you'd typically run
before the first push.`,
Expand Down Expand Up @@ -112,10 +111,13 @@ func newDatasetPushCmd() *cobra.Command {
cmd := &cobra.Command{
Use: "push <local-path>",
Short: "Stage a local dataset to the cluster's shared PVC",
Long: `Stages a local image_classification dataset to the parent client
release's shared PVC, then (in PR-b) submits an ingestion run.
Long: `Stages a local dataset to the parent client release's shared PVC,
submits an ingestion run to jobs-manager, and watches the ingestor Job
to completion. Supports 9 task categories (image classification,
object/keypoint detection, text classification, masked language
modeling, and the tabular / time-series family); pick one with --category.

Expected local layout:
Expected local layout (image_classification shown):

<local-path>/
labels.csv (required)
Expand Down
Loading