Create repositories through an authenticated API

GIT_PLAN.md

@@ -11,9 +11,9 @@ rules remain outside the Rust core.
 - Classic receive-pack push, branches, annotated tags, stale-write rejection,
   fast-forward policy, connectivity validation, and malformed-input rejection.
 - Atomic publication of refs and the sparse object catalog.
-- Arbitrary configured repository paths with isolated WAL identities and an
-  explicit SHA-1 or SHA-256 format. Adding a repository requires configuration,
-  not source changes.
+- Arbitrary repository paths with isolated WAL identities and an explicit
+  SHA-1 or SHA-256 format. Authorized API creation requires no per-repository
+  configuration or restart; named access policies remain optional overrides.
 - Per-repository reader, writer, and administrator permissions; read-only mode;
   a persisted default branch; and recovery without a local repository cache.
 - Automatic maintenance that prunes unreachable Git objects and invokes bounded
@@ -100,9 +100,12 @@ in #39.
 The declarative repository map supplies canonical nested paths, stable WAL
 identities, immutable object formats, default branches and independent
 read/write/admin groups. Duplicate identities, aliases and malformed paths are
-rejected. Unknown repositories fail closed. Reads and administration open
-existing WALs; an authorized first-push discovery can materialize a configured
-repository before accepting a pack.
+rejected. A wildcard policy admits authenticated API creation of new names;
+missing repositories remain absent on reads, administration and wildcard push.
+Explicitly provisioned names also support authorized first-push discovery.
+The format and default branch are published through the existing repository WAL.
+Scheduled maintenance currently requires named entries; dynamic discovery is
+separate work.
 
 Remote tests added a nested SHA-256 repository through configuration while
 preserving the component artifact. Read discovery returned 404 with no durable

HANDOFF.md

@@ -58,9 +58,12 @@ The Git service supports unchanged SHA-1 and SHA-256 clients, protocol-v2 clone
 and have-aware fetch, shallow history, classic push, branches, tags, access
 control, cold recovery, automatic tail checkpoints, and explicit maintenance.
 Refs and its sparse object catalog publish atomically through one WAL commit.
-Local MinIO tests pass with unmodified go-git. Repository paths, formats, default
-branches and per-action groups now come from configuration. Only an authorized
-writer can materialize a configured repository; reads open existing state.
+Local MinIO tests pass with unmodified go-git. Repository creation is available through an authenticated API without per-name
+configuration. A wildcard access policy supplies defaults; exact entries replace
+it. Canonical names derive isolated WAL identities, and the creation publication
+persists the format and default branch. Reads and wildcard pushes only open
+existing Git roots; explicitly configured names also retain first-push provisioning.
+The host maintenance worker still visits only named configuration entries.
 Cognito access-token validation and a separate administration-only machine
 client are wired. Password mode is the explicit local default and requires a
 password. Public EC2 HTTPS qualification has passed real Cognito browser login,

examples/git/README.md

@@ -141,7 +141,7 @@ All values are Spin variables. The defaults target the local MinIO setup above.
 | `wal_collection_candidates` | `1000` | Maximum entries in a new deletion plan, capped by the durable graph limit |
 | `wal_max_collection_objects` | `100000` | Complete publication graph and distinct collection live-object bound; shared paths count separately for publication |
 | `wal_recover_retentions_after_drain` | `false` | Exclusive lost-retention recovery mode |
-| `git_repositories` | two demo entries | JSON map of paths, WAL identities, formats, default branches and permissions |
+| `git_repositories` | default policy and two demo entries | JSON access policies; optional fixed identities, formats and default branches |
 | `git_auth_mode` | `password` | `password`, `cognito`, or explicit local `anonymous` mode |
 | `git_password` | empty | Required password for local password mode |
 | `git_cognito_issuer` | empty | HTTPS Cognito user-pool issuer |
@@ -186,17 +186,43 @@ configured limit high enough to read them; lowering it blocks those objects.
 
 ## Repositories and permissions
 
-Repository names come from configuration, including nested names. For example:
+Create a repository through the API, then use ordinary Git. Names can be nested,
+and the `.git` suffix is optional; both URLs select the same repository.
+
+```sh
+curl --fail-with-body -u git:local-git-password \
+  -H 'Content-Type: application/json' \
+  -d '{"format":"sha1","default_branch":"main"}' \
+  http://127.0.0.1:19100/team/project/create
+git push http://127.0.0.1:19100/team/project HEAD:main
+```
+
+Creation requires write permission and publishes the format and default branch
+through the repository's WAL. Defaults are SHA-1 and `main`; choose SHA-256 for a
+SHA-256 client repository. A successful creation returns 201; an existing
+repository returns 409. A pending publication returns 503: check the repository
+before retrying. Reads and ordinary pushes to a missing name do not create it.
+The two explicitly configured demo repositories also support first-push setup.
+
+One `"*"` policy grants access to new names without a configuration edit or
+restart. For Cognito, configure the groups once:
 
 ```toml
-git_repositories = '{"team/project.git":{"log_id":"team-project","format":"sha1","default_branch":"main","read_groups":["developers"],"write_groups":["developers"],"admin_groups":["operators"]}}'
+git_repositories = '{"*":{"read_groups":["developers"],"write_groups":["developers"],"admin_groups":["operators"]},"team/private":{"read_groups":["private-team"],"write_groups":["private-team"]}}'
 ```
 
-Each name has its own WAL identity. Duplicate identities, ambiguous names and
-invalid formats are rejected. Add another entry and reload Spin to provision a
-repository without rebuilding. Keep each identity stable. The first authorized
-push discovery persists its format and default branch; subsequent configuration
-cannot reinterpret that stored format or change its default branch.
+An exact entry replaces the whole default policy; omitted groups deny access.
+An omitted `log_id` derives an isolated identity from the canonical name, so
+adding a permission override preserves the existing repository. Explicit IDs
+must be unique and cannot use the reserved `auto-` prefix. An optional `format`
+pins that policy to SHA-1 or SHA-256; otherwise readers recover the stored format.
+Creation settings cannot change permissions or reinterpret an existing repository.
+The local shared-password mode grants the password holder access to all names;
+anonymous mode allows public creation and writes.
+
+The optional host maintenance worker currently visits named configuration entries.
+Repositories created under `"*"` still checkpoint their log during pushes and expose
+the same maintenance endpoints, but need an operator to schedule collection by name.
 
 Cognito mode checks signed access tokens, issuer, client, expiry and scope before
 opening storage. Git supplies the access token as its Basic password through an

examples/git/codec.go

@@ -25,16 +25,20 @@ func decodeRoot(data []byte, format config.ObjectFormat, children int) (rootMeta
 	if err := decodeMetadata(data, &meta); err != nil {
 		return meta, err
 	}
-	if !meta.Validated || meta.Format != format || len(meta.Buckets) != children {
+	validFormat := meta.Format == config.SHA1 || meta.Format == config.SHA256
+	if !meta.Validated || !validFormat || len(meta.Buckets) != children {
 		return meta, fmt.Errorf("invalid repository root")
 	}
+	if format != "" && meta.Format != format {
+		return meta, fmt.Errorf("%w: repository format differs from stored state", config.ErrInvalidObjectFormat)
+	}
 	if meta.Head != "" {
 		head := plumbing.ReferenceName(meta.Head)
 		if !head.IsBranch() || head.Validate() != nil {
 			return meta, fmt.Errorf("invalid repository head")
 		}
 	}
-	if err := validateRefs(format, meta.Refs); err != nil {
+	if err := validateRefs(meta.Format, meta.Refs); err != nil {
 		return meta, err
 	}
 	for i, prefix := range meta.Buckets {

examples/git/codec_test.go

@@ -21,6 +21,31 @@ type closeProbe struct {
 	closeErr error
 }
 
+func TestRootInfersOnlyAuthenticatedSupportedFormat(t *testing.T) {
+	for _, format := range []config.ObjectFormat{config.SHA1, config.SHA256} {
+		data, err := json.Marshal(rootMeta{Validated: true, Format: format, Head: "refs/heads/main", Refs: map[string]string{
+			"refs/heads/main": strings.Repeat("1", format.HexSize()),
+		}})
+		if err != nil {
+			t.Fatal(err)
+		}
+		root, err := decodeRoot(data, "", 0)
+		if err != nil || root.Format != format {
+			t.Fatalf("stored format not recovered: %v", err)
+		}
+		other := config.SHA1
+		if format == other {
+			other = config.SHA256
+		}
+		if _, err := decodeRoot(data, other, 0); !errors.Is(err, config.ErrInvalidObjectFormat) {
+			t.Fatal("configured format mismatch accepted")
+		}
+	}
+	if _, err := decodeRoot([]byte(`{"Validated":true,"Format":"sha512","Refs":{}}`), "", 0); err == nil {
+		t.Fatal("unsupported stored format accepted")
+	}
+}
+
 func (r *closeProbe) Close() error { r.closed = true; return r.closeErr }
 
 func TestLooseRoundTripAndValidation(t *testing.T) {

examples/git/main.go

@@ -133,6 +133,7 @@ func serve(response http.ResponseWriter, r *http.Request) {
 		return
 	}
 	service, method := route.Service, route.Method
+	creating := service == "create"
 	maintenance := service == "maintenance"
 	collect := service == "collect"
 	pruneInvalidRefs := service == "prune-invalid-refs"
@@ -150,7 +151,7 @@ func serve(response http.ResponseWriter, r *http.Request) {
 		http.Error(response, message, status)
 		return
 	}
-	if limits.readOnly && (service == transport.ReceivePackService || maintenance || collect || pruneInvalidRefs) {
+	if limits.readOnly && (route.Action == gitWrite || maintenance || collect || pruneInvalidRefs) {
 		http.Error(response, "repository is read-only", http.StatusForbidden)
 		return
 	}
@@ -164,15 +165,26 @@ func serve(response http.ResponseWriter, r *http.Request) {
 		http.Error(response, err.Error(), operationStatus(err))
 		return
 	}
+	if creating {
+		route.Repository, e = repositoryCreation(r.Body, route.Repository)
+		if e != nil {
+			status := operationStatus(e)
+			if status == http.StatusInternalServerError {
+				status = http.StatusBadRequest
+			}
+			http.Error(response, e.Error(), status)
+			return
+		}
+	}
+
 	settings, e := walSettings(getConfig, route.Repository.LogID, limits)
 	if e != nil {
 		http.Error(response, e.Error(), http.StatusInternalServerError)
 		return
 	}
 	session, e := unwrap(func() wt.Result[*wal.Session, wal.Failure] {
-		// First-push discovery requires an empty repository advertisement. Only
-		// a configured repository and an authorized writer may create its head.
-		if route.Action == gitWrite {
+		// Only creation and explicitly provisioned writer requests may open a log.
+		if creating || route.Action == gitWrite && route.Configured {
 			return wal.Open(settings)
 		}
 		return wal.OpenExisting(settings)
@@ -274,16 +286,45 @@ func serve(response http.ResponseWriter, r *http.Request) {
 	s, e := retryOpenStore(open, refresh)
 	if e != nil {
 		log.Printf("git request setup failed stage=open-store: %v", e)
-		http.Error(w, "Git storage unavailable", operationStatus(e))
+		status := operationStatus(e)
+		if creating && errors.Is(e, config.ErrInvalidObjectFormat) {
+			status = http.StatusConflict
+		}
+		http.Error(w, "Git storage unavailable", status)
 		return
 	}
 	defer func() {
 		if s != nil {
 			s.Close()
 		}
 	}()
+	if creating {
+		if s.stateRoot != nil {
+			http.Error(w, "repository already exists", http.StatusConflict)
+			return
+		}
+		if err := s.publish(s.meta.Refs); err != nil {
+			var pending *pendingError
+			if errors.As(err, &pending) {
+				http.Error(w, "repository creation pending; check the repository before retrying", http.StatusServiceUnavailable)
+				return
+			}
+			status := operationStatus(err)
+			if errors.Is(err, errPublicationConflict) {
+				status = http.StatusConflict
+			}
+			http.Error(w, "repository creation was not confirmed", status)
+			return
+		}
+		w.Header().Set("Content-Type", "application/json")
+		w.WriteHeader(http.StatusCreated)
+		_ = json.NewEncoder(w).Encode(struct {
+			Repository string `json:"repository"`
+		}{route.Name})
+		return
+	}
 	if s.stateRoot == nil {
-		if route.Action != gitWrite {
+		if route.Action != gitWrite || !route.Configured {
 			http.NotFound(w, r)
 			return
 		}

examples/git/qualification/aws/hosting.tf

@@ -49,19 +49,19 @@ variable "host_git_max_object_bytes" {
 }
 
 variable "host_repositories" {
-  description = "Explicit repository provisioning and independent group permissions."
+  description = "Repository policies; '*' supplies defaults for API-created repositories."
   type = map(object({
-    log_id         = string
-    format         = string
+    log_id         = optional(string, "")
+    format         = optional(string, "")
     default_branch = optional(string, "main")
     read_groups    = optional(list(string), [])
     write_groups   = optional(list(string), [])
     admin_groups   = optional(list(string), [])
   }))
   default = {}
   validation {
-    condition     = alltrue([for repo in var.host_repositories : contains(["sha1", "sha256"], repo.format)])
-    error_message = "Repository format must be sha1 or sha256."
+    condition     = alltrue([for repo in var.host_repositories : contains(["", "sha1", "sha256"], repo.format)])
+    error_message = "Repository format must be empty, sha1 or sha256."
   }
 }
 
@@ -183,7 +183,7 @@ locals {
       token_url        = "${local.cognito_login_origin}/oauth2/token"
       service_url      = "http://127.0.0.1:8081"
       validation_url   = "http://127.0.0.1:3000/_validate_backend"
-      repositories     = sort(keys(var.host_repositories))
+      repositories     = sort([for name in keys(var.host_repositories) : name if name != "*"])
       budget_seconds   = var.host_maintenance_budget_seconds
       pause_seconds    = var.host_maintenance_pause_seconds
     }))
@@ -192,7 +192,7 @@ locals {
     retry_after = max(1, length(var.host_repositories) * var.host_maintenance_pause_seconds)
     admin_paths = jsonencode(flatten([for name in sort(keys(var.host_repositories)) : [
       for operation in ["maintenance", "collect", "prune-invalid-refs", "recover-retentions-after-drain"] : "/${name}/${operation}"
-    ]]))
+    ] if name != "*"]))
   }) : ""
 }