diff --git a/README.md b/README.md index e4f3799..e7e540c 100644 --- a/README.md +++ b/README.md @@ -119,6 +119,18 @@ ensemble update 3. Run `ensemble push` to sync your local app (screens, widgets, scripts, etc.) with the cloud 4. Optionally run `ensemble pull` to refresh local artifacts from the cloud when other collaborators change them +### Environment files + +Config and secrets sync on `push`, `pull`, and `release use`: + +- **Base**: `.env.config`, `.env.secrets` (shared defaults) +- **Per-alias** (non-default alias, or when both scoped files exist): `.env.config.`, `.env.secrets.` +- Default alias with only base files uses the base pair; other aliases never overwrite base on pull +- **Missing** local file → that side skipped on push (no cloud wipe) +- **Empty** local file → wipe cloud keys on that side (with confirmation) + +Details: [docs/Env-config-aliases.md](docs/Env-config-aliases.md). + ### Versions / releases (snapshots) You can save and use snapshots of your app state in the cloud: diff --git a/docs/Env-config-aliases.md b/docs/Env-config-aliases.md index 61e9b1d..642928c 100644 --- a/docs/Env-config-aliases.md +++ b/docs/Env-config-aliases.md @@ -1,6 +1,6 @@ # Environment config + secrets files (`.env.config` / `.env.secrets` + per-alias overrides) -This document proposes an environment-variable architecture for the Ensemble CLI that supports **multiple app environments** (or “targets”) cleanly and safely. +This document describes the environment-variable architecture used by the Ensemble CLI for **multiple app environments** (or “targets”). The chosen approach is: @@ -67,97 +67,68 @@ api_url=https://prod.ensemble.com --- -## Resolution / precedence rules +## Resolution rules -When running a command for a given alias, apply the same rules to **config** and **secrets**: +For the active alias (`--app` or `ensemble.config.json` → `default`): -- Config: - 1. Read `.env.config` if present (base defaults). - 2. Read `.env.config.` if present (alias overrides). - 3. Merge by key where **alias overrides win**. -- Secrets: - 1. Read `.env.secrets` if present (base defaults). - 2. Read `.env.secrets.` if present (alias overrides). - 3. Merge by key where **alias overrides win**. +| Situation | Files used | +| ---------------------------------------------- | --------------------------------------------------------------------------- | +| Alias is **default** and only base files exist | `.env.config` + `.env.secrets` | +| Alias is **not default** | `.env.config.` + `.env.secrets.` (created on pull if missing) | +| Alias has **both** scoped files (any alias) | scoped pair wins over base | -If only `.env.config` exists, behavior matches today (backwards compatible). Secrets files are additive and optional. +No mixing across tiers. Config and secrets always come from the same tier. -### Why this precedence? - -- Shared values (common across envs) live in one place. -- Environment-specific values override without duplicating the whole file. +Pulling a non-default alias (e.g. `ensemble pull --app uat`) writes cloud env into `.env.config.uat` / `.env.secrets.uat` and leaves base files untouched. --- -## CLI behavior (proposed) - -### Reading env config +## CLI behavior -Commands that need env config should use the **effective env config** for the selected `--app` alias. +### Reading env files -- If `--app` is omitted, treat it as `default` (existing behavior). -- If `.env.config.` is missing, fall back to `.env.config` only. +Commands use the resolved pair for the selected `--app` alias (see resolution rules above). -### Pushing env variables +`--app` is optional and defaults to `ensemble.config.json` → `default`. -If the CLI supports pushing env vars to the cloud, it should be **explicitly scoped**: +### Missing vs empty (push) -- `ensemble push --app prod` may only push the **prod effective env config**. -- It must never push dev values to prod unless the user explicitly made them prod values (via `.env.config.prod` or identical base defaults). +| Local state | Push behavior | +| ----------------------- | ----------------------------------------------------------------- | +| File **missing** | Ignored — no env push for that side, no cloud wipe | +| File **present, empty** | Wipe — warn + `[y/N]` before deleting all cloud keys on that side | -Recommended sync semantics: +### Pushing env variables -- Default: **upsert/patch** (add/update keys present in local effective env config). -- Optional: `--delete-missing` (dangerous) to remove remote keys not present locally. -- Optional: `--dry-run` to show changes without applying. +- `ensemble push --app ` pushes the **effective** env for that alias. +- Config and secrets are pushed independently (missing file → that side skipped). ### Pulling env variables -Similarly, pulling should be scoped: +- `ensemble pull --app ` writes cloud env into the scoped target file when in scoped mode (`.env.config.` / `.env.secrets.`), leaving the base file untouched. +- In legacy mode, pull continues to write `.env.config` / `.env.secrets`. -- `ensemble pull --app prod` should update **only** `.env.config.prod` (or optionally print a diff). -- Avoid writing prod keys into `.env.config` unless explicitly requested. +### Release use + +- `ensemble release use` restores snapshot config into the same write target as pull (scoped or base). --- ## Asset-generated keys and `.env.config` -Today, the CLI “upserts” `.env.config` to ensure asset-related keys exist after: +The CLI upserts `.env.config` for asset-related keys after: - `ensemble add asset` - `ensemble push` (asset upload) -- `ensemble pull` (asset sync) - -This design proposes: - -- Keep `.env.config` as a base defaults file for users. -- Write **asset-generated keys into the alias file** by default (because assets are associated with a specific app target). -Suggested split: - -- `.env.config`: user-managed shared defaults (checked in or not—team choice) -- `.env.config.`: app-target-specific values, including: - - `assets=` for that target - - any cloud-provided asset usage env keys for that target - -Backwards-compatibility note: - -- If alias files are not in use yet, continue writing to `.env.config` as today. -- Once alias files exist (or a new setting/flag opts into alias-mode), write to alias files. +Pull writes asset env keys (`assets=`, per-asset keys) into the resolved config file for the active alias (base or scoped). `ensemble add asset` still upserts the base `.env.config`. --- -## Safety and production protections - -To reduce “oops pushed dev to prod” failures: +## Safety -- **Require explicit target** for sensitive operations (recommended UX): - - For example, pushing env vars could require `--app` when multiple apps exist in `ensemble.config.json`. -- **Stronger confirmations** for production-like aliases (e.g. `prod`, `production`): - - Show a diff summary - - Require a typed confirmation or `--yes` -- **Never default to destructive deletes**: - - `--delete-missing` must be opt-in. +- **Never default to destructive deletes** except when a local env file exists but is empty (explicit wipe semantics above). +- **`--delete-missing`** is not implemented; local-only keys are not auto-deleted from cloud on push. --- @@ -181,14 +152,6 @@ At minimum, consider adding these to `.gitignore`: If you _do_ want to commit alias files for non-secret config, use a more selective ignore pattern or separate “public” vs “secret” configs. -### CLI handling expectations for secrets - -If/when the CLI reads or syncs secrets: - -- Never print secret values in logs (even in `--verbose`). -- Prefer diff output that only shows keys changed (and counts), not values. -- Consider stronger confirmations / restrictions for production aliases. - --- ## Examples @@ -233,16 +196,11 @@ assets=https://assets.prod.ensemble.com/ --- -## Migration plan (incremental) +## Migration plan -1. **Introduce alias file support** in read-paths: - - merge base + alias override (alias wins) -2. **Introduce alias-aware write-paths**: - - write generated keys (assets) into `.env.config.` when applicable -3. **Add env push/pull commands or flags** (if desired): - - ensure all operations are scoped to `--app ` -4. **Add guardrails**: - - diffs, confirmations for prod, optional delete-missing +1. Single-app projects: no change — keep using `.env.config` / `.env.secrets`. +2. Multi-app projects: add `.env.config.` / `.env.secrets.` for per-target overrides; shared defaults stay in the base files. +3. Existing single-app repos can opt in early by creating a scoped file (e.g. `.env.config.dev`). --- diff --git a/src/cloud/firestoreClient.ts b/src/cloud/firestoreClient.ts index db35f9c..1c5f047 100644 --- a/src/cloud/firestoreClient.ts +++ b/src/cloud/firestoreClient.ts @@ -12,8 +12,11 @@ import type { ScreenDTO, ThemeDTO, TranslationDTO, + ConfigDTO, + SecretDTO, } from '../core/dto.js'; import { EnsembleDocumentType } from '../core/dto.js'; +import { configDtoToEnvEntries, secretsDtoToEnvEntries } from '../core/envSync.js'; import { getArtifactConfig, type ArtifactProp } from '../core/artifacts.js'; import { processWithConcurrency } from '../core/concurrency.js'; import { uploadProjectAssetsForPush } from '../core/pushAssets.js'; @@ -195,6 +198,8 @@ export type CloudApp = Pick< | 'theme' | 'translations' | 'assets' + | 'config' + | 'secrets' >; /** Metadata for a saved version (commit); snapshot stored in same doc. */ @@ -211,6 +216,8 @@ export interface VersionMetadata { export type VersionDoc = VersionMetadata; export interface CreateVersionParams { + /** Firestore version document id; must match the storage object name. */ + id: string; message: string; createdAt: string; createdBy: { name: string; email?: string; id: string }; @@ -289,6 +296,7 @@ interface PushPayloadShape { actions?: YamlArtifactPushOperation[]; translations?: YamlArtifactPushOperation[]; theme?: YamlArtifactPushOperation; + assets?: YamlArtifactPushOperation[]; } type CreateYamlOp = Extract; @@ -316,6 +324,86 @@ function getFirestoreConcurrency(): number { return Math.floor(parsed); } +async function applyAssetArchiveOperations( + appId: string, + idToken: string, + project: string, + ops: YamlArtifactPushOperation[] | undefined, + options?: FirestoreClientOptions +): Promise { + if (!ops || ops.length === 0) return; + + const baseCollectionUrl = `https://firestore.googleapis.com/v1/projects/${project}/databases/(default)/documents/apps/${appId}/artifacts`; + const concurrency = getFirestoreConcurrency(); + + await processWithConcurrency( + ops, + async (op) => { + if (op.operation !== 'update') return; + + const docId = op.id; + const docUrl = `${baseCollectionUrl}/${encodeURIComponent(docId)}`; + const historyFields = encodeHistoryFields(op.history); + const historyUrl = `${docUrl}/history`; + + logDebug(options, { + kind: 'push_operation', + appId, + operation: 'update', + artifactKind: 'screens', + documentId: docId, + }); + logDebug(options, { + kind: 'request', + method: 'POST', + url: historyUrl, + context: 'submitCliPush/writeAssetHistory', + }); + const historyRes = await fetch(historyUrl, { + method: 'POST', + headers: { + Authorization: `Bearer ${idToken}`, + 'Content-Type': 'application/json', + }, + body: JSON.stringify({ fields: historyFields }), + }); + if (!historyRes.ok) { + throw await toFirestoreError( + `write history for asset "${op.history.name}"`, + historyRes, + options + ); + } + + const { fields: updateFields, fieldPaths } = encodeUpdateFields('screens', op.updates); + if (fieldPaths.length === 0) return; + + const params = fieldPaths + .map((path) => `updateMask.fieldPaths=${encodeURIComponent(path)}`) + .join('&'); + const patchUrl = `${docUrl}?${params}`; + logDebug(options, { + kind: 'request', + method: 'PATCH', + url: patchUrl, + context: 'submitCliPush/patchAsset', + }); + const patchRes = await fetch(patchUrl, { + method: 'PATCH', + headers: { + Authorization: `Bearer ${idToken}`, + 'Content-Type': 'application/json', + }, + body: JSON.stringify({ fields: updateFields }), + }); + if (!patchRes.ok) { + throw await toFirestoreError(`archive asset "${op.history.name}"`, patchRes, options); + } + }, + concurrency + ); +} + async function applyYamlOperationsForKind( kind: 'screens' | 'widgets' | 'scripts' | 'actions' | 'translations' | 'theme', appId: string, @@ -473,6 +561,7 @@ export async function submitCliPush( if (p.theme) { await applyYamlOperationsForKind('theme', appId, idToken, project, [p.theme], options); } + await applyAssetArchiveOperations(appId, idToken, project, p.assets, options); const names = extras?.assetFileNames?.filter((n) => n.trim() !== '') ?? []; if (names.length === 0 || !extras?.projectRoot) { @@ -773,6 +862,198 @@ function toAssetDTO(doc: FirestoreDocument): AssetDTO { }; } +function parseFirestoreMapField( + field: { mapValue?: { fields?: Record } } | undefined +): Record | undefined { + const mapFields = field?.mapValue?.fields; + if (!mapFields || typeof mapFields !== 'object') return undefined; + const result: Record = {}; + for (const [key, value] of Object.entries(mapFields)) { + if (typeof (value as { stringValue?: string }).stringValue === 'string') { + result[key] = (value as { stringValue: string }).stringValue; + continue; + } + if (typeof (value as { booleanValue?: boolean }).booleanValue === 'boolean') { + result[key] = (value as { booleanValue: boolean }).booleanValue; + continue; + } + if ((value as { integerValue?: string }).integerValue !== undefined) { + result[key] = Number((value as { integerValue: string }).integerValue); + } + } + return Object.keys(result).length > 0 ? result : undefined; +} + +function encodeFirestoreStringMap(values: Record): { + mapValue: { fields: Record }; +} { + const fields: Record = {}; + for (const [key, value] of Object.entries(values)) { + fields[key] = { stringValue: value }; + } + return { mapValue: { fields } }; +} + +function dtoToStringMap(entries: Array<{ key: string; value: string }>): Record { + return Object.fromEntries(entries.map((entry) => [entry.key, entry.value])); +} + +function parseJsonObjectField(content: string | undefined): T | undefined { + if (!content) return undefined; + try { + const parsed = JSON.parse(content) as T; + return parsed && typeof parsed === 'object' ? parsed : undefined; + } catch { + return undefined; + } +} + +function toConfigDTO(doc: FirestoreDocument): ConfigDTO | undefined { + const fields = (doc.fields ?? {}) as FirestoreFields; + const fromContent = parseJsonObjectField( + parseFirestoreString(fields.content as { stringValue?: string }) + ); + const envVariablesFromMap = parseFirestoreMapField( + fields.envVariables as { mapValue?: { fields?: Record } } + ); + const baseUrl = parseFirestoreString(fields.baseUrl as { stringValue?: string }); + const useBrowserUrl = parseFirestoreBoolean(fields.useBrowserUrl as { booleanValue?: boolean }); + const envVariables = + envVariablesFromMap ?? (fromContent?.envVariables as Record | undefined); + if (!envVariables && baseUrl === undefined && useBrowserUrl === undefined) { + return undefined; + } + return { + ...(envVariables && { envVariables }), + ...(baseUrl !== undefined && { baseUrl }), + ...(useBrowserUrl !== undefined && { useBrowserUrl }), + }; +} + +function toSecretDTO(doc: FirestoreDocument): SecretDTO | undefined { + const fields = (doc.fields ?? {}) as FirestoreFields; + const fromContent = parseJsonObjectField( + parseFirestoreString(fields.content as { stringValue?: string }) + ); + const secretsFromMap = parseFirestoreMapField( + fields.secrets as { mapValue?: { fields?: Record } } + ); + if (secretsFromMap) { + return { secrets: secretsFromMap as Record }; + } + if (fromContent) return fromContent; + + const flat = parseFirestoreMapField( + fields as { mapValue?: { fields?: Record } } + ); + return flat ? (flat as SecretDTO) : undefined; +} + +async function upsertEnvArtifactDocument( + appId: string, + idToken: string, + project: string, + documentId: string, + typeValue: string, + contentJson: string, + mapFieldName: 'envVariables' | 'secrets', + mapValues: Record, + options?: FirestoreClientOptions +): Promise { + const collectionUrl = `https://firestore.googleapis.com/v1/projects/${project}/databases/(default)/documents/apps/${appId}/artifacts`; + const docUrl = `${collectionUrl}/${encodeURIComponent(documentId)}`; + const updatedAt = new Date().toISOString(); + const patchFields: FirestoreWriteFields = { + content: { stringValue: contentJson }, + [mapFieldName]: encodeFirestoreStringMap(mapValues), + updatedAt: { timestampValue: updatedAt }, + }; + const fieldPaths = ['content', mapFieldName, 'updatedAt']; + + logDebug(options, { + kind: 'request', + method: 'PATCH', + url: `${docUrl}?${fieldPaths.map((p) => `updateMask.fieldPaths=${encodeURIComponent(p)}`).join('&')}`, + context: 'submitEnvDocumentsPush/patch', + }); + const patchRes = await fetch( + `${docUrl}?${fieldPaths.map((p) => `updateMask.fieldPaths=${encodeURIComponent(p)}`).join('&')}`, + { + method: 'PATCH', + headers: { + Authorization: `Bearer ${idToken}`, + 'Content-Type': 'application/json', + }, + body: JSON.stringify({ fields: patchFields }), + } + ); + + if (patchRes.ok) return; + + if (patchRes.status !== 404) { + throw await toFirestoreError(`update ${documentId}`, patchRes, options); + } + + const createUrl = `${collectionUrl}?documentId=${encodeURIComponent(documentId)}`; + const createFields: FirestoreWriteFields = { + name: { stringValue: documentId }, + type: { stringValue: typeValue }, + ...patchFields, + }; + logDebug(options, { + kind: 'request', + method: 'POST', + url: createUrl, + context: 'submitEnvDocumentsPush/create', + }); + const createRes = await fetch(createUrl, { + method: 'POST', + headers: { + Authorization: `Bearer ${idToken}`, + 'Content-Type': 'application/json', + }, + body: JSON.stringify({ fields: createFields }), + }); + if (!createRes.ok) { + throw await toFirestoreError(`create ${documentId}`, createRes, options); + } +} + +export async function submitEnvDocumentsPush( + appId: string, + idToken: string, + payload: { config?: ConfigDTO; secrets?: SecretDTO }, + options?: FirestoreClientOptions +): Promise { + const project = getEnsembleFirebaseProject(); + if (payload.config) { + await upsertEnvArtifactDocument( + appId, + idToken, + project, + 'appConfig', + EnsembleDocumentType.Environment, + JSON.stringify(payload.config), + 'envVariables', + dtoToStringMap(configDtoToEnvEntries(payload.config)), + options + ); + } + if (payload.secrets) { + await upsertEnvArtifactDocument( + appId, + idToken, + project, + 'secrets', + EnsembleDocumentType.Secrets, + JSON.stringify(payload.secrets), + 'secrets', + dtoToStringMap(secretsDtoToEnvEntries(payload.secrets)), + options + ); + } +} + function getCollaboratorRole( collaboratorsField: | { mapValue?: { fields?: Record } } @@ -1028,6 +1309,8 @@ export async function fetchCloudApp( const translations: TranslationDTO[] = []; const assets: AssetDTO[] = []; let theme: ThemeDTO | undefined; + let config: ConfigDTO | undefined; + let secrets: SecretDTO | undefined; const i18nDocs: FirestoreDocument[] = []; for (const doc of artifacts) { const docId = getDocId(doc.name); @@ -1035,6 +1318,8 @@ export async function fetchCloudApp( if (type === 'screen') screens.push(toScreenDTO(doc)); else if (type === 'i18n') i18nDocs.push(doc); else if (type === 'asset') assets.push(toAssetDTO(doc)); + else if (type === 'config' || docId === 'appConfig') config = toConfigDTO(doc) ?? config; + else if (type === 'secrets' || docId === 'secrets') secrets = toSecretDTO(doc) ?? secrets; else if (type === 'theme') { if (!theme || docId === 'theme') { theme = toThemeDTO(doc); @@ -1065,6 +1350,8 @@ export async function fetchCloudApp( ...(theme && { theme }), ...(translations.length > 0 && { translations }), ...(assets.length > 0 && { assets }), + ...(config && { config }), + ...(secrets && { secrets }), }; } @@ -1133,11 +1420,6 @@ export async function fetchRootScreenName( return parseFirestoreString(doc.fields.name as { stringValue?: string }); } -/** Generate a URL-safe ID for version documents. */ -function generateVersionId(): string { - return crypto.randomUUID().replace(/-/g, ''); -} - /** * Create a version (snapshot) document under apps/{appId}/versions. * expiresAt must be stored as Firestore Timestamp for TTL policy to work. @@ -1149,7 +1431,7 @@ export async function createVersion( options?: FirestoreClientOptions ): Promise<{ id: string }> { const project = getEnsembleFirebaseProject(); - const versionId = generateVersionId(); + const versionId = params.id; const parent = `projects/${project}/databases/(default)/documents/apps/${appId}`; const url = `https://firestore.googleapis.com/v1/${parent}/versions?documentId=${encodeURIComponent(versionId)}`; diff --git a/src/commands/pull.ts b/src/commands/pull.ts index 46c0196..e60908a 100644 --- a/src/commands/pull.ts +++ b/src/commands/pull.ts @@ -21,8 +21,9 @@ import { type RootManifest } from '../core/manifest.js'; import { writeVerboseJson } from '../core/debugFiles.js'; import { computePullPlan, type PullSummary } from '../core/sync.js'; import { applyCloudAssetsToFs, buildEnvConfigForCloudAssets } from '../core/pullAssets.js'; +import { upsertEnvFile } from '../core/envConfig.js'; +import { applyCloudEnvToFs, readProjectEnvFiles } from '../core/envSync.js'; import { ui } from '../core/ui.js'; -import { upsertEnvConfig } from '../core/envConfig.js'; export interface PullOptions { verbose?: boolean; @@ -267,6 +268,7 @@ export async function pullCommand(options: PullOptions = {}): Promise { localFiles, manifestExisting, enabledByProp, + localEnv: await readProjectEnvFiles(projectRoot, appKey, config.default), }); if (plan.allArtifactsMatch && plan.manifestMatch) { @@ -362,16 +364,32 @@ export async function pullCommand(options: PullOptions = {}): Promise { } } - // Always (best-effort) update .env.config for assets so ${env.assets}${env.} references work after pull. + // Always (best-effort) update env config for assets so ${env.assets}${env.} references work after pull. + const envLayout = await readProjectEnvFiles(projectRoot, appKey, config.default); const envResult = buildEnvConfigForCloudAssets(cloudApp.assets); if (envResult.entries.length > 0) { - await upsertEnvConfig(projectRoot, envResult.entries); + await upsertEnvFile(projectRoot, envLayout.configWriteFile, envResult.entries); } if (envResult.failures.length > 0) { ui.warn( - `Some assets had invalid metadata and may be missing from .env.config (${envResult.failures.length}).` + `Some assets had invalid metadata and may be missing from ${envLayout.configWriteFile} (${envResult.failures.length}).` ); } + + await applyCloudEnvToFs( + projectRoot, + { + config: cloudApp.config, + secrets: cloudApp.secrets, + }, + (cloudApp.assets ?? []) + .map((asset) => asset.fileName) + .filter( + (fileName): fileName is string => typeof fileName === 'string' && fileName.length > 0 + ), + appKey, + config.default + ); }); printPullSummary(pullSummary); diff --git a/src/commands/push.ts b/src/commands/push.ts index 5127f6d..601fb5c 100644 --- a/src/commands/push.ts +++ b/src/commands/push.ts @@ -6,6 +6,7 @@ import { checkAppAccess, fetchCloudApp, submitCliPush, + submitEnvDocumentsPush, FirestoreClientError, type FirestoreClientOptions, type CloudApp, @@ -21,6 +22,8 @@ import { withSpinner } from '../lib/spinner.js'; import { writeVerboseJson } from '../core/debugFiles.js'; import { computePushPlan, type PushSummary, type PushCounts } from '../core/sync.js'; import { buildAndWriteManifest } from '../core/manifest.js'; +import { prepareEnvPushState } from '../core/envSync.js'; +import { writeEnvFile } from '../core/envConfig.js'; import { ui } from '../core/ui.js'; export interface PushOptions { @@ -64,11 +67,14 @@ function yamlArtifactChangeTotal(summary: PushSummary): number { ); } -function printPushSummary(summary: PushSummary, options: { verbose?: boolean; isNoop?: boolean }) { +function printPushSummary( + summary: PushSummary, + options: { verbose?: boolean; isNoop?: boolean; envChanged?: boolean } +) { const { appName, environment, counts } = summary; const totalChanges = counts.created + counts.updated + counts.deleted; - if (options.isNoop || totalChanges === 0) { + if ((options.isNoop || totalChanges === 0) && !options.envChanged) { ui.info( `Pushed app "${appName}" to environment "${environment}" (no changes; already up to date).` ); @@ -79,6 +85,7 @@ function printPushSummary(summary: PushSummary, options: { verbose?: boolean; is if (counts.created > 0) parts.push(`${counts.created} created`); if (counts.updated > 0) parts.push(`${counts.updated} updated`); if (counts.deleted > 0) parts.push(`${counts.deleted} deleted`); + if (options.envChanged) parts.push('env files updated'); ui.success(`Pushed app "${appName}" to environment "${environment}" (${parts.join(', ')}).`); @@ -330,31 +337,88 @@ export async function pushCommand(options: PushOptions = {}): Promise { updatedBy, }); bundle = plan.bundle; - await writeVerboseJson(root, 'ensemble-bundle.json', bundle, { - verbose, + + const assetFileNames = data.assetFiles ?? []; + const envPush = await prepareEnvPushState({ + projectRoot: root, + appKey, + defaultAppKey: config.default, + cloudEnv: { config: cloudApp.config, secrets: cloudApp.secrets }, + assetFileNames, + cloudAssets: cloudApp.assets, }); - await writeVerboseJson(root, 'ensemble-diff.json', plan.diff, { + const { + diff: envPushDiff, + pushConfigDto, + pushSecretsDto: localSecretsDto, + pendingLocalEnvConfigWrite, + localEnv: envLocal, + } = envPush; + const envConfigChanged = envPushDiff.configChanged; + const envSecretsChanged = envPushDiff.secretsChanged; + const { wouldClearConfig, wouldClearSecrets } = envPushDiff; + + await writeVerboseJson(root, 'ensemble-bundle.json', bundle, { verbose, }); + await writeVerboseJson( + root, + 'ensemble-diff.json', + { + ...plan.diff, + env: envPushDiff, + }, + { + verbose, + } + ); const summary = plan.summary; const yamlChangeTotal = yamlArtifactChangeTotal(summary); const assetsToUpload = plan.diff.assets.new .map((item) => (item as { fileName?: string }).fileName) .filter((fn): fn is string => typeof fn === 'string' && fn.length > 0); + const assetsToArchive = plan.diff.assets.changed.filter((item) => item.isArchived === true); - if (yamlChangeTotal === 0 && assetsToUpload.length === 0) { + if ( + yamlChangeTotal === 0 && + assetsToUpload.length === 0 && + assetsToArchive.length === 0 && + !envConfigChanged && + !envSecretsChanged + ) { ui.info('Up to date. Nothing to push.'); return; } const pushPayload = buildPushPayload(bundle!, plan.diff, cloudApp, updatedBy); - await writeVerboseJson(root, 'ensemble-push-payload.json', pushPayload, { - verbose, - }); + await writeVerboseJson( + root, + 'ensemble-push-payload.json', + { + ...pushPayload, + ...(envConfigChanged || envSecretsChanged + ? { + env: { + ...(pushConfigDto && { config: pushConfigDto }), + ...(localSecretsDto && { secrets: localSecretsDto }), + }, + } + : {}), + }, + { + verbose, + } + ); if (options.dryRun) { printPushDryRun(plan.diff); + if (envConfigChanged || envSecretsChanged) { + ui.note('Env file changes would also be pushed (.env.config / .env.secrets).'); + } + if (wouldClearConfig || wouldClearSecrets) { + ui.warn('Push would delete all cloud env/secrets (local env file present but empty).'); + } return; } @@ -363,6 +427,16 @@ export async function pushCommand(options: PushOptions = {}): Promise { // eslint-disable-next-line no-console console.log(line); } + if (envConfigChanged) { + // eslint-disable-next-line no-console + console.log(` env:\n ✏️ modified ${envLocal.configWriteFile}`); + } + if (envSecretsChanged) { + // eslint-disable-next-line no-console + console.log(` env:\n ✏️ modified ${envLocal.secretsWriteFile}`); + } + + const isInteractive = Boolean(process.stdout.isTTY && process.stdin.isTTY); const appHome = appConfig.appHome as string | undefined; const cloudHome = getCloudHomeScreenName(cloudApp); @@ -381,10 +455,40 @@ export async function pushCommand(options: PushOptions = {}): Promise { } } + if (wouldClearConfig || wouldClearSecrets) { + const targets = [ + wouldClearConfig && `env variables (${envLocal.configWriteFile})`, + wouldClearSecrets && `secrets (${envLocal.secretsWriteFile})`, + ].filter((t): t is string => Boolean(t)); + ui.warn(`Pushing will delete all cloud ${targets.join(' and ')}.`); + + if (!options.yes) { + if (!isInteractive) { + ui.error( + 'Refusing to clear cloud env/secrets non-interactively without --yes. Re-run with --dry-run to inspect changes.' + ); + process.exitCode = 1; + return; + } + const { proceed: clearEnv } = await prompts({ + type: 'confirm', + name: 'proceed', + message: `Delete all ${targets.join(' and ')} from cloud? Continue? [y/N]`, + initial: false, + }); + if (!clearEnv) { + ui.warn('Push cancelled.'); + process.exitCode = 130; + return; + } + } else { + ui.note('Proceeding without interactive confirmation because --yes was provided.'); + } + } + const manifestNeedsRefresh = hasManifestRelevantChanges(cloudApp, plan.diff); let confirmed = options.yes ?? false; - const isInteractive = Boolean(process.stdout.isTTY && process.stdin.isTTY); const hasDeletes = summary.counts.deleted > 0; const largeChangeSet = yamlChangeTotal >= DESTRUCTIVE_CHANGE_PROMPT_THRESHOLD; @@ -415,10 +519,12 @@ export async function pushCommand(options: PushOptions = {}): Promise { hasDeletes || largeChangeSet ? `This will delete ${summary.counts.deleted} item(s) and apply ${yamlCreatedOrUpdated} other change(s)${ assetsToUpload.length > 0 ? `, and upload ${assetsToUpload.length} asset(s)` : '' + }${ + envConfigChanged || envSecretsChanged ? ', and update env files' : '' }. Continue? [y/N]` : `Proceed with push${ assetsToUpload.length > 0 ? ` and upload ${assetsToUpload.length} asset(s)` : '' - }?`; + }${envConfigChanged || envSecretsChanged ? ' and update env files' : ''}?`; const { proceed } = await prompts({ type: 'confirm', @@ -439,7 +545,11 @@ export async function pushCommand(options: PushOptions = {}): Promise { } try { - if (yamlChangeTotal > 0 || assetsToUpload.length > 0) { + if (pendingLocalEnvConfigWrite) { + await writeEnvFile(root, envLocal.configWriteFile, pendingLocalEnvConfigWrite); + } + + if (yamlChangeTotal > 0 || assetsToUpload.length > 0 || assetsToArchive.length > 0) { const { assetsUploaded } = await withSpinner('Pushing changes to cloud...', () => submitCliPush(appId, idToken, pushPayload, firestoreOptions, { projectRoot: root, @@ -451,6 +561,20 @@ export async function pushCommand(options: PushOptions = {}): Promise { } } + if (envConfigChanged || envSecretsChanged) { + await withSpinner('Pushing env files to cloud...', () => + submitEnvDocumentsPush( + appId, + idToken, + { + ...(pushConfigDto && { config: pushConfigDto }), + ...(localSecretsDto && { secrets: localSecretsDto }), + }, + firestoreOptions + ) + ); + } + if (manifestNeedsRefresh && bundle) { // Only refresh manifest when artifact changes can affect its contents. try { @@ -491,6 +615,10 @@ export async function pushCommand(options: PushOptions = {}): Promise { return; } - printPushSummary(summary, { verbose, isNoop: false }); + printPushSummary(summary, { + verbose, + isNoop: false, + envChanged: envConfigChanged || envSecretsChanged, + }); } } diff --git a/src/commands/release.ts b/src/commands/release.ts index b89d5dc..c198cca 100644 --- a/src/commands/release.ts +++ b/src/commands/release.ts @@ -1,3 +1,4 @@ +import { Command } from 'commander'; import crypto from 'crypto'; import prompts from 'prompts'; @@ -17,6 +18,11 @@ import { uploadReleaseSnapshot, } from '../cloud/storageClient.js'; import { applyCloudStateToFs } from '../core/applyToFs.js'; +import { + applyReleaseConfigToFs, + buildConfigDtoFromEnvEntries, + readProjectEnvFiles, +} from '../core/envSync.js'; import { buildDocumentsFromParsed } from '../core/buildDocuments.js'; import { ArtifactProps, type ArtifactProp } from '../core/artifacts.js'; import { collectAppFiles } from '../core/appCollector.js'; @@ -58,6 +64,19 @@ function formatReleaseLine(index: number, v: VersionDoc): string { return `${index + 1}. ${date} — ${msg} [hash: ${v.id}]`; } +function releaseUseHint(appKey: string, defaultAppKey: string): string { + return appKey === defaultAppKey ? 'ensemble release use' : `ensemble release use --app ${appKey}`; +} + +function releasePushHint(appKey: string, defaultAppKey: string): string { + return appKey === defaultAppKey ? 'ensemble push' : `ensemble push --app ${appKey}`; +} + +/** Commander stores --app on the release parent when subcommands also declare it; read parent opts. */ +export function resolveReleaseAppKey(command: Command): string | undefined { + return command.parent?.opts()?.app as string | undefined; +} + export async function releaseCreateCommand(options: ReleaseCreateOptions = {}): Promise { const root = process.cwd(); const { config, appKey, appId } = await resolveAppContext(options.appKey); @@ -105,6 +124,8 @@ export async function releaseCreateCommand(options: ReleaseCreateOptions = {}): const appName = (appConfig.name as string | undefined) ?? 'App'; const appHome = appConfig.appHome as string | undefined; const localFiles = await collectAppFiles(root); + const localEnv = await readProjectEnvFiles(root, appKey, config.default); + const localConfig = buildConfigDtoFromEnvEntries(localEnv.envConfig); const localApp = buildDocumentsFromParsed(localFiles, appId, appName, appHome, undefined); const snapshot: CloudApp = { id: localApp.id, @@ -119,6 +140,7 @@ export async function releaseCreateCommand(options: ReleaseCreateOptions = {}): localApp.translations.length > 0 && { translations: localApp.translations }), ...(localApp.theme && { theme: localApp.theme }), ...(localApp.assets && localApp.assets.length > 0 && { assets: localApp.assets }), + ...(localConfig && { config: localConfig }), }; try { @@ -132,6 +154,7 @@ export async function releaseCreateCommand(options: ReleaseCreateOptions = {}): appId, idToken, { + id: versionId, message: message.trim(), createdAt: now.toISOString(), createdBy: { name: session.name ?? 'User', id: userId }, @@ -140,7 +163,7 @@ export async function releaseCreateCommand(options: ReleaseCreateOptions = {}): }, firestoreOptions ); - ui.success('Release saved. Run "ensemble release use" to use it.'); + ui.success(`Release saved. Run "${releaseUseHint(appKey, config.default)}" to use it.`); } catch (err) { if (err instanceof FirestoreClientError) { ui.error(err.message); @@ -234,7 +257,7 @@ export async function releaseListCommand(options: ReleaseListOptions = {}): Prom return; } - ui.heading(`Releases for app "${appConfig.name ?? appKey}":`); + ui.heading(`Releases for "${appKey}":`); versions.forEach((v, idx) => { ui.note(formatReleaseLine(idx, v)); }); @@ -384,8 +407,9 @@ export async function releaseUseCommand(options: ReleaseUseOptions = {}): Promis }, }) ); + await applyReleaseConfigToFs(projectRoot, snapshot.config, appKey, config.default); ui.success( - 'Local files updated to selected release. Run "ensemble push" to apply to the cloud.' + `Local files updated to selected release. Run "${releasePushHint(appKey, config.default)}" to apply to the cloud.` ); } catch (err) { if (err instanceof FirestoreClientError) { diff --git a/src/core/bundleDiff.ts b/src/core/bundleDiff.ts index fa52152..19ac3f2 100644 --- a/src/core/bundleDiff.ts +++ b/src/core/bundleDiff.ts @@ -129,9 +129,10 @@ function diffAssets( localAssets: AssetDTO[] | undefined, cloudAssets: AssetDTO[] | undefined ): { changed: ArtifactWithContent[]; new: ArtifactWithContent[] } { - const cloudActiveByFile = new Set( - (cloudAssets ?? []).filter((a) => a.isArchived !== true).map((a) => a.fileName) - ); + const localByFile = new Set((localAssets ?? []).map((asset) => asset.fileName)); + const cloudActive = (cloudAssets ?? []).filter((asset) => asset.isArchived !== true); + const cloudActiveByFile = new Map(cloudActive.map((asset) => [asset.fileName, asset])); + const newItems: ArtifactWithContent[] = []; for (const local of localAssets ?? []) { if (!cloudActiveByFile.has(local.fileName)) { @@ -143,7 +144,21 @@ function diffAssets( } as ArtifactWithContent); } } - return { changed: [], new: newItems }; + + const changedItems: ArtifactWithContent[] = []; + for (const cloud of cloudActive) { + if (!localByFile.has(cloud.fileName)) { + changedItems.push({ + id: cloud.id, + name: cloud.name, + content: cloud.content ?? '', + fileName: cloud.fileName, + isArchived: true, + } as ArtifactWithContent); + } + } + + return { changed: changedItems, new: newItems }; } type ArtifactDisplay = ArtifactWithContent & { fileName?: string }; @@ -258,7 +273,7 @@ export function computeBundleDiff( ); const assets = diffAssets( - localAppForAssets?.assets ?? bundle.assets, + localAppForAssets?.assets ?? [], cloudApp.assets as AssetDTO[] | undefined ); @@ -369,6 +384,39 @@ function buildYamlPushItems( return items; } +function buildAssetPushItems( + diff: { changed: ArtifactWithContent[] }, + cloudAssets: AssetDTO[] | undefined, + now: string, + updatedBy: { name: string; email?: string; id: string } +): YamlArtifactPushItem[] { + const cloudById = new Map((cloudAssets ?? []).map((asset) => [asset.id, asset])); + const items: YamlArtifactPushItem[] = []; + + for (const bundle of diff.changed) { + if (!bundle.isArchived) continue; + const cloud = cloudById.get(bundle.id); + if (!cloud) continue; + const cloudWithMeta = cloud as ArtifactWithContent & { + type?: string; + updatedAt?: string; + updatedBy?: object; + }; + items.push({ + operation: 'update', + id: cloud.id, + history: buildHistoryEntry(cloudWithMeta), + updates: { + isArchived: true, + updatedAt: now, + updatedBy, + }, + }); + } + + return items; +} + /** * Build push payload with history + partial updates for YAML artifacts. * - create: full document for new items @@ -575,6 +623,13 @@ export function buildPushPayload( } } + const assets = buildAssetPushItems( + diff.assets, + cloudApp.assets as AssetDTO[] | undefined, + now, + updatedBy + ); + return { id: bundle.id, name: bundle.name, @@ -585,5 +640,6 @@ export function buildPushPayload( ...(actions.length > 0 && { actions }), ...(translations.length > 0 && { translations }), ...(theme && { theme }), + ...(assets.length > 0 && { assets }), }; } diff --git a/src/core/envConfig.ts b/src/core/envConfig.ts index f428eb9..7b02ed3 100644 --- a/src/core/envConfig.ts +++ b/src/core/envConfig.ts @@ -1,7 +1,24 @@ import fs from 'fs/promises'; import path from 'path'; -function parseEnvConfig(raw: string): { +export interface EnvEntry { + key: string; + value: string; + overwrite?: boolean; +} + +export const ENV_CONFIG_BASE = '.env.config'; +export const ENV_SECRETS_BASE = '.env.secrets'; + +export function envConfigScopedFile(appKey: string): string { + return `${ENV_CONFIG_BASE}.${appKey}`; +} + +export function envSecretsScopedFile(appKey: string): string { + return `${ENV_SECRETS_BASE}.${appKey}`; +} + +function parseEnvFile(raw: string): { lines: string[]; keyToLineIndex: Map; } { @@ -18,18 +35,47 @@ function parseEnvConfig(raw: string): { return { lines, keyToLineIndex }; } -export async function upsertEnvConfig( +export async function envFileExists(projectRoot: string, fileName: string): Promise { + try { + await fs.access(path.join(projectRoot, fileName)); + return true; + } catch { + return false; + } +} + +export async function readEnvFile(projectRoot: string, fileName: string): Promise { + const envPath = path.join(projectRoot, fileName); + let raw = ''; + try { + raw = await fs.readFile(envPath, 'utf8'); + } catch { + return []; + } + const parsed = parseEnvFile(raw); + const entries: EnvEntry[] = []; + for (const [key, lineIndex] of parsed.keyToLineIndex) { + const line = parsed.lines[lineIndex] ?? ''; + const eq = line.indexOf('='); + if (eq <= 0) continue; + entries.push({ key, value: line.slice(eq + 1) }); + } + return entries.sort((a, b) => a.key.localeCompare(b.key)); +} + +export async function upsertEnvFile( projectRoot: string, - entries: Array<{ key: string; value: string; overwrite?: boolean }> + fileName: string, + entries: EnvEntry[] ): Promise { - const envPath = path.join(projectRoot, '.env.config'); + const envPath = path.join(projectRoot, fileName); let raw = ''; try { raw = await fs.readFile(envPath, 'utf8'); } catch { raw = ''; } - const parsed = parseEnvConfig(raw); + const parsed = parseEnvFile(raw); while (parsed.lines.length > 0 && parsed.lines[parsed.lines.length - 1].trim() === '') { parsed.lines.pop(); } @@ -46,3 +92,20 @@ export async function upsertEnvConfig( const normalized = parsed.lines.join('\n').replace(/\n*$/, '\n'); await fs.writeFile(envPath, normalized, 'utf8'); } + +export async function writeEnvFile( + projectRoot: string, + fileName: string, + entries: EnvEntry[] +): Promise { + const envPath = path.join(projectRoot, fileName); + const normalized = entries + .map((entry) => `${entry.key}=${entry.value}`) + .join('\n') + .replace(/\n*$/, '\n'); + await fs.writeFile(envPath, normalized, 'utf8'); +} + +export async function upsertEnvConfig(projectRoot: string, entries: EnvEntry[]): Promise { + await upsertEnvFile(projectRoot, '.env.config', entries); +} diff --git a/src/core/envSync.ts b/src/core/envSync.ts new file mode 100644 index 0000000..09f5b3a --- /dev/null +++ b/src/core/envSync.ts @@ -0,0 +1,522 @@ +import path from 'node:path'; + +import type { ConfigDTO, SecretDTO } from './dto.js'; +import { deriveAssetEnvKey, resolveAssetEnvKey } from './pullAssets.js'; +import { + ENV_CONFIG_BASE, + ENV_SECRETS_BASE, + envConfigScopedFile, + envFileExists, + envSecretsScopedFile, + readEnvFile, + upsertEnvFile, + writeEnvFile, + type EnvEntry, +} from './envConfig.js'; + +export interface CloudEnvState { + config?: ConfigDTO; + secrets?: SecretDTO; +} + +export interface LocalEnvFiles { + appKey: string; + useScoped: boolean; + configWriteFile: string; + secretsWriteFile: string; + envConfig: EnvEntry[]; + envSecrets: EnvEntry[]; + baseConfig: EnvEntry[]; + scopedConfig: EnvEntry[]; + baseSecrets: EnvEntry[]; + scopedSecrets: EnvEntry[]; + envConfigPresent: boolean; + envSecretsPresent: boolean; + baseConfigPresent: boolean; + scopedConfigPresent: boolean; + baseSecretsPresent: boolean; + scopedSecretsPresent: boolean; +} + +export type CloudAssetEnvRef = { + fileName?: string; + copyText?: string; + isArchived?: boolean; +}; + +type AssetKeyContext = { + localKeys: Set; + excludedKeys: Set; + staleKeys: Set; + cloudByFile: Map; +}; + +const activeCloudAssets = (cloudAssets?: CloudAssetEnvRef[]) => + (cloudAssets ?? []).filter( + (a) => typeof a.fileName === 'string' && a.fileName !== '' && a.isArchived !== true + ); + +function buildAssetKeyContext( + localAssetFileNames: string[], + cloudAssets?: CloudAssetEnvRef[] +): AssetKeyContext { + const cloudByFile = new Map(activeCloudAssets(cloudAssets).map((a) => [a.fileName as string, a])); + const localFiles = new Set(localAssetFileNames); + const localKeys = new Set(['assets']); + const staleKeys = new Set(); + + for (const fileName of localAssetFileNames) { + const cloudAsset = cloudByFile.get(fileName); + localKeys.add(resolveAssetEnvKey({ fileName, copyText: cloudAsset?.copyText })); + localKeys.add(deriveAssetEnvKey(fileName)); + } + for (const [fileName, asset] of cloudByFile) { + if (!localFiles.has(fileName)) { + staleKeys.add(resolveAssetEnvKey({ fileName, copyText: asset.copyText })); + } + } + + return { + localKeys, + staleKeys, + excludedKeys: new Set([...localKeys, ...staleKeys]), + cloudByFile, + }; +} + +function entriesEqual(a: EnvEntry[], b: EnvEntry[]): boolean { + const mapB = new Map(b.map((e) => [e.key, e.value])); + return a.length === mapB.size && a.every((e) => mapB.get(e.key) === e.value); +} + +function configEntriesEqual(a?: ConfigDTO, b?: ConfigDTO): boolean { + return entriesEqual(configDtoToEnvEntries(a), configDtoToEnvEntries(b)); +} + +function entriesFromRecord( + record: Record | undefined, + skip?: (key: string) => boolean +): EnvEntry[] { + if (!record) return []; + return Object.entries(record) + .filter(([key, value]) => !skip?.(key) && value !== undefined && value !== null) + .map(([key, value]) => ({ key, value: String(value) })) + .sort((a, b) => a.key.localeCompare(b.key)); +} + +function dtoFromEntries(entries: EnvEntry[], excludeKeys?: Set): ConfigDTO | undefined { + const envVariables: Record = {}; + for (const entry of entries) { + if (!excludeKeys?.has(entry.key)) envVariables[entry.key] = entry.value; + } + return Object.keys(envVariables).length > 0 ? { envVariables } : undefined; +} + +function omitAssetKeys( + entries: EnvEntry[], + assetFileNames: string[], + cloudAssets?: CloudAssetEnvRef[] +): ConfigDTO | undefined { + const excludeKeys = cloudAssets + ? buildAssetKeyContext(assetFileNames, cloudAssets).excludedKeys + : collectAssetEnvKeys(assetFileNames); + return dtoFromEntries(entries, excludeKeys); +} + +function assetFileNameFromEnvValue(rawValue: string): string | undefined { + const value = rawValue.trim(); + if (!value) return undefined; + const base = path.basename(value.split('?')[0] ?? value); + return base.includes('.') ? base : undefined; +} + +export function configDtoToEnvEntries(config: ConfigDTO | undefined): EnvEntry[] { + return entriesFromRecord(config?.envVariables as Record | undefined); +} + +export function secretsDtoToEnvEntries(secrets: SecretDTO | undefined): EnvEntry[] { + if (!secrets || typeof secrets !== 'object') return []; + const nested = + secrets.secrets && typeof secrets.secrets === 'object' + ? (secrets.secrets as Record) + : (secrets as Record); + return entriesFromRecord(nested, (key) => key === 'secrets'); +} + +export function collectAssetEnvKeys(assetFileNames: string[] = []): Set { + return new Set(['assets', ...assetFileNames.map(deriveAssetEnvKey)]); +} + +export function stripAssetKeysFromConfigDto( + config: ConfigDTO | undefined, + assetFileNames: string[] = [], + cloudAssets?: CloudAssetEnvRef[] +): ConfigDTO | undefined { + return omitAssetKeys(configDtoToEnvEntries(config), assetFileNames, cloudAssets); +} + +export function buildConfigDtoFromEnvConfigFile( + entries: EnvEntry[], + assetFileNames: string[] = [], + cloudAssets?: CloudAssetEnvRef[] +): ConfigDTO | undefined { + return omitAssetKeys(entries, assetFileNames, cloudAssets); +} + +export const buildConfigDtoFromEnvEntries = dtoFromEntries; + +export function buildSecretsDtoFromEnvSecretsFile(entries: EnvEntry[]): SecretDTO | undefined { + return entries.length > 0 + ? { secrets: Object.fromEntries(entries.map((e) => [e.key, e.value])) } + : undefined; +} + +function localNonAssetConfigEntries( + localEnv: LocalEnvFiles, + assetFileNames: string[], + cloudAssets?: CloudAssetEnvRef[] +): EnvEntry[] { + if (!localEnv.envConfigPresent) return []; + return configDtoToEnvEntries( + buildConfigDtoFromEnvConfigFile(localEnv.envConfig, assetFileNames, cloudAssets) + ); +} + +function wouldClearConfigOnPush( + localEnv: LocalEnvFiles, + cloudConfig: ConfigDTO | undefined, + assetFileNames: string[], + cloudAssets?: CloudAssetEnvRef[] +): boolean { + if (!localEnv.envConfigPresent) return false; + const cloudNonAsset = configDtoToEnvEntries( + stripAssetKeysFromConfigDto(cloudConfig, assetFileNames, cloudAssets) + ); + return ( + cloudNonAsset.length > 0 && + localNonAssetConfigEntries(localEnv, assetFileNames, cloudAssets).length === 0 + ); +} + +function wouldClearSecretsOnPush(localEnv: LocalEnvFiles, cloudSecrets?: SecretDTO): boolean { + if (!localEnv.envSecretsPresent) return false; + return secretsDtoToEnvEntries(cloudSecrets).length > 0 && localEnv.envSecrets.length === 0; +} + +export function pruneStaleAssetEnvEntries( + entries: EnvEntry[], + assetFileNames: string[], + cloudAssets?: CloudAssetEnvRef[] +): EnvEntry[] { + const { staleKeys } = buildAssetKeyContext(assetFileNames, cloudAssets); + const localFiles = new Set(assetFileNames); + return entries.filter((entry) => { + if (entry.key === 'assets') return assetFileNames.length > 0; + if (staleKeys.has(entry.key)) return false; + const fileName = assetFileNameFromEnvValue(entry.value); + return !(fileName && !localFiles.has(fileName) && entry.key === deriveAssetEnvKey(fileName)); + }); +} + +export function buildPushConfigDto( + localEnv: LocalEnvFiles, + cloudConfig: ConfigDTO | undefined, + assetFileNames: string[] = [], + cloudAssets?: CloudAssetEnvRef[] +): ConfigDTO { + const ctx = buildAssetKeyContext(assetFileNames, cloudAssets); + const localAsset: Record = {}; + for (const entry of localEnv.envConfig) { + if (ctx.localKeys.has(entry.key)) localAsset[entry.key] = entry.value; + } + + const envVariables: Record = { + ...((dtoFromEntries(localEnv.envConfig, ctx.excludedKeys)?.envVariables ?? {}) as Record< + string, + string + >), + }; + + if (assetFileNames.length === 0) return { envVariables }; + + const cloudAssetVars = cloudConfig?.envVariables ?? {}; + const assetsBase = + (typeof localAsset.assets === 'string' ? localAsset.assets.trim() : '') || + (typeof cloudAssetVars.assets === 'string' ? cloudAssetVars.assets.trim() : ''); + if (assetsBase) envVariables.assets = assetsBase; + + for (const fileName of assetFileNames) { + const envKey = resolveAssetEnvKey({ + fileName, + copyText: ctx.cloudByFile.get(fileName)?.copyText, + }); + const value = localAsset[envKey] ?? localAsset[deriveAssetEnvKey(fileName)]; + if (typeof value === 'string') envVariables[envKey] = value; + } + + return { envVariables }; +} + +export async function readProjectEnvFiles( + projectRoot: string, + appKey: string, + defaultAppKey: string +): Promise { + const scopedConfigFile = envConfigScopedFile(appKey); + const scopedSecretsFile = envSecretsScopedFile(appKey); + const [baseConfigPresent, scopedConfigPresent, baseSecretsPresent, scopedSecretsPresent] = + await Promise.all([ + envFileExists(projectRoot, ENV_CONFIG_BASE), + envFileExists(projectRoot, scopedConfigFile), + envFileExists(projectRoot, ENV_SECRETS_BASE), + envFileExists(projectRoot, scopedSecretsFile), + ]); + const scopedPairPresent = scopedConfigPresent && scopedSecretsPresent; + const useScoped = scopedPairPresent || appKey !== defaultAppKey; + const configWriteFile = useScoped ? scopedConfigFile : ENV_CONFIG_BASE; + const secretsWriteFile = useScoped ? scopedSecretsFile : ENV_SECRETS_BASE; + + const baseConfig = baseConfigPresent ? await readEnvFile(projectRoot, ENV_CONFIG_BASE) : []; + const scopedConfig = scopedConfigPresent ? await readEnvFile(projectRoot, scopedConfigFile) : []; + const baseSecrets = baseSecretsPresent ? await readEnvFile(projectRoot, ENV_SECRETS_BASE) : []; + const scopedSecrets = scopedSecretsPresent + ? await readEnvFile(projectRoot, scopedSecretsFile) + : []; + + return { + appKey, + useScoped, + configWriteFile, + secretsWriteFile, + baseConfig, + scopedConfig, + baseSecrets, + scopedSecrets, + envConfig: useScoped ? scopedConfig : baseConfig, + envSecrets: useScoped ? scopedSecrets : baseSecrets, + baseConfigPresent, + scopedConfigPresent, + baseSecretsPresent, + scopedSecretsPresent, + envConfigPresent: useScoped ? scopedConfigPresent : baseConfigPresent, + envSecretsPresent: useScoped ? scopedSecretsPresent : baseSecretsPresent, + }; +} + +export function mergeAssetFileNamesForEnvCompare( + localAssetFileNames: string[] = [], + cloudAssets: Array<{ fileName?: string; isArchived?: boolean }> | undefined = [] +): string[] { + const fromCloud = (cloudAssets ?? []) + .filter((a) => a.isArchived !== true && typeof a.fileName === 'string' && a.fileName !== '') + .map((a) => a.fileName as string); + return [...new Set([...localAssetFileNames, ...fromCloud])]; +} + +export function envConfigEntriesMatchCloud( + localEntries: EnvEntry[], + cloudConfig: ConfigDTO | undefined, + assetFileNames: string[] = [], + cloudAssets?: CloudAssetEnvRef[] +): boolean { + if ( + !configEntriesEqual( + buildConfigDtoFromEnvConfigFile(localEntries, assetFileNames, cloudAssets), + stripAssetKeysFromConfigDto(cloudConfig, assetFileNames, cloudAssets) + ) + ) { + return false; + } + + const assetKeys = collectAssetEnvKeys(assetFileNames); + const localMap = new Map(localEntries.map((entry) => [entry.key, entry.value])); + const cloudAssetVars = cloudConfig?.envVariables ?? {}; + for (const key of assetKeys) { + const cloudValue = cloudAssetVars[key]; + if (cloudValue !== undefined && localMap.get(key) !== String(cloudValue)) return false; + } + return true; +} + +export function envSecretsEntriesMatchCloud( + localEntries: EnvEntry[], + cloudSecrets: SecretDTO | undefined +): boolean { + return entriesEqual(localEntries, secretsDtoToEnvEntries(cloudSecrets)); +} + +export interface EnvPushDiff { + configChanged: boolean; + secretsChanged: boolean; + wouldClearConfig: boolean; + wouldClearSecrets: boolean; + local: { config?: ConfigDTO; secrets?: SecretDTO }; + cloud: { config?: ConfigDTO; secrets?: SecretDTO }; +} + +export function buildEnvPushDiff( + localEnv: LocalEnvFiles, + cloudEnv: CloudEnvState, + assetFileNames: string[] = [], + cloudAssets?: CloudAssetEnvRef[] +): EnvPushDiff { + const pushConfig = buildPushConfigDto(localEnv, cloudEnv.config, assetFileNames, cloudAssets); + const wouldClearConfig = wouldClearConfigOnPush( + localEnv, + cloudEnv.config, + assetFileNames, + cloudAssets + ); + const wouldClearSecrets = wouldClearSecretsOnPush(localEnv, cloudEnv.secrets); + const configChanged = + wouldClearConfig || + (localEnv.envConfigPresent && !configEntriesEqual(pushConfig, cloudEnv.config)); + const secretsChanged = + wouldClearSecrets || + (localEnv.envSecretsPresent && + !entriesEqual(localEnv.envSecrets, secretsDtoToEnvEntries(cloudEnv.secrets))); + const pushSecrets: SecretDTO = { + secrets: Object.fromEntries(localEnv.envSecrets.map((e) => [e.key, e.value])), + }; + + return { + configChanged, + secretsChanged, + wouldClearConfig, + wouldClearSecrets, + local: { + ...(configChanged && { config: pushConfig }), + ...(secretsChanged && { secrets: pushSecrets }), + }, + cloud: { + ...(configChanged && cloudEnv.config && { config: cloudEnv.config }), + ...(secretsChanged && cloudEnv.secrets && { secrets: cloudEnv.secrets }), + }, + }; +} + +export interface EnvPullChanges { + assetFileNames: string[]; + configMatch: boolean; + secretsMatch: boolean; + match: boolean; + filesToUpdate: string[]; +} + +export function computeEnvPullChanges( + localEnv: LocalEnvFiles | undefined, + cloudConfig: ConfigDTO | undefined, + cloudSecrets: SecretDTO | undefined, + localAssetFileNames: string[] = [], + cloudAssets: Array<{ fileName?: string; isArchived?: boolean }> | undefined = [] +): EnvPullChanges { + const assetFileNames = mergeAssetFileNamesForEnvCompare(localAssetFileNames, cloudAssets); + const configMatch = envConfigEntriesMatchCloud( + localEnv?.envConfig ?? [], + cloudConfig, + assetFileNames, + cloudAssets + ); + const secretsMatch = envSecretsEntriesMatchCloud(localEnv?.envSecrets ?? [], cloudSecrets); + const filesToUpdate: string[] = []; + if (!configMatch) filesToUpdate.push(localEnv?.configWriteFile ?? ENV_CONFIG_BASE); + if (!secretsMatch) filesToUpdate.push(localEnv?.secretsWriteFile ?? ENV_SECRETS_BASE); + return { + assetFileNames, + configMatch, + secretsMatch, + match: configMatch && secretsMatch, + filesToUpdate, + }; +} + +export interface EnvPushState { + localEnv: LocalEnvFiles; + diff: EnvPushDiff; + pushConfigDto?: ConfigDTO; + pushSecretsDto?: SecretDTO; + pendingLocalEnvConfigWrite?: EnvEntry[]; +} + +export async function prepareEnvPushState(params: { + projectRoot: string; + appKey: string; + defaultAppKey: string; + cloudEnv: CloudEnvState; + assetFileNames: string[]; + cloudAssets?: CloudAssetEnvRef[]; +}): Promise { + const localEnvRaw = await readProjectEnvFiles( + params.projectRoot, + params.appKey, + params.defaultAppKey + ); + const prunedConfigSource = localEnvRaw.envConfigPresent + ? pruneStaleAssetEnvEntries(localEnvRaw.envConfig, params.assetFileNames, params.cloudAssets) + : localEnvRaw.envConfig; + const localEnv: LocalEnvFiles = { + ...localEnvRaw, + envConfig: prunedConfigSource, + ...(localEnvRaw.useScoped + ? { scopedConfig: prunedConfigSource } + : { baseConfig: prunedConfigSource }), + }; + const diff = buildEnvPushDiff( + localEnv, + params.cloudEnv, + params.assetFileNames, + params.cloudAssets + ); + + return { + localEnv, + diff, + pushConfigDto: diff.local.config, + pushSecretsDto: diff.local.secrets, + ...(localEnvRaw.envConfigPresent && + !entriesEqual(prunedConfigSource, localEnvRaw.envConfig) && { + pendingLocalEnvConfigWrite: prunedConfigSource, + }), + }; +} + +export async function applyReleaseConfigToFs( + projectRoot: string, + config: ConfigDTO | undefined, + appKey: string, + defaultAppKey: string +): Promise { + const configEntries = configDtoToEnvEntries(config); + if (configEntries.length === 0) return; + const { configWriteFile } = await readProjectEnvFiles(projectRoot, appKey, defaultAppKey); + await writeEnvFile(projectRoot, configWriteFile, configEntries); +} + +export async function applyCloudEnvToFs( + projectRoot: string, + cloudEnv: CloudEnvState, + assetFileNames: string[] = [], + appKey = 'default', + defaultAppKey = appKey +): Promise { + const layout = await readProjectEnvFiles(projectRoot, appKey, defaultAppKey); + const configWriteFile = layout.configWriteFile; + const secretsWriteFile = layout.secretsWriteFile; + + const assetKeys = collectAssetEnvKeys(assetFileNames); + const cloudVars = cloudEnv.config?.envVariables ?? {}; + const assetEntries = [...assetKeys] + .map((key) => ({ key, value: cloudVars[key] })) + .filter((entry): entry is EnvEntry => typeof entry.value === 'string'); + if (assetEntries.length > 0) { + await upsertEnvFile(projectRoot, configWriteFile, assetEntries); + } + + const existing = await readEnvFile(projectRoot, configWriteFile); + const keptAssetEntries = existing.filter((entry) => assetKeys.has(entry.key)); + const nonAssetEntries = configDtoToEnvEntries( + stripAssetKeysFromConfigDto(cloudEnv.config, assetFileNames) + ); + await writeEnvFile(projectRoot, configWriteFile, [...keptAssetEntries, ...nonAssetEntries]); + await writeEnvFile(projectRoot, secretsWriteFile, secretsDtoToEnvEntries(cloudEnv.secrets)); +} diff --git a/src/core/pullAssets.ts b/src/core/pullAssets.ts index 2bfa32a..16d90d5 100644 --- a/src/core/pullAssets.ts +++ b/src/core/pullAssets.ts @@ -72,11 +72,14 @@ function extractEnvKeyFromCopyText(copyText: string | undefined): string | undef return nonAssets.length === 1 ? nonAssets[0] : (nonAssets[0] ?? unique[0]); } -function deriveEnvKeyFromFileName(fileName: string): string { - // Match the common behavior used in tests/mocks: replace non-word characters with underscores. +export function deriveAssetEnvKey(fileName: string): string { return fileName.replace(/[^\w]+/g, '_'); } +export function resolveAssetEnvKey(asset: { fileName: string; copyText?: string }): string { + return extractEnvKeyFromCopyText(asset.copyText) ?? deriveAssetEnvKey(asset.fileName); +} + function tryDeriveAssetBaseAndValue( publicUrl: string, fileName: string @@ -144,7 +147,7 @@ export function buildEnvConfigForCloudAssets( const derived = tryDeriveAssetBaseAndValue(url, fileName); if (!derived) continue; - const envKey = extractEnvKeyFromCopyText(a.copyText) ?? deriveEnvKeyFromFileName(fileName); + const envKey = resolveAssetEnvKey({ fileName, copyText: a.copyText }); derivedByFile.set(fileName, { ...derived, envKey, fileName }); baseCounts.set(derived.baseUrl, (baseCounts.get(derived.baseUrl) ?? 0) + 1); } diff --git a/src/core/sync.ts b/src/core/sync.ts index cfc7f40..e6534ee 100644 --- a/src/core/sync.ts +++ b/src/core/sync.ts @@ -1,6 +1,7 @@ import type { CloudApp } from '../cloud/firestoreClient.js'; import type { ParsedAppFiles } from './appCollector.js'; import type { ApplicationDTO } from './dto.js'; +import { computeEnvPullChanges, type LocalEnvFiles } from './envSync.js'; import { ArtifactProps, type ArtifactProp, @@ -73,10 +74,11 @@ function computeKindCounts(items: BundleDiff['screens']): PushCounts { } function computeAssetCounts(items: BundleDiff['assets']): PushCounts { + const deleted = items.changed.filter((item) => item.isArchived === true).length; return { created: items.new.length, - updated: 0, - deleted: 0, + updated: items.changed.filter((item) => item.isArchived !== true).length, + deleted, }; } @@ -213,6 +215,7 @@ export interface ComputePullPlanArgs { localFiles: ParsedAppFiles; manifestExisting: RootManifest; enabledByProp: Record; + localEnv?: LocalEnvFiles; } export function computePullPlan({ @@ -222,6 +225,7 @@ export function computePullPlan({ localFiles, manifestExisting, enabledByProp, + localEnv, }: ComputePullPlanArgs): PullPlan { const matchesByProp: Partial> = {}; let assetsMatch = true; @@ -309,8 +313,17 @@ export function computePullPlan({ } } + const envPull = computeEnvPullChanges( + localEnv, + cloudApp.config, + cloudApp.secrets, + localFiles.assetFiles ?? [], + cloudApp.assets + ); + const envMatch = envPull.match; + const allArtifactsMatch = - ArtifactProps.every((prop) => matchesByProp[prop] ?? true) && assetsMatch; + ArtifactProps.every((prop) => matchesByProp[prop] ?? true) && assetsMatch && envMatch; const changes: PullChange[] = []; let createdCount = 0; @@ -438,6 +451,11 @@ export function computePullPlan({ ).length; skippedCount += missingPublicUrl; + for (const envFile of envPull.filesToUpdate) { + updatedCount += 1; + changes.push({ kind: 'env', file: envFile, operation: 'update' }); + } + const summary: PullSummary = { appName, environment, diff --git a/src/index.ts b/src/index.ts index 0428b82..394a06f 100644 --- a/src/index.ts +++ b/src/index.ts @@ -14,7 +14,12 @@ import { initCommand } from './commands/init.js'; import { pushCommand } from './commands/push.js'; import { addCommand } from './commands/add.js'; import { pullCommand } from './commands/pull.js'; -import { releaseCreateCommand, releaseListCommand, releaseUseCommand } from './commands/release.js'; +import { + releaseCreateCommand, + releaseListCommand, + releaseUseCommand, + resolveReleaseAppKey, +} from './commands/release.js'; import { updateCommand } from './commands/update.js'; import { printCliError, resolveDebugFlag } from './core/cliError.js'; import { ui } from './core/ui.js'; @@ -93,17 +98,19 @@ program const releaseCmd = program .command('release') .description('Manage releases (snapshots) of your app.') - .option('--app ', 'App alias to use (defaults to "default")'); + .option( + '--app ', + 'App alias (defaults to ensemble.config.json default; place before or after subcommand)' + ); releaseCmd .command('create') .description('Create a release (snapshot) from the current cloud state (no push required).') - .option('--app ', 'App alias to use (defaults to "default")') .option('-m, --message ', 'Release message (skips prompt)') .option('-y, --yes', 'Skip message prompt (use empty message)') - .action(async (options: { app?: string; message?: string; yes?: boolean }) => { + .action(async (options: { message?: string; yes?: boolean }, command) => { await releaseCreateCommand({ - appKey: options.app, + appKey: resolveReleaseAppKey(command), message: options.message, yes: options.yes, }); @@ -112,12 +119,11 @@ releaseCmd releaseCmd .command('list') .description('List releases for an app.') - .option('--app ', 'App alias to use (defaults to "default")') .option('--limit ', 'Maximum number of releases to show (default: 20)', (v) => Number(v), 20) .option('--json', 'Print releases as JSON (for scripts)', false) - .action(async (options: { app?: string; limit?: number; json?: boolean }) => { + .action(async (options: { limit?: number; json?: boolean }, command) => { await releaseListCommand({ - appKey: options.app, + appKey: resolveReleaseAppKey(command), limit: options.limit, json: options.json, }); @@ -128,10 +134,9 @@ releaseCmd .description( 'Use a release (snapshot) to update local files (run "ensemble push" to sync cloud).' ) - .option('--app ', 'App alias to use (defaults to "default")') .option('--hash ', 'Release hash to use (non-interactive).') - .action(async (options: { app?: string; hash?: string }) => { - await releaseUseCommand({ appKey: options.app, hash: options.hash }); + .action(async (options: { hash?: string }, command) => { + await releaseUseCommand({ appKey: resolveReleaseAppKey(command), hash: options.hash }); }); // If user runs just `ensemble release`, offer an interactive menu. diff --git a/tests/cli/releaseAppOption.test.ts b/tests/cli/releaseAppOption.test.ts new file mode 100644 index 0000000..be0c904 --- /dev/null +++ b/tests/cli/releaseAppOption.test.ts @@ -0,0 +1,33 @@ +import { Command } from 'commander'; +import { describe, expect, it } from 'vitest'; + +import { resolveReleaseAppKey } from '../../src/commands/release.js'; + +function buildReleaseCli(onAppKey: (appKey: string | undefined) => void): Command { + const program = new Command(); + const releaseCmd = program.command('release').option('--app ', 'App alias'); + releaseCmd.command('create').action((_options, command) => { + onAppKey(resolveReleaseAppKey(command)); + }); + releaseCmd.command('list').action((_options, command) => { + onAppKey(resolveReleaseAppKey(command)); + }); + return program; +} + +describe('release --app CLI parsing', () => { + it.each([ + ['release create --app uat', 'uat'], + ['release --app uat create', 'uat'], + ['release list --app uat', 'uat'], + ['release --app uat list', 'uat'], + ['release create', undefined], + ])('parses %s', (argv, expectedAppKey) => { + let resolved: string | undefined; + const program = buildReleaseCli((appKey) => { + resolved = appKey; + }); + program.parse(argv.split(' '), { from: 'user' }); + expect(resolved).toBe(expectedAppKey); + }); +}); diff --git a/tests/cloud/firestoreClient.test.ts b/tests/cloud/firestoreClient.test.ts index 8447d5b..2a613f3 100644 --- a/tests/cloud/firestoreClient.test.ts +++ b/tests/cloud/firestoreClient.test.ts @@ -4,6 +4,7 @@ import { fetchCloudApp, fetchRootScreenName, submitCliPush, + submitEnvDocumentsPush, listVersions, createVersion, getVersion, @@ -314,6 +315,117 @@ describe('fetchCloudApp', () => { expect(result.theme).toBeDefined(); expect(result.theme?.content).toBe('random theme'); }); + + it('fetches appConfig and secrets into config and secrets', async () => { + const appDoc = { name: 'projects/p/databases/(default)/documents/apps/app-1' }; + const fetchForArtifacts = (artifacts: unknown[]) => async (input: RequestInfo | URL) => { + const urlStr = + typeof input === 'string' ? input : input instanceof URL ? input.toString() : input.url; + if (urlStr.includes('/documents/apps/app-1') && !urlStr.includes('/artifacts')) { + return new Response(JSON.stringify(appDoc), { status: 200 }); + } + if (urlStr.includes('/artifacts')) { + return new Response(JSON.stringify({ documents: artifacts }), { status: 200 }); + } + if (urlStr.includes('internal_artifacts')) { + return new Response(JSON.stringify({ documents: [] }), { status: 200 }); + } + return new Response('Not found', { status: 404 }); + }; + + globalThis.fetch = fetchForArtifacts([ + { + name: 'projects/p/databases/(default)/documents/apps/app-1/artifacts/appConfig', + fields: { + type: { stringValue: 'config' }, + name: { stringValue: 'appConfig' }, + content: { + stringValue: JSON.stringify({ envVariables: { API_URL: 'https://api.example.com' } }), + }, + }, + }, + { + name: 'projects/p/databases/(default)/documents/apps/app-1/artifacts/secrets', + fields: { + type: { stringValue: 'secrets' }, + name: { stringValue: 'secrets' }, + content: { stringValue: JSON.stringify({ secrets: { S1: 'secret-value' } }) }, + }, + }, + ]); + const withContent = await fetchCloudApp('app-1', 'token'); + expect(withContent.config?.envVariables?.API_URL).toBe('https://api.example.com'); + expect(withContent.secrets?.secrets).toEqual({ S1: 'secret-value' }); + + globalThis.fetch = fetchForArtifacts([ + { + name: 'projects/p/databases/(default)/documents/apps/app-1/artifacts/appConfig', + fields: { + type: { stringValue: 'config' }, + name: { stringValue: 'appConfig' }, + content: { + stringValue: JSON.stringify({ envVariables: { E1: 'from-content' } }), + }, + envVariables: { + mapValue: { fields: { E1: { stringValue: 'from-map' } } }, + }, + }, + }, + ]); + const withMap = await fetchCloudApp('app-1', 'token'); + expect(withMap.config?.envVariables?.E1).toBe('from-map'); + }); +}); + +describe('submitEnvDocumentsPush', () => { + const originalFetch = globalThis.fetch; + + afterEach(() => { + globalThis.fetch = originalFetch; + vi.restoreAllMocks(); + }); + + it('patches content, map fields, and updatedAt for env documents', async () => { + const patches: Array<{ url: string; body: string }> = []; + + globalThis.fetch = async (input: RequestInfo | URL, init?: RequestInit) => { + const urlStr = + typeof input === 'string' ? input : input instanceof URL ? input.toString() : input.url; + if (urlStr.includes('/artifacts/') && init?.method === 'PATCH') { + patches.push({ url: urlStr, body: (init.body as string) ?? '' }); + return new Response('{}', { status: 200 }); + } + return new Response('Not found', { status: 404 }); + }; + + await submitEnvDocumentsPush('app-1', 'token', { + config: { envVariables: { E1: 'EV11', assets: 'https://cdn.example.com/' } }, + secrets: { secrets: { S1: 'SK1', S2: 'SK22' } }, + }); + + const configPatch = patches.find((patch) => patch.url.includes('/artifacts/appConfig')); + const secretsPatch = patches.find((patch) => patch.url.includes('/artifacts/secrets')); + expect(configPatch?.url).toContain('updateMask.fieldPaths=envVariables'); + expect(secretsPatch?.url).toContain('updateMask.fieldPaths=secrets'); + + const configBody = JSON.parse(configPatch!.body) as { + fields: { + content?: { stringValue?: string }; + envVariables?: { mapValue?: { fields?: Record } }; + }; + }; + expect(JSON.parse(configBody.fields.content?.stringValue ?? '{}')).toEqual({ + envVariables: { E1: 'EV11', assets: 'https://cdn.example.com/' }, + }); + expect(configBody.fields.envVariables?.mapValue?.fields?.E1?.stringValue).toBe('EV11'); + + const secretsBody = JSON.parse(secretsPatch!.body) as { + fields: { + secrets?: { mapValue?: { fields?: Record } }; + }; + }; + expect(secretsBody.fields.secrets?.mapValue?.fields?.S2?.stringValue).toBe('SK22'); + }); }); describe('fetchRootScreenName', () => { @@ -710,7 +822,7 @@ describe('createVersion', () => { globalThis.fetch = originalFetch; }); - it('POSTs to app versions collection and returns id', async () => { + it('POSTs to app versions collection with provided id', async () => { let capturedUrl: string | null = null; globalThis.fetch = async (input: RequestInfo | URL, init?: RequestInit) => { const urlStr = @@ -727,17 +839,16 @@ describe('createVersion', () => { }; const result = await createVersion('app1', 'token', { + id: 'abc123', message: 'Release 1', createdAt: '2025-01-15T12:00:00Z', expiresAt: '2025-02-15T12:00:00Z', createdBy: { name: 'User', id: 'uid1' }, - snapshotPath: 'releases/app1/ver-123.json', + snapshotPath: 'releases/app1/abc123.json', }); - expect(typeof result.id).toBe('string'); - expect(result.id.length).toBeGreaterThan(0); - expect(capturedUrl).toContain('/documents/apps/app1/versions'); - expect(capturedUrl).toContain('documentId='); + expect(result.id).toBe('abc123'); + expect(capturedUrl).toContain('documentId=abc123'); }); it('throws FirestoreClientError on 403', async () => { @@ -756,6 +867,7 @@ describe('createVersion', () => { await expect( createVersion('app1', 'token', { + id: 'ver-123', message: 'v1', createdAt: '2025-01-15T12:00:00Z', expiresAt: '2025-02-15T12:00:00Z', diff --git a/tests/commands/pushPull.test.ts b/tests/commands/pushPull.test.ts index 1a7cabd..8bfe090 100644 --- a/tests/commands/pushPull.test.ts +++ b/tests/commands/pushPull.test.ts @@ -82,6 +82,7 @@ const cloudModuleMock = vi.hoisted(() => { return { assetsUploaded: 0 }; } ), + submitEnvDocumentsPush: vi.fn(async () => {}), }; }); @@ -990,4 +991,115 @@ describe('push/pull integration (commands)', () => { process.exitCode = originalExitCode; errorSpy.mockRestore(); }); + + it('push uploads local env changes via submitEnvDocumentsPush', async () => { + await fs.writeFile( + path.join(projectRoot, '.env.config'), + 'API_URL=https://local.example.com\n', + 'utf8' + ); + await fs.writeFile(path.join(projectRoot, '.env.secrets'), 'S1=local-secret\n', 'utf8'); + (resolveAppContext as ReturnType).mockResolvedValueOnce({ + projectRoot, + config: { + default: 'dev', + apps: { + dev: { appId: 'app1', name: 'App', appHome: undefined, options: appOptionsRef.value }, + }, + }, + appKey: 'dev', + appId: 'app1', + }); + (cloudModuleMock.fetchCloudApp as ReturnType).mockResolvedValueOnce({ + id: 'app1', + name: 'App', + screens: [], + widgets: [], + scripts: [], + translations: [], + config: { envVariables: { API_URL: 'https://cloud.example.com' } }, + secrets: { secrets: { S1: 'cloud-secret' } }, + }); + + await pushCommand({ yes: true }); + + const { submitEnvDocumentsPush } = cloudModuleMock; + expect(submitEnvDocumentsPush).toHaveBeenCalledTimes(1); + const rawCall = submitEnvDocumentsPush.mock.calls[0]; + expect(rawCall).toBeDefined(); + const payload = ( + rawCall as unknown as [ + string, + string, + { + config?: { envVariables?: Record }; + secrets?: { secrets?: Record }; + }, + ] + )[2]; + expect(payload.config?.envVariables?.API_URL).toBe('https://local.example.com'); + expect(payload.secrets?.secrets?.S1).toBe('local-secret'); + }); + + it('push clears cloud secrets when .env.secrets is empty', async () => { + await fs.writeFile(path.join(projectRoot, '.env.secrets'), '', 'utf8'); + (resolveAppContext as ReturnType).mockResolvedValueOnce({ + projectRoot, + config: { + default: 'dev', + apps: { + dev: { appId: 'app1', name: 'App', appHome: undefined, options: appOptionsRef.value }, + }, + }, + appKey: 'dev', + appId: 'app1', + }); + (cloudModuleMock.fetchCloudApp as ReturnType).mockResolvedValueOnce({ + id: 'app1', + name: 'App', + screens: [], + widgets: [], + scripts: [], + translations: [], + secrets: { secrets: { S1: 'cloud-secret' } }, + }); + + await pushCommand({ yes: true }); + + const payload = ( + (cloudModuleMock.submitEnvDocumentsPush as ReturnType).mock.calls[0] as [ + string, + string, + { secrets?: { secrets?: Record } }, + ] + )[2]; + expect(payload.secrets?.secrets).toEqual({}); + }); + + it('push skips secrets when .env.secrets is missing', async () => { + (resolveAppContext as ReturnType).mockResolvedValueOnce({ + projectRoot, + config: { + default: 'dev', + apps: { + dev: { appId: 'app1', name: 'App', appHome: undefined, options: appOptionsRef.value }, + }, + }, + appKey: 'dev', + appId: 'app1', + }); + (cloudModuleMock.fetchCloudApp as ReturnType).mockResolvedValueOnce({ + id: 'app1', + name: 'App', + screens: [], + widgets: [], + scripts: [], + translations: [], + secrets: { secrets: { S1: 'cloud-secret' } }, + }); + + await pushCommand({ yes: true }); + + expect(cloudModuleMock.submitEnvDocumentsPush).not.toHaveBeenCalled(); + }); }); diff --git a/tests/commands/release.test.ts b/tests/commands/release.test.ts index 825bde7..172bbd9 100644 --- a/tests/commands/release.test.ts +++ b/tests/commands/release.test.ts @@ -21,25 +21,7 @@ const uiNoteMock = vi.hoisted(() => vi.fn()); let projectRoot: string; vi.mock('../../src/config/projectConfig.js', () => ({ - resolveAppContext: vi.fn(async (requestedAppKey?: string) => { - const appKey = requestedAppKey ?? 'default'; - return { - projectRoot: projectRootRef.value, - config: { - default: 'default', - apps: { - default: { - appId: 'app1', - name: 'App', - appHome: undefined, - options: appOptionsRef.value, - }, - }, - }, - appKey, - appId: 'app1', - }; - }), + resolveAppContext: vi.fn(), })); vi.mock('../../src/auth/session.js', () => ({ @@ -91,12 +73,52 @@ import { releaseListCommand, releaseUseCommand, } from '../../src/commands/release.js'; +import { resolveAppContext } from '../../src/config/projectConfig.js'; +import type { CloudApp } from '../../src/cloud/firestoreClient.js'; +import { EnsembleDocumentType } from '../../src/core/dto.js'; + +function defaultAppContext(requestedAppKey?: string) { + const appKey = requestedAppKey ?? 'default'; + return { + projectRoot: projectRootRef.value, + config: { + default: 'default', + apps: { + default: { + appId: 'app1', + name: 'App', + appHome: undefined, + options: appOptionsRef.value, + }, + }, + }, + appKey, + appId: 'app1', + }; +} + +async function writeEnvConfig(projectRoot: string, lines: string[]): Promise { + await fs.writeFile(path.join(projectRoot, '.env.config'), `${lines.join('\n')}\n`, 'utf8'); +} + +function snapshotFromUploadMock(): CloudApp { + expect(uploadReleaseSnapshotMock).toHaveBeenCalledTimes(1); + const snapshotJson = uploadReleaseSnapshotMock.mock.calls[0]?.[3]; + expect(typeof snapshotJson).toBe('string'); + return JSON.parse(snapshotJson as string) as CloudApp; +} describe('release commands', () => { + const originalCwd = process.cwd(); + beforeEach(async () => { projectRoot = await fs.mkdtemp(path.join(os.tmpdir(), 'ensemble-cli-release-')); projectRootRef.value = projectRoot; appOptionsRef.value = {}; + process.chdir(projectRoot); + vi.mocked(resolveAppContext).mockImplementation(async (requestedAppKey?: string) => + defaultAppContext(requestedAppKey) + ); // Minimal app files for buildDocumentsFromParsed: appHome is "Home". await fs.mkdir(path.join(projectRoot, 'screens'), { recursive: true }); @@ -142,19 +164,187 @@ describe('release commands', () => { }); afterEach(async () => { + process.chdir(originalCwd); await fs.rm(projectRoot, { recursive: true, force: true }).catch(() => {}); process.exitCode = 0; vi.clearAllMocks(); }); - it('release create builds snapshot from local files and calls createVersion', async () => { - await releaseCreateCommand({ message: 'My release', yes: true }); + it('release create stores env config in snapshot without secrets or asset publicUrl', async () => { + const assetsDir = path.join(projectRoot, 'assets'); + await fs.mkdir(assetsDir, { recursive: true }); + await fs.writeFile(path.join(assetsDir, 'logo.png'), 'png-bytes', 'utf8'); + await fs.writeFile(path.join(assetsDir, 'Case1_Working.png'), 'png-bytes', 'utf8'); + await writeEnvConfig(projectRoot, [ + 'assets=https://cdn.example.com/base/', + 'logo_png=logo.png?token=abc', + 'E1=EV1', + ]); + await fs.writeFile(path.join(projectRoot, '.env.secrets'), 'S1=SK1\n', 'utf8'); + + await releaseCreateCommand({ message: 'env snapshot', yes: true }); - // We don't assert createVersionMock directly here (module wiring in ESM tests), - // but we do verify the happy-path success message. expect(uiSuccessMock).toHaveBeenCalledWith( 'Release saved. Run "ensemble release use" to use it.' ); + const snapshot = snapshotFromUploadMock(); + expect(snapshot.config?.envVariables).toEqual({ + assets: 'https://cdn.example.com/base/', + logo_png: 'logo.png?token=abc', + E1: 'EV1', + }); + expect(snapshot.secrets).toBeUndefined(); + expect(snapshot.config?.envVariables?.Case1_Working_png).toBeUndefined(); + for (const asset of snapshot.assets ?? []) { + expect(asset.publicUrl).toBeUndefined(); + expect(asset.copyText).toBeUndefined(); + } + }); + + it('release create hints alias-specific use command for non-default app', async () => { + vi.mocked(resolveAppContext).mockResolvedValueOnce({ + projectRoot, + config: { + default: 'dev', + apps: { + dev: { appId: 'app-dev', name: 'Dev App' }, + uat: { appId: 'app-uat', name: 'Uat App' }, + }, + }, + appKey: 'uat', + appId: 'app-uat', + }); + + await releaseCreateCommand({ appKey: 'uat', message: 'uat release', yes: true }); + + expect(uiSuccessMock).toHaveBeenCalledWith( + 'Release saved. Run "ensemble release use --app uat" to use it.' + ); + }); + + it('release create passes the same version id to storage upload and Firestore', async () => { + await releaseCreateCommand({ message: 'sync ids', yes: true }); + + const uploadVersionId = uploadReleaseSnapshotMock.mock.calls[0]?.[2]; + const createParams = createVersionMock.mock.calls[0]?.[2] as { id: string }; + expect(typeof uploadVersionId).toBe('string'); + expect(createParams.id).toBe(uploadVersionId); + }); + + it('release use restores config to scoped alias file for non-default app', async () => { + vi.mocked(resolveAppContext).mockResolvedValueOnce({ + projectRoot, + config: { + default: 'dev', + apps: { + dev: { appId: 'app-dev', name: 'Dev App' }, + uat: { appId: 'app-uat', name: 'Uat App' }, + }, + }, + appKey: 'uat', + appId: 'app-uat', + }); + getVersionMock.mockResolvedValue({ + id: 'hash-1', + message: 'Uat release', + createdAt: '2025-01-15T12:00:00Z', + createdBy: { name: 'User', id: 'uid1' }, + expiresAt: '2025-02-15T12:00:00Z', + snapshotPath: 'releases/app-uat/hash-1.json', + }); + downloadReleaseSnapshotJsonMock.mockResolvedValueOnce( + JSON.stringify({ + id: 'app-uat', + name: 'Uat App', + screens: [], + config: { envVariables: { E1: 'UAT-EV1' } }, + } satisfies CloudApp) + ); + await fs.writeFile(path.join(projectRoot, '.env.config'), 'E1=dev\n', 'utf8'); + await fs.writeFile(path.join(projectRoot, '.env.config.uat'), 'E1=old-uat\n', 'utf8'); + + Object.defineProperty(process.stdout, 'isTTY', { value: false, configurable: true }); + Object.defineProperty(process.stdin, 'isTTY', { value: false, configurable: true }); + + await releaseUseCommand({ appKey: 'uat', hash: 'hash-1' }); + + const baseConfig = await fs.readFile(path.join(projectRoot, '.env.config'), 'utf8'); + const uatConfig = await fs.readFile(path.join(projectRoot, '.env.config.uat'), 'utf8'); + expect(baseConfig).toContain('E1=dev'); + expect(uatConfig).toContain('E1=UAT-EV1'); + expect(uatConfig).not.toContain('old-uat'); + }); + + it('release use hints alias-specific push command for non-default app', async () => { + vi.mocked(resolveAppContext).mockResolvedValueOnce({ + projectRoot, + config: { + default: 'dev', + apps: { + dev: { appId: 'app-dev', name: 'Dev App' }, + uat: { appId: 'app-uat', name: 'Uat App' }, + }, + }, + appKey: 'uat', + appId: 'app-uat', + }); + downloadReleaseSnapshotJsonMock.mockResolvedValueOnce( + JSON.stringify({ + id: 'app-uat', + name: 'Uat App', + screens: [], + config: { envVariables: { E1: 'UAT-EV1' } }, + } satisfies CloudApp) + ); + + Object.defineProperty(process.stdout, 'isTTY', { value: false, configurable: true }); + Object.defineProperty(process.stdin, 'isTTY', { value: false, configurable: true }); + + await releaseUseCommand({ appKey: 'uat', hash: 'hash-1' }); + + expect(uiSuccessMock).toHaveBeenCalledWith( + 'Local files updated to selected release. Run "ensemble push --app uat" to apply to the cloud.' + ); + }); + + it('release use restores snapshot config and never touches secrets', async () => { + downloadReleaseSnapshotJsonMock.mockResolvedValueOnce( + JSON.stringify({ + id: 'app1', + name: 'App', + screens: [], + assets: [ + { + id: 'asset:Case1_Working.png', + name: 'Case1_Working.png', + fileName: 'Case1_Working.png', + content: '', + type: EnsembleDocumentType.Asset, + }, + ], + config: { envVariables: { assets: 'https://cdn.example.com/base/', E1: 'EV1' } }, + secrets: { secrets: { S1: 'SNAPSHOT-SECRET' } }, + } satisfies CloudApp) + ); + await writeEnvConfig(projectRoot, [ + 'assets=https://cdn.example.com/old/', + 'Case1_Working_png=Case1_Working.png?token=old', + 'E1=EV-WRONG', + ]); + await fs.writeFile(path.join(projectRoot, '.env.secrets'), 'S1=LOCAL-SECRET\n', 'utf8'); + + Object.defineProperty(process.stdout, 'isTTY', { value: false, configurable: true }); + Object.defineProperty(process.stdin, 'isTTY', { value: false, configurable: true }); + + await releaseUseCommand({ hash: 'hash-1' }); + + const envConfig = await fs.readFile(path.join(projectRoot, '.env.config'), 'utf8'); + const envSecrets = await fs.readFile(path.join(projectRoot, '.env.secrets'), 'utf8'); + expect(envConfig).toContain('assets=https://cdn.example.com/base/'); + expect(envConfig).toContain('E1=EV1'); + expect(envConfig).not.toContain('Case1_Working_png='); + expect(envSecrets).toContain('S1=LOCAL-SECRET'); + expect(envSecrets).not.toContain('SNAPSHOT-SECRET'); }); it('release list prints heading and lines when versions exist', async () => { diff --git a/tests/core/bundleDiff.test.ts b/tests/core/bundleDiff.test.ts index da44d0e..3096ef8 100644 --- a/tests/core/bundleDiff.test.ts +++ b/tests/core/bundleDiff.test.ts @@ -55,6 +55,57 @@ describe('computeBundleDiff', () => { expect(diff.assets.new[0]).toMatchObject({ fileName: 'new.png' }); }); + it('detects assets deleted locally as archive changes', () => { + const cloudApp = { + id: 'app1', + name: 'App', + assets: [asset('a1', 'logo.png', ''), asset('a2', 'old.png', '')], + }; + const localApp = { + id: 'app1', + name: 'App', + assets: [asset('asset:logo.png', 'logo.png', '')], + }; + const diff = computeBundleDiff(localApp, cloudApp, localApp); + expect(diff.assets.changed).toHaveLength(1); + expect(diff.assets.changed[0]).toMatchObject({ + id: 'a2', + fileName: 'old.png', + isArchived: true, + }); + }); + + it('does not treat cloud-archived assets as new when local assets dir is empty', () => { + const archived = (fileName: string, id: string) => ({ + ...asset(id, fileName, `builds/app/assets/${fileName}`), + isArchived: true, + }); + const cloudApp = { + id: 'app1', + name: 'App', + assets: [ + archived('gone.pdf', 'a1'), + archived('also-gone.png', 'a2'), + asset('a3', 'still-active.png', 'builds/app/assets/still-active.png'), + ], + }; + const bundle = { + id: 'app1', + name: 'App', + assets: cloudApp.assets.map((item) => ({ ...item, isArchived: true })), + }; + const localApp = { id: 'app1', name: 'App' }; + + const diff = computeBundleDiff(bundle, cloudApp, localApp); + + expect(diff.assets.new).toHaveLength(0); + expect(diff.assets.changed).toHaveLength(1); + expect(diff.assets.changed[0]).toMatchObject({ + fileName: 'still-active.png', + isArchived: true, + }); + }); + it('detects changed screens', () => { const cloud: ApplicationDTO = { id: 'app1', @@ -316,6 +367,28 @@ describe('buildPushPayload', () => { } }); + it('archives assets removed locally', () => { + const cloudApp: ApplicationDTO = { + id: 'app1', + name: 'App', + assets: [asset('a1', 'old.png', '')], + }; + const bundle: ApplicationDTO = { + id: 'app1', + name: 'App', + assets: [], + }; + const diff = computeBundleDiff(bundle, cloudApp, bundle); + const payload = buildPushPayload(bundle, diff, cloudApp, updatedBy); + expect(payload.assets).toHaveLength(1); + const archiveItem = payload.assets![0]; + expect(archiveItem.operation).toBe('update'); + if (archiveItem.operation === 'update') { + expect(archiveItem.id).toBe('a1'); + expect(archiveItem.updates.isArchived).toBe(true); + } + }); + it('includes createdAt/createdBy/updatedBy for new artifacts', () => { const cloudApp: ApplicationDTO = { id: 'app1', diff --git a/tests/core/envSync.test.ts b/tests/core/envSync.test.ts new file mode 100644 index 0000000..97dd683 --- /dev/null +++ b/tests/core/envSync.test.ts @@ -0,0 +1,575 @@ +import fs from 'fs/promises'; +import os from 'os'; +import path from 'path'; + +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; + +import { writeEnvFile, type EnvEntry } from '../../src/core/envConfig.js'; +import { + applyCloudEnvToFs, + applyReleaseConfigToFs, + buildEnvPushDiff, + buildPushConfigDto, + computeEnvPullChanges, + pruneStaleAssetEnvEntries, + prepareEnvPushState, + readProjectEnvFiles, + type CloudEnvState, + type LocalEnvFiles, +} from '../../src/core/envSync.js'; +import { envConfigScopedFile, envSecretsScopedFile } from '../../src/core/envConfig.js'; + +function localEnv( + overrides: Partial & Pick +): LocalEnvFiles { + const baseConfig = overrides.baseConfig ?? overrides.envConfig; + return { + appKey: 'default', + useScoped: false, + configWriteFile: '.env.config', + secretsWriteFile: '.env.secrets', + envSecrets: [], + baseConfig, + scopedConfig: [], + baseSecrets: [], + scopedSecrets: [], + envConfigPresent: true, + envSecretsPresent: false, + baseConfigPresent: true, + scopedConfigPresent: false, + baseSecretsPresent: false, + scopedSecretsPresent: false, + ...overrides, + envConfig: overrides.envConfig, + }; +} + +function localEnvFromParts( + configEntries: EnvEntry[], + assetEntries: EnvEntry[] = [] +): LocalEnvFiles { + const envConfig = [...configEntries, ...assetEntries]; + return localEnv({ envConfig, baseConfig: envConfig }); +} + +describe('envSync', () => { + let tmpDir: string; + + beforeEach(async () => { + tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), 'ensemble-envSync-')); + }); + + afterEach(async () => { + await fs.rm(tmpDir, { recursive: true, force: true }); + }); + + it('applyCloudEnvToFs syncs cloud env, preserves asset keys, and drops removed keys', async () => { + await fs.writeFile( + path.join(tmpDir, '.env.config'), + 'assets=https://cdn.example.com/\nlogo_png=logo.png\nE1=EV1\nE2=EV2\n', + 'utf8' + ); + + await applyCloudEnvToFs( + tmpDir, + { + config: { envVariables: { API_URL: 'https://api.example.com', E1: 'EV1' } }, + secrets: { secrets: { S1: 'secret-value' } }, + }, + ['logo.png'], + 'default' + ); + + const envConfig = await fs.readFile(path.join(tmpDir, '.env.config'), 'utf8'); + const envSecrets = await fs.readFile(path.join(tmpDir, '.env.secrets'), 'utf8'); + expect(envConfig).toContain('assets=https://cdn.example.com/'); + expect(envConfig).toContain('logo_png=logo.png'); + expect(envConfig).toContain('API_URL=https://api.example.com'); + expect(envConfig).toContain('E1=EV1'); + expect(envConfig).not.toContain('E2='); + expect(envSecrets).toContain('S1=secret-value'); + }); + + it.each<{ + name: string; + local: LocalEnvFiles; + cloud: CloudEnvState; + assets: string[]; + cloudAssets?: Array<{ fileName?: string; copyText?: string }>; + configChanged: boolean; + secretsChanged: boolean; + wouldClearConfig?: boolean; + wouldClearSecrets?: boolean; + cloudConfig?: Record; + localConfig?: Record; + }>([ + { + name: 'detects config and secrets changes', + local: localEnv({ + envConfig: [{ key: 'E1', value: 'local' }], + envSecrets: [{ key: 'S1', value: 'local' }], + envSecretsPresent: true, + }), + cloud: { + config: { envVariables: { E1: 'cloud' } }, + secrets: { secrets: { S1: 'cloud' } }, + }, + assets: [], + configChanged: true, + secretsChanged: true, + }, + { + name: 'omits snapshots when in sync', + local: localEnv({ + envConfig: [{ key: 'E1', value: 'EV1' }], + envSecrets: [{ key: 'S1', value: 'SK1' }], + envSecretsPresent: true, + }), + cloud: { + config: { envVariables: { E1: 'EV1' } }, + secrets: { secrets: { S1: 'SK1' } }, + }, + assets: [], + configChanged: false, + secretsChanged: false, + }, + { + name: 'skips env push when env files are missing but cloud has values', + local: localEnv({ + envConfig: [], + baseConfig: [], + envConfigPresent: false, + envSecretsPresent: false, + baseConfigPresent: false, + }), + cloud: { + config: { envVariables: { E1: 'EV1' } }, + secrets: { secrets: { S1: 'SK1' } }, + }, + assets: [], + configChanged: false, + secretsChanged: false, + wouldClearConfig: false, + wouldClearSecrets: false, + }, + { + name: 'wipes cloud when env files are present but empty', + local: localEnv({ + envConfig: [], + baseConfig: [], + envSecrets: [], + envConfigPresent: true, + envSecretsPresent: true, + }), + cloud: { + config: { envVariables: { E1: 'EV1' } }, + secrets: { secrets: { S1: 'SK1' } }, + }, + assets: [], + configChanged: true, + secretsChanged: true, + wouldClearConfig: true, + wouldClearSecrets: true, + }, + { + name: 'shows full push config vs cloud including asset keys', + local: localEnv({ + envConfig: [ + { key: 'E1', value: 'EV11' }, + { key: 'assets', value: 'https://cdn.example.com/' }, + { key: 'logo_png', value: 'logo.png?local=abc' }, + ], + }), + cloud: { + config: { + envVariables: { + assets: 'https://cdn.example.com/', + logo_png: 'logo.png', + E1: 'EV1', + E2: 'EV2', + }, + }, + }, + assets: ['logo.png'], + configChanged: true, + secretsChanged: false, + cloudConfig: { + assets: 'https://cdn.example.com/', + logo_png: 'logo.png', + E1: 'EV1', + E2: 'EV2', + }, + localConfig: { + assets: 'https://cdn.example.com/', + logo_png: 'logo.png?local=abc', + E1: 'EV11', + }, + }, + { + name: 'flags configChanged when only stale asset env keys differ from cloud', + local: localEnv({ + envConfig: [{ key: 'E1', value: 'EV1' }], + }), + cloud: { + config: { + envVariables: { + assets: 'https://cdn.example.com/', + DSA_Viva_rtf: 'DSA%20Viva.rtf?token=abc', + github_html: 'first%20token%20github.html?token=def', + E1: 'EV1', + }, + }, + }, + assets: [], + cloudAssets: [ + { fileName: 'DSA Viva.rtf' }, + { fileName: 'first token github.html', copyText: '${env.github_html}' }, + ], + configChanged: true, + secretsChanged: false, + localConfig: { E1: 'EV1' }, + cloudConfig: { + assets: 'https://cdn.example.com/', + DSA_Viva_rtf: 'DSA%20Viva.rtf?token=abc', + github_html: 'first%20token%20github.html?token=def', + E1: 'EV1', + }, + }, + ])( + '$name', + ({ + local, + cloud, + assets, + cloudAssets, + configChanged, + secretsChanged, + wouldClearConfig = false, + wouldClearSecrets = false, + cloudConfig, + localConfig, + }) => { + const diff = buildEnvPushDiff(local, cloud, assets, cloudAssets); + expect(diff.configChanged).toBe(configChanged); + expect(diff.secretsChanged).toBe(secretsChanged); + expect(diff.wouldClearConfig).toBe(wouldClearConfig); + expect(diff.wouldClearSecrets).toBe(wouldClearSecrets); + if (!configChanged && !secretsChanged) { + expect(diff.local).toEqual({}); + expect(diff.cloud).toEqual({}); + return; + } + if (cloudConfig) expect(diff.cloud.config?.envVariables).toEqual(cloudConfig); + if (localConfig) expect(diff.local.config?.envVariables).toEqual(localConfig); + } + ); + + it('buildPushConfigDto keeps local assets and drops deleted cloud asset keys', () => { + expect( + buildPushConfigDto( + localEnvFromParts( + [ + { key: 'E1', value: 'EV11' }, + { key: 'E2', value: 'EV2' }, + ], + [ + { key: 'assets', value: 'https://cdn.example.com/' }, + { key: 'logo_png', value: 'logo.png?local=abc' }, + ] + ), + { + envVariables: { + assets: 'https://cdn.example.com/', + logo_png: 'logo.png?token=abc', + E1: 'EV1', + }, + }, + ['logo.png'] + ).envVariables + ).toEqual({ + assets: 'https://cdn.example.com/', + logo_png: 'logo.png?local=abc', + E1: 'EV11', + E2: 'EV2', + }); + + expect( + buildPushConfigDto( + localEnvFromParts( + [{ key: 'E1', value: 'EV11' }], + [ + { key: 'assets', value: 'https://cdn.example.com/' }, + { key: 'report_html', value: 'report.html?local=abc' }, + ] + ), + { + envVariables: { + assets: 'https://cdn.example.com/', + report_html: 'report.html?token=abc', + sheet_xlsx: 'sheet.xlsx?token=def', + E1: 'EV1', + }, + }, + ['report.html'], + [ + { fileName: 'report.html', copyText: '${env.report_html}' }, + { fileName: 'sheet.xlsx', copyText: '${env.sheet_xlsx}' }, + ] + ).envVariables + ).toEqual({ + assets: 'https://cdn.example.com/', + report_html: 'report.html?local=abc', + E1: 'EV11', + }); + + expect( + buildPushConfigDto( + localEnvFromParts([{ key: 'E1', value: 'EV11' }]), + { + envVariables: { + assets: 'https://cdn.example.com/', + logo_png: 'logo.png?token=abc', + E1: 'EV1', + E2: 'EV2', + }, + }, + [] + ).envVariables + ).toEqual({ E1: 'EV11' }); + }); + + it.each<{ + name: string; + entries: Array<{ key: string; value: string }>; + assetFileNames: string[]; + cloudAssets?: Array<{ fileName?: string; copyText?: string }>; + expected: Array<{ key: string; value: string }>; + }>([ + { + name: 'removes copyText and derived keys for deleted assets', + entries: [ + { key: 'assets', value: 'https://cdn.example.com/' }, + { key: 'report_html', value: 'report.html?token=abc' }, + { key: 'sheet_xlsx', value: 'sheet.xlsx?token=def' }, + { key: 'MIH_4735_pdf', value: 'MIH-4735.pdf?token=xyz' }, + { key: 'E1', value: 'EV1' }, + ], + assetFileNames: [], + cloudAssets: [ + { fileName: 'report.html', copyText: '${env.report_html}' }, + { fileName: 'sheet.xlsx', copyText: '${env.sheet_xlsx}' }, + ], + expected: [{ key: 'E1', value: 'EV1' }], + }, + { + name: 'keeps local assets and user config', + entries: [ + { key: 'assets', value: 'https://cdn.example.com/' }, + { key: 'img1_png', value: 'img1.png?token=abc' }, + { key: 'deleted_png', value: 'deleted.png?token=def' }, + { key: 'E1', value: 'EV1' }, + ], + assetFileNames: ['img1.png'], + expected: [ + { key: 'assets', value: 'https://cdn.example.com/' }, + { key: 'img1_png', value: 'img1.png?token=abc' }, + { key: 'E1', value: 'EV1' }, + ], + }, + { + name: 'keeps non-asset keys whose value looks like a file URL', + entries: [{ key: 'download_url', value: 'https://cdn.example.com/guide.pdf?token=abc' }], + assetFileNames: [], + expected: [{ key: 'download_url', value: 'https://cdn.example.com/guide.pdf?token=abc' }], + }, + ])('$name', ({ entries, assetFileNames, cloudAssets, expected }) => { + expect(pruneStaleAssetEnvEntries(entries, assetFileNames, cloudAssets)).toEqual(expected); + }); + + it('computeEnvPullChanges flags config mismatch including missing asset env keys', () => { + const result = computeEnvPullChanges( + localEnv({ + envConfig: [ + { key: 'assets', value: 'https://cdn.example.com/' }, + { key: 'E1', value: 'EV111' }, + ], + envSecrets: [{ key: 'S1', value: 'SK1' }], + envSecretsPresent: true, + }), + { + envVariables: { + assets: 'https://cdn.example.com/', + Case1_Working_png: 'Case1_Working.png?alt=media&token=abc', + E1: 'EV111', + }, + }, + { secrets: { S1: 'SK1' } }, + ['Case1_Working.png'], + [{ fileName: 'Case1_Working.png' }] + ); + + expect(result.configMatch).toBe(false); + expect(result.secretsMatch).toBe(true); + expect(result.filesToUpdate).toEqual(['.env.config']); + }); + + it('computeEnvPullChanges flags missing asset keys in scoped alias config files', () => { + const result = computeEnvPullChanges( + localEnv({ + useScoped: true, + configWriteFile: '.env.config.uat', + envConfig: [{ key: 'E2', value: 'EK2' }], + envConfigPresent: true, + }), + { + envVariables: { + assets: 'https://cdn.example.com/', + logo_png: 'logo.png?token=abc', + E2: 'EK2', + }, + }, + undefined, + ['logo.png'], + [{ fileName: 'logo.png' }] + ); + + expect(result.configMatch).toBe(false); + expect(result.filesToUpdate).toEqual(['.env.config.uat']); + }); + + it('prepareEnvPushState omits cloud asset keys when no local asset files exist', async () => { + await fs.writeFile(path.join(tmpDir, '.env.config'), 'E1=EV11\n', 'utf8'); + + const state = await prepareEnvPushState({ + projectRoot: tmpDir, + appKey: 'default', + defaultAppKey: 'default', + cloudEnv: { + config: { envVariables: { assets: 'https://cdn.example.com/', E1: 'EV1' } }, + }, + assetFileNames: [], + }); + + expect(state.diff.configChanged).toBe(true); + expect(state.pushConfigDto?.envVariables).toEqual({ E1: 'EV11' }); + }); + + it('prepareEnvPushState prunes stale asset env keys only after confirm', async () => { + await fs.writeFile( + path.join(tmpDir, '.env.config'), + 'assets=https://cdn.example.com/\nimg1_png=img1.png?token=abc\ndel_png=del.png?token=def\nE1=EV11\n', + 'utf8' + ); + + const state = await prepareEnvPushState({ + projectRoot: tmpDir, + appKey: 'default', + defaultAppKey: 'default', + cloudEnv: { config: { envVariables: { E1: 'EV1' } } }, + assetFileNames: ['img1.png'], + }); + + const envConfigBeforeConfirm = await fs.readFile(path.join(tmpDir, '.env.config'), 'utf8'); + expect(envConfigBeforeConfirm).toContain('del_png='); + expect(state.pendingLocalEnvConfigWrite?.some((entry) => entry.key === 'del_png')).toBe(false); + + await writeEnvFile(tmpDir, '.env.config', state.pendingLocalEnvConfigWrite!); + + const envConfig = await fs.readFile(path.join(tmpDir, '.env.config'), 'utf8'); + expect(envConfig).toContain('img1_png=img1.png?token=abc'); + expect(envConfig).not.toContain('del_png='); + }); + + it('applyReleaseConfigToFs restores full snapshot config', async () => { + await applyReleaseConfigToFs( + tmpDir, + { + envVariables: { + assets: 'https://cdn.example.com/', + logo_png: 'logo.png', + E1: 'EV1', + }, + }, + 'default', + 'default' + ); + + const envConfig = await fs.readFile(path.join(tmpDir, '.env.config'), 'utf8'); + expect(envConfig).toContain('assets=https://cdn.example.com/'); + expect(envConfig).toContain('logo_png=logo.png'); + expect(envConfig).toContain('E1=EV1'); + }); + + it('readProjectEnvFiles uses scoped pair when both alias files exist', async () => { + await fs.writeFile(path.join(tmpDir, '.env.config'), 'E1=base\nE2=shared\n', 'utf8'); + await fs.writeFile(path.join(tmpDir, '.env.secrets'), 'S1=base\n', 'utf8'); + await fs.writeFile(path.join(tmpDir, envConfigScopedFile('uat')), 'E1=uat\n', 'utf8'); + await fs.writeFile(path.join(tmpDir, envSecretsScopedFile('uat')), 'S1=uat\n', 'utf8'); + + const env = await readProjectEnvFiles(tmpDir, 'uat', 'dev'); + expect(env.useScoped).toBe(true); + expect(env.configWriteFile).toBe('.env.config.uat'); + expect(env.secretsWriteFile).toBe('.env.secrets.uat'); + expect(env.envConfig).toEqual([{ key: 'E1', value: 'uat' }]); + expect(env.envSecrets).toEqual([{ key: 'S1', value: 'uat' }]); + }); + + it('readProjectEnvFiles uses base for default alias when only base files exist', async () => { + await fs.writeFile(path.join(tmpDir, '.env.config'), 'E1=base\n', 'utf8'); + await fs.writeFile(path.join(tmpDir, '.env.secrets'), 'S1=base\n', 'utf8'); + + const env = await readProjectEnvFiles(tmpDir, 'dev', 'dev'); + expect(env.useScoped).toBe(false); + expect(env.configWriteFile).toBe('.env.config'); + expect(env.envConfig).toEqual([{ key: 'E1', value: 'base' }]); + expect(env.envSecrets).toEqual([{ key: 'S1', value: 'base' }]); + }); + + it('readProjectEnvFiles targets scoped paths for non-default alias even before files exist', async () => { + await fs.writeFile(path.join(tmpDir, '.env.config'), 'E1=base\n', 'utf8'); + await fs.writeFile(path.join(tmpDir, '.env.secrets'), 'S1=base\n', 'utf8'); + + const env = await readProjectEnvFiles(tmpDir, 'uat', 'dev'); + expect(env.useScoped).toBe(true); + expect(env.configWriteFile).toBe('.env.config.uat'); + expect(env.secretsWriteFile).toBe('.env.secrets.uat'); + expect(env.envConfig).toEqual([]); + expect(env.envSecrets).toEqual([]); + expect(env.envConfigPresent).toBe(false); + expect(env.envSecretsPresent).toBe(false); + }); + + it('applyCloudEnvToFs creates scoped files for non-default alias without touching base', async () => { + await fs.writeFile(path.join(tmpDir, '.env.config'), 'E1=base\n', 'utf8'); + await fs.writeFile(path.join(tmpDir, '.env.secrets'), 'S1=base\n', 'utf8'); + await applyCloudEnvToFs( + tmpDir, + { + config: { + envVariables: { + assets: 'https://cdn.example.com/', + logo_png: 'logo.png', + E1: 'uat', + E2: 'new', + }, + }, + secrets: { secrets: { S1: 'sk' } }, + }, + ['logo.png'], + 'uat', + 'dev' + ); + + const baseConfig = await fs.readFile(path.join(tmpDir, '.env.config'), 'utf8'); + const baseSecrets = await fs.readFile(path.join(tmpDir, '.env.secrets'), 'utf8'); + const scoped = await fs.readFile(path.join(tmpDir, envConfigScopedFile('uat')), 'utf8'); + const secrets = await fs.readFile(path.join(tmpDir, envSecretsScopedFile('uat')), 'utf8'); + expect(baseConfig).toContain('E1=base'); + expect(baseSecrets).toContain('S1=base'); + expect(scoped).toContain('E1=uat'); + expect(scoped).toContain('E2=new'); + expect(scoped).toContain('assets=https://cdn.example.com/'); + expect(scoped).toContain('logo_png=logo.png'); + expect(secrets).toContain('S1=sk'); + }); +});