Repository navigation
fix(@angular/build): preserve asset precedence and expand glob-prefixed ignores in watch mode - #34470
Conversation
There was a problem hiding this comment.
Code Review
This pull request enhances the Angular library builder's asset tracking and emission logic, particularly in watch mode. It ensures that higher-priority assets are not overwritten by lower-priority ones, restores lower-priority fallback assets when overriding assets are deleted, and correctly handles directory-to-file replacements. Feedback on these changes highlights a potential performance bottleneck in BuilderState where a nested loop introduces an O(N * M) complexity during asset emission; maintaining a reverse mapping of output paths to their emitting sources is recommended to achieve O(1) eviction.
…ed ignores in watch mode
Previously in library watch mode, ignore patterns containing glob characters that did not
end with `*` (such as `**/<dir>` or `{a,b}`) were not expanded with `/**` because
`normalizeGlobPattern` checked `picomatch.scan(result).isGlob`. Update
`normalizeGlobPattern` to append `/**` whenever the stripped pattern does not end with
`*`, matching `tinyglobby` directory expansion semantics.
In `collectAssetsToEmit`, incremental watch builds previously only deduplicated colliding
output paths among files present in the current `modifiedFiles` set. Modifying a
lower-priority asset could therefore overwrite an existing higher-priority asset on disk,
and deleting a higher-priority asset did not restore an existing lower-priority fallback
asset. Resolve candidate sources across all asset matchers, evict overridden asset
sources in `BuilderState.recordEmittedOutputs`, and re-emit lower-priority fallback
assets when an overriding asset is deleted. Also preserve cached `statSync` results
during deleted-directory checks so a newly created file replacing a directory at the same
path is still matched and emitted.
69749ae to
0174a19
Compare
| function normalizeGlobPattern(pattern: string): string { | ||
| let result = pattern.endsWith('/') ? pattern.slice(0, -1) : pattern; | ||
| if (pattern.endsWith('/') || !picomatch.scan(result).isGlob) { | ||
| if (pattern.endsWith('/') || !result.endsWith('*')) { |
There was a problem hiding this comment.
We can drop pattern.endsWith('/') || here so that patterns ending with */ (like foo/*/) do not become foo/*/**:
| if (pattern.endsWith('/') || !result.endsWith('*')) { | |
| if (!result.endsWith('*')) { |
| if (matchers.length > 1) { | ||
| // Ensure lower-priority modified assets do not overwrite existing higher-priority assets on disk. | ||
| for (const [outputPath, { file, matcherIndex }] of filesToEmit) { | ||
| for (let laterIndex = matchers.length - 1; laterIndex > matcherIndex; laterIndex--) { | ||
| const laterMatcher = matchers[laterIndex]; | ||
| const candidateFile = getCandidateSourceForOutput(laterMatcher, outputPath); | ||
| if (!candidateFile || candidateFile === file.source) { | ||
| continue; | ||
| } | ||
|
|
||
| if (isMatchingAssetFile(candidateFile, laterMatcher.asset.followSymlinks)) { | ||
| if (emittedAssets.get(candidateFile)?.has(outputPath)) { | ||
| filesToEmit.delete(outputPath); | ||
| } else { | ||
| filesToEmit.set(outputPath, { | ||
| file: createDiskOutputFile(candidateFile, outputPath), | ||
| matcherIndex: laterIndex, | ||
| }); | ||
| } | ||
| break; | ||
| } else if (emittedAssets.get(candidateFile)?.has(outputPath)) { | ||
| filesToDelete.add(candidateFile); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // Restore lower-priority fallback assets when a higher-priority overriding asset is deleted. | ||
| for (const deletedFile of filesToDelete) { | ||
| for ( | ||
| let deletedMatcherIndex = 0; | ||
| deletedMatcherIndex < matchers.length; | ||
| deletedMatcherIndex++ | ||
| ) { | ||
| const { asset, prefix, isMatch } = matchers[deletedMatcherIndex]; | ||
| if (!deletedFile.startsWith(prefix)) { | ||
| continue; | ||
| } | ||
|
|
||
| const relative = deletedFile.slice(prefix.length); | ||
| if (!isMatch(relative)) { | ||
| continue; | ||
| } | ||
|
|
||
| const outputPath = path.posix.join(asset.output, relative); | ||
| if (filesToEmit.has(outputPath)) { | ||
| continue; | ||
| } | ||
|
|
||
| for (let candidateIndex = matchers.length - 1; candidateIndex >= 0; candidateIndex--) { | ||
| if (candidateIndex === deletedMatcherIndex) { | ||
| continue; | ||
| } | ||
|
|
||
| const candidateMatcher = matchers[candidateIndex]; | ||
| const candidateFile = getCandidateSourceForOutput(candidateMatcher, outputPath); | ||
| if (!candidateFile || filesToDelete.has(candidateFile)) { | ||
| continue; | ||
| } | ||
|
|
||
| if (isMatchingAssetFile(candidateFile, candidateMatcher.asset.followSymlinks)) { | ||
| if ( | ||
| candidateIndex < deletedMatcherIndex || | ||
| !emittedAssets.get(candidateFile)?.has(outputPath) | ||
| ) { | ||
| filesToEmit.set(outputPath, { | ||
| file: createDiskOutputFile(candidateFile, outputPath), | ||
| matcherIndex: candidateIndex, | ||
| }); | ||
| } | ||
| break; | ||
| } else if (emittedAssets.get(candidateFile)?.has(outputPath)) { | ||
| filesToDelete.add(candidateFile); | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
This can be simplified quite a bit. In watch mode, emittedAssets is already populated from the initial build, and any created or deleted files are already in modifiedFiles.
In the first loop, we only need to check emittedAssets and filesToDelete in memory instead of checking the disk with isMatchingAssetFile.
In the second loop, emittedAssets.get(deletedFile) already has the output paths for deletedFile, so we can loop over those directly instead of matching deletedFile against matchers again.
| if (matchers.length > 1) { | |
| // Ensure lower-priority modified assets do not overwrite existing higher-priority assets on disk. | |
| for (const [outputPath, { file, matcherIndex }] of filesToEmit) { | |
| for (let laterIndex = matchers.length - 1; laterIndex > matcherIndex; laterIndex--) { | |
| const laterMatcher = matchers[laterIndex]; | |
| const candidateFile = getCandidateSourceForOutput(laterMatcher, outputPath); | |
| if (!candidateFile || candidateFile === file.source) { | |
| continue; | |
| } | |
| if (isMatchingAssetFile(candidateFile, laterMatcher.asset.followSymlinks)) { | |
| if (emittedAssets.get(candidateFile)?.has(outputPath)) { | |
| filesToEmit.delete(outputPath); | |
| } else { | |
| filesToEmit.set(outputPath, { | |
| file: createDiskOutputFile(candidateFile, outputPath), | |
| matcherIndex: laterIndex, | |
| }); | |
| } | |
| break; | |
| } else if (emittedAssets.get(candidateFile)?.has(outputPath)) { | |
| filesToDelete.add(candidateFile); | |
| } | |
| } | |
| } | |
| // Restore lower-priority fallback assets when a higher-priority overriding asset is deleted. | |
| for (const deletedFile of filesToDelete) { | |
| for ( | |
| let deletedMatcherIndex = 0; | |
| deletedMatcherIndex < matchers.length; | |
| deletedMatcherIndex++ | |
| ) { | |
| const { asset, prefix, isMatch } = matchers[deletedMatcherIndex]; | |
| if (!deletedFile.startsWith(prefix)) { | |
| continue; | |
| } | |
| const relative = deletedFile.slice(prefix.length); | |
| if (!isMatch(relative)) { | |
| continue; | |
| } | |
| const outputPath = path.posix.join(asset.output, relative); | |
| if (filesToEmit.has(outputPath)) { | |
| continue; | |
| } | |
| for (let candidateIndex = matchers.length - 1; candidateIndex >= 0; candidateIndex--) { | |
| if (candidateIndex === deletedMatcherIndex) { | |
| continue; | |
| } | |
| const candidateMatcher = matchers[candidateIndex]; | |
| const candidateFile = getCandidateSourceForOutput(candidateMatcher, outputPath); | |
| if (!candidateFile || filesToDelete.has(candidateFile)) { | |
| continue; | |
| } | |
| if (isMatchingAssetFile(candidateFile, candidateMatcher.asset.followSymlinks)) { | |
| if ( | |
| candidateIndex < deletedMatcherIndex || | |
| !emittedAssets.get(candidateFile)?.has(outputPath) | |
| ) { | |
| filesToEmit.set(outputPath, { | |
| file: createDiskOutputFile(candidateFile, outputPath), | |
| matcherIndex: candidateIndex, | |
| }); | |
| } | |
| break; | |
| } else if (emittedAssets.get(candidateFile)?.has(outputPath)) { | |
| filesToDelete.add(candidateFile); | |
| } | |
| } | |
| } | |
| } | |
| } | |
| if (matchers.length > 1) { | |
| // Ensure lower priority modified assets do not overwrite existing higher priority assets. | |
| for (const [outputPath, { matcherIndex }] of filesToEmit) { | |
| for (let laterIndex = matchers.length - 1; laterIndex > matcherIndex; laterIndex--) { | |
| const candidateFile = getCandidateSourceForOutput(matchers[laterIndex], outputPath); | |
| if ( | |
| candidateFile && | |
| !filesToDelete.has(candidateFile) && | |
| emittedAssets.get(candidateFile)?.has(outputPath) | |
| ) { | |
| filesToEmit.delete(outputPath); | |
| break; | |
| } | |
| } | |
| } | |
| // Restore lower priority fallback assets when a higher priority overriding asset is deleted. | |
| for (const deletedFile of filesToDelete) { | |
| const deletedOutputs = emittedAssets.get(deletedFile); | |
| if (!deletedOutputs) { | |
| continue; | |
| } | |
| for (const outputPath of deletedOutputs) { | |
| if (filesToEmit.has(outputPath)) { | |
| continue; | |
| } | |
| for (let candidateIndex = matchers.length - 1; candidateIndex >= 0; candidateIndex--) { | |
| const candidateMatcher = matchers[candidateIndex]; | |
| const candidateFile = getCandidateSourceForOutput(candidateMatcher, outputPath); | |
| if (!candidateFile || filesToDelete.has(candidateFile)) { | |
| continue; | |
| } | |
| if (isMatchingAssetFile(candidateFile, candidateMatcher.asset.followSymlinks)) { | |
| filesToEmit.set(outputPath, { | |
| file: createDiskOutputFile(candidateFile, outputPath), | |
| matcherIndex: candidateIndex, | |
| }); | |
| break; | |
| } | |
| } | |
| } | |
| } | |
| } |
| for (const emittedAsset of toEmit) { | ||
| let outputs = this.#emittedAssets.get(emittedAsset.source); | ||
| if (outputs?.has(emittedAsset.path)) { | ||
| continue; | ||
| } | ||
|
|
||
| if (this.#hasEmittedAssets) { | ||
| for (const [source, prevOutputs] of this.#emittedAssets) { | ||
| if (prevOutputs.delete(emittedAsset.path)) { | ||
| if (prevOutputs.size === 0) { | ||
| this.#emittedAssets.delete(source); | ||
| } | ||
| break; | ||
| } | ||
| } | ||
| } | ||
|
|
||
| if (!outputs) { | ||
| outputs = new Set(); | ||
| this.#emittedAssets.set(emittedAsset.source, outputs); | ||
| } | ||
| outputs.add(emittedAsset.path); | ||
| } | ||
|
|
||
| if (toDelete.length) { | ||
| const candidateDeletes = new Set<string>(); | ||
| for (const source of toDelete) { | ||
| const outputs = this.#emittedAssets.get(source); | ||
| if (outputs) { | ||
| for (const output of outputs) { | ||
| candidateDeletes.add(output); | ||
| } | ||
| this.#emittedAssets.delete(source); | ||
| } | ||
| } | ||
|
|
||
| for (const outputs of this.#emittedAssets.values()) { | ||
| for (const source of toDelete) { | ||
| const outputs = this.#emittedAssets.get(source); | ||
| if (outputs) { | ||
| for (const output of outputs) { | ||
| candidateDeletes.delete(output); | ||
| filesToDelete.add(output); | ||
| } | ||
| } | ||
|
|
||
| for (const output of candidateDeletes) { | ||
| filesToDelete.add(output); | ||
| this.#emittedAssets.delete(source); | ||
| } | ||
| } |
There was a problem hiding this comment.
Instead of looping over this.#emittedAssets for every item in toEmit, we can keep a Map<string, string> from output path to source (for example this.#emittedAssetSources) to look up and remove the previous source directly:
for (const emittedAsset of toEmit) {
let outputs = this.#emittedAssets.get(emittedAsset.source);
if (outputs?.has(emittedAsset.path)) {
continue;
}
const prevSource = this.#emittedAssetSources.get(emittedAsset.path);
if (prevSource !== undefined) {
const prevOutputs = this.#emittedAssets.get(prevSource);
if (prevOutputs) {
prevOutputs.delete(emittedAsset.path);
if (prevOutputs.size === 0) {
this.#emittedAssets.delete(prevSource);
}
}
}
if (!outputs) {
outputs = new Set();
this.#emittedAssets.set(emittedAsset.source, outputs);
}
outputs.add(emittedAsset.path);
this.#emittedAssetSources.set(emittedAsset.path, emittedAsset.source);
}
for (const source of toDelete) {
const outputs = this.#emittedAssets.get(source);
if (outputs) {
for (const output of outputs) {
this.#emittedAssetSources.delete(output);
filesToDelete.add(output);
}
this.#emittedAssets.delete(source);
}
}|
This PR was merged into the repository. The changes were merged into the following branches:
|
Previously in library watch mode, ignore patterns containing glob characters that did not end with
*(such as**/<dir>or{a,b}) were not expanded with/**becausenormalizeGlobPatterncheckedpicomatch.scan(result).isGlob. UpdatenormalizeGlobPatternto append/**whenever the stripped pattern does not end with*, matchingtinyglobbydirectory expansion semantics.In
collectAssetsToEmit, incremental watch builds previously only deduplicated colliding output paths among files present in the currentmodifiedFilesset. Modifying a lower-priority asset could therefore overwrite an existing higher-priority asset on disk, and deleting a higher-priority asset did not restore an existing lower-priority fallback asset. Resolve candidate sources across all asset matchers, evict overridden asset sources inBuilderState.recordEmittedOutputs, and re-emit lower-priority fallback assets when an overriding asset is deleted. Also preserve cachedstatSyncresults during deleted-directory checks so a newly created file replacing a directory at the same path is still matched and emitted.