Add a failure cooldown for dead URLs

CHANGELOG.md

@@ -202,6 +202,7 @@ In Claude Code, you can also install them as a plugin, and they load on their ow
 - Add `View/imagePipeline(_:)` and `EnvironmentValues/imagePipeline` to set the pipeline of every `LazyImage` in a view hierarchy. `LazyImage/pipeline(_:)` takes precedence – https://github.com/kean/Nuke/pull/1094
 - Add `ImageProcessors/Crop`, `ImageProcessors/Flip`, `ImageProcessors/Rotate`, and `ImageProcessors/Tint`, with `ImageProcessing/crop(rect:unit:)`, `ImageProcessing/flip(_:)`, `ImageProcessing/rotate(degrees:)`, and `ImageProcessing/tint(color:)` – https://github.com/kean/Nuke/pull/1103
 - Add `ImageDecoders/PDF`, an opt-in decoder that draws a page of a PDF document at the size the request asks for, and `AssetType/pdf`. It isn't available on tvOS – https://github.com/kean/Nuke/pull/1107
+- Add `ImagePipeline/Configuration-swift.struct/failureCooldown`, off by default, which stops requesting a URL for a while after a `404` or a `410`. Those loads fail with `ImagePipeline/Error/urlInCooldown(error:)`, and `ImageRequest/Options-swift.struct/skipFailureCooldown` and `ImageLoader/reload()` skip it – https://github.com/kean/Nuke/pull/1106
 
 ### Performance
 

Documentation/Nuke.docc/Customization/LoadingData/loading-data.md

@@ -39,6 +39,41 @@ let pipeline = ImagePipeline(delegate: AuthenticatedPipelineDelegate()) {
 
 The method is called on every URL-based request, after resumable data headers are applied but before the request is passed to ``DataLoading``. Throwing an error cancels the request and surfaces the error as ``ImagePipeline/Error/dataLoadingFailed(error:)``. The hook is not called for requests that use a custom data fetch closure or that target local file resources.
 
+## Failure Cooldown
+
+A dead URL in a feed costs a request every time a cell shows it. ``ImagePipeline/Configuration-swift.struct/failureCooldown`` makes the pipeline remember the answer for a while. It's off by default:
+
+```swift
+let pipeline = ImagePipeline {
+    $0.failureCooldown = .init(duration: 300)
+}
+```
+
+After a `404` or a `410`, the loads of that URL fail with ``ImagePipeline/Error/urlInCooldown(error:)`` for five minutes, without a request and without waiting for ``ImagePipeline/Configuration-swift.struct/dataLoadingQueue``. A successful load ends the cooldown early. The error is a case of its own, rather than ``ImagePipeline/Error/dataLoadingFailed(error:)``, because no request was made, and a `404` that isn't in the network log would be hard to explain. ``ImagePipeline/Error/dataLoadingError`` returns the original error for both.
+
+The cooldown is narrow on purpose:
+
+- **It covers URLs, after the caches.** An image that is in the memory or disk cache is still served, and an `ImageRequest` with a `data` or `image` closure is never affected. All requests for a URL share one cooldown, whatever their processors, thumbnail, or ``ImageRequest/imageID``, and each pipeline has its own.
+- **It only counts answers about the URL.** A connection failure, a timeout, or a cancellation doesn't count, and neither do `401`, `403`, `429`, or `5xx`, which end when a token is refreshed or a server recovers. A load in cooldown never gets retried. ``ImagePipeline/FailureCooldown/statusCodes`` changes the list.
+- **It's kept in memory.** The window has an end, the pipeline keeps about a thousand URLs at most, and an app launch starts with none.
+
+> Important: A cooldown trades a wasted request for a stale answer. An image uploaded during the window stays missing until the window ends, a single `404` from a misconfigured CDN takes the image away for all of it, and the app can't see the URL recover. Keep the duration in minutes, not hours.
+
+### Asking Again
+
+A user who taps Retry has to reach the server, or the button will look broken. `ImageLoader.reload()` in `NukeUI` skips the cooldown on its own. To do the same for any other load, use ``ImageRequest/Options-swift.struct/skipFailureCooldown``, which keeps using the caches, or ``ImageRequest/Options-swift.struct/reloadIgnoringCachedData``, which doesn't:
+
+```swift
+let request = ImageRequest(url: url, options: [.skipFailureCooldown])
+let image = try await pipeline.image(for: request)
+```
+
+When something changes that can bring all of the images back, such as the user signing in to another account, end the cooldown for every URL:
+
+```swift
+await pipeline.resetFailureCooldown()
+```
+
 ## Monitoring Network Requests
 
 The pipeline can be used with [Pulse](https://github.com/kean/Pulse) for monitoring network traffic.

Documentation/Nuke.docc/Essentials/troubleshooting.md

@@ -36,6 +36,7 @@ The URLs in the lines have no query, which can carry a token. ``ImageTask/Metric
 | `decoderNotRegistered` | The pipeline has a custom ``ImagePipeline/Configuration-swift.struct/makeImageDecoder`` that returned `nil` | Fall back to ``ImageDecoders/Default`` |
 | `Response status code was unacceptable` | The server answered with an error | Check the URL. ``DataLoader`` rejects everything but `2xx` |
 | `NSURLErrorDomain` and a code | `URLSession` failed. `-1009` is no connection, `-1004` is a refused connection, `-1022` is App Transport Security, and `-1001` is a timeout | See `URLError.Code` |
+| `urlInCooldown` | The URL failed with a `404` or a `410` a moment ago, and ``ImagePipeline/Configuration-swift.struct/failureCooldown`` is on, so the pipeline didn't request it again | Expected. See <doc:loading-data#Failure-Cooldown> to retry it |
 | `dataMissingInCache` | The request has ``ImageRequest/Options-swift.struct/returnCacheDataDontLoad``, and the image isn't cached | Expected, unless you meant to load it |
 | `cancelled` | The view went away, or something called `cancel()`. A cell that scrolls off the screen does it | Expected. To keep the download going when a `LazyImage` disappears, use `onDisappear(.lowerPriority)` |
 | `success`, and the view is blank | The image loaded. The problem is in the view | `LazyImage(url:)` shows the image at its natural size. Check its frame, and call `resizable()` in the `content` closure |

Documentation/Nuke.docc/Extensions/ImagePipeline-Extension.md

@@ -151,3 +151,5 @@ do {
 ### Error Handling
 
 - ``Error``
+- ``FailureCooldown``
+- ``resetFailureCooldown()``

Documentation/Nuke.docc/Extensions/ImagePipelineConfiguration-Extension.md

@@ -35,6 +35,7 @@ To learn more about caching, see <doc:caching>.
 - ``isProgressiveDecodingEnabled``
 - ``progressiveDecodingInterval``
 - ``isResumableDataEnabled``
+- ``failureCooldown``
 
 ### Diagnostics
 

Sources/Nuke/Diagnostics/DiagnosticsSummaries.swift

@@ -35,7 +35,7 @@ extension ImageTask.Metrics.ImageSummary {
 extension ImagePipeline.Diagnostics.ErrorSummary {
     init(_ error: ImagePipeline.Error) {
         let underlying: (any Swift.Error)? = switch error {
-        case .dataLoadingFailed(let error), .decodingFailed(_, _, let error), .processingFailed(_, _, let error): error
+        case .dataLoadingFailed(let error), .urlInCooldown(let error), .decodingFailed(_, _, let error), .processingFailed(_, _, let error): error
         default: nil
         }
         let nsError = underlying.map { $0 as NSError }
@@ -49,6 +49,7 @@ extension ImagePipeline.Error {
         switch self {
         case .dataMissingInCache: "dataMissingInCache"
         case .dataLoadingFailed: "dataLoadingFailed"
+        case .urlInCooldown: "urlInCooldown"
         case .dataIsEmpty: "dataIsEmpty"
         case .decoderNotRegistered: "decoderNotRegistered"
         case .decodingFailed: "decodingFailed"
@@ -138,7 +139,8 @@ extension ImageRequest.Options {
         (.disableDiskCacheWrites, "disableDiskCacheWrites"),
         (.returnCacheDataDontLoad, "returnCacheDataDontLoad"),
         (.skipDecompression, "skipDecompression"),
-        (.skipDataLoadingQueue, "skipDataLoadingQueue")
+        (.skipDataLoadingQueue, "skipDataLoadingQueue"),
+        (.skipFailureCooldown, "skipFailureCooldown")
     ]
 
     var diagnosticsNames: [String] {

Sources/Nuke/Diagnostics/ImagePipeline+Diagnostics.swift

@@ -426,6 +426,9 @@ extension ImagePipeline.Diagnostics {
         case file
         /// The `data` or `image` closure of the request.
         case closure
+        /// Nothing: the URL is in cooldown, so the download ended before it
+        /// started. See ``ImagePipeline/Configuration-swift.struct/failureCooldown``.
+        case failureCooldown
         case unknown
     }
 

Sources/Nuke/ImageRequest.swift

@@ -345,7 +345,9 @@ public struct ImageRequest: CustomStringConvertible, Sendable, ExpressibleByStri
         ///
         /// This option only works with ``ImageCaching`` and ``DataCaching``, but not
         /// `URLCache`. If you want to ignore `URLCache`, initialize the request
-        /// with `URLRequest` with the respective policy.
+        /// with `URLRequest` with the respective policy. It also ignores
+        /// ``ImagePipeline/Configuration-swift.struct/failureCooldown``, as a
+        /// failure the pipeline remembers is cached data too.
         public static let reloadIgnoringCachedData: Options = [.disableMemoryCacheReads, .disableDiskCacheReads]
 
         /// Use existing cache data and fail if no cached data is available.
@@ -361,6 +363,11 @@ public struct ImageRequest: CustomStringConvertible, Sendable, ExpressibleByStri
         /// - important: If there is an outstanding task for loading the same
         /// resource but without this option, a new task will be created.
         public static let skipDataLoadingQueue = Options(rawValue: 1 << 6)
+
+        /// Requests the URL even if it is in cooldown, while still using the
+        /// caches (see ``ImagePipeline/Configuration-swift.struct/failureCooldown``).
+        /// Use it when the user asks for the image again.
+        public static let skipFailureCooldown = Options(rawValue: 1 << 7)
     }
 
     /// A key used in `userInfo` for providing custom request options.

Sources/Nuke/Pipeline/ImagePipeline+Configuration.swift

@@ -139,6 +139,38 @@ extension ImagePipeline {
         /// `Last-Modified`). Resumable downloads are enabled by default.
         public var isResumableDataEnabled = true
 
+        /// Stops the pipeline from requesting a URL for a while after the
+        /// server said the image isn't there. `nil` by default, so every load
+        /// that misses the caches reaches the data loader.
+        ///
+        /// A dead URL in a feed otherwise costs a request every time a cell
+        /// shows it. With a cooldown, the loads of that URL fail with
+        /// ``ImagePipeline/Error/urlInCooldown(error:)`` until it's over,
+        /// without going to the network:
+        ///
+        /// ```swift
+        /// let pipeline = ImagePipeline {
+        ///     $0.failureCooldown = .init(duration: 300)
+        /// }
+        /// ```
+        ///
+        /// The cooldown only applies to URLs, after the caches: a cached image
+        /// is still served. It counts `404` and `410` responses by default,
+        /// and never a connection failure, a timeout, or a cancellation. See
+        /// ``ImagePipeline/FailureCooldown/statusCodes``. The pipeline keeps
+        /// the URLs in memory, up to about a thousand of them, and each
+        /// pipeline has its own.
+        ///
+        /// - important: A cooldown trades a wasted request for a stale answer.
+        /// An image that gets uploaded during the window stays missing until
+        /// the window ends, and a server that sends `404` by mistake is
+        /// believed for all of it. Keep the duration short, because the app
+        /// can't see the URL recover. `ImageLoader.reload()` in `NukeUI` and
+        /// ``ImageRequest/Options-swift.struct/skipFailureCooldown`` ignore the
+        /// cooldown, and ``ImagePipeline/resetFailureCooldown()`` ends it. See
+        /// <doc:loading-data#Failure-Cooldown>.
+        public var failureCooldown: FailureCooldown?
+
         /// If enabled, the pipeline will load the local resources (`file` and
         /// `data` schemes) inline without using the data loader. By default, `true`.
         public var isLocalResourcesSupportEnabled = true

Sources/Nuke/Pipeline/ImagePipeline+Error.swift

@@ -18,6 +18,11 @@ extension ImagePipeline {
         case dataMissingInCache
         /// Data loader failed to load image data with a wrapped error.
         case dataLoadingFailed(error: Swift.Error)
+        /// The pipeline didn't request the URL because it failed recently and
+        /// is in cooldown (see ``ImagePipeline/Configuration-swift.struct/failureCooldown``).
+        /// The error is the one the failure ended with, which
+        /// ``ImagePipeline/Error/dataLoadingError`` returns as well.
+        case urlInCooldown(error: Swift.Error)
         /// Data loader returned empty data.
         case dataIsEmpty
         /// No decoder registered for the given data.
@@ -58,7 +63,7 @@ extension ImagePipeline.Error {
     /// Returns underlying data loading error.
     public var dataLoadingError: Swift.Error? {
         switch self {
-        case .dataLoadingFailed(let error):
+        case .dataLoadingFailed(let error), .urlInCooldown(let error):
             return error
         default:
             return nil
@@ -75,6 +80,8 @@ extension ImagePipeline.Error {
             return "Failed to load data from cache and download is disabled."
         case let .dataLoadingFailed(error):
             return "Failed to load image data. Underlying error: \(error)."
+        case let .urlInCooldown(error):
+            return "Didn't request the URL because it failed recently and is in cooldown. Underlying error: \(error)."
         case .dataIsEmpty:
             return "Data loader returned empty data."
         case let .decoderNotRegistered(context):

Sources/Nuke/Pipeline/ImagePipeline+FailureCooldown.swift

@@ -0,0 +1,133 @@
+// The MIT License (MIT)
+//
+// Copyright (c) 2015-2026 Alexander Grebenyuk (github.com/kean).
+
+import Foundation
+
+extension ImagePipeline {
+    /// How long the pipeline stops requesting a URL after the server said the
+    /// image isn't there, and which responses say so.
+    ///
+    /// Set it with ``ImagePipeline/Configuration-swift.struct/failureCooldown``.
+    public struct FailureCooldown: Sendable {
+        /// How long a URL stays in cooldown after the failure, in seconds.
+        public var duration: TimeInterval
+
+        /// The HTTP status codes that put a URL in cooldown. `404` and `410` by
+        /// default.
+        ///
+        /// Think twice before adding to it. `401` and `403` end when a token is
+        /// refreshed, and `429` and the `5xx` codes when the server recovers, so
+        /// none of them say anything about the URL.
+        ///
+        /// The pipeline reads the code from
+        /// ``DataLoader/Error/statusCodeUnacceptable(_:)``, which is what
+        /// ``DataLoader`` reports by default. A custom ``DataLoading``, or a
+        /// custom `validate` closure, has to report the same error for a URL to
+        /// enter the cooldown.
+        public var statusCodes: Set<Int>
+
+        // Keeps running while the device sleeps, and doesn't follow the system
+        // date, which would stretch the window by however far it moved back.
+        var now: @Sendable () -> TimeInterval = {
+            TimeInterval(clock_gettime_nsec_np(CLOCK_MONOTONIC)) / 1_000_000_000
+        }
+
+        /// Initializes the cooldown with the given duration.
+        ///
+        /// - parameters:
+        ///   - duration: How long a URL stays in cooldown after the failure, in
+        ///   seconds.
+        ///   - statusCodes: The HTTP status codes that put a URL in cooldown.
+        public init(duration: TimeInterval, statusCodes: Set<Int> = [404, 410]) {
+            self.duration = duration
+            self.statusCodes = statusCodes
+        }
+    }
+
+    /// Takes every URL out of the cooldown, so the next load of each of them
+    /// reaches the data loader. Does nothing if
+    /// ``ImagePipeline/Configuration-swift.struct/failureCooldown`` isn't set.
+    ///
+    /// Use it when something happened that can bring the images back, such as
+    /// the user signing in to another account. To retry one image, use
+    /// ``ImageRequest/Options-swift.struct/skipFailureCooldown``.
+    public func resetFailureCooldown() {
+        failureRegistry?.removeAll()
+    }
+}
+
+/// The URLs in cooldown. It has no lock because the fetch tasks that use it all
+/// run on the pipeline's actor, and it's a class of its own so that a pipeline
+/// with no cooldown pays for one `nil` check per download and nothing else.
+@ImagePipelineActor
+final class FailureRegistry {
+    /// Enough for the dead URLs of a long feed, at a few hundred bytes each. A
+    /// URL pushed out by it is only requested again sooner than it had to be.
+    nonisolated static var countLimit: Int { 1024 }
+
+    private struct Entry {
+        let error: any Error
+        let expiresAt: TimeInterval
+    }
+
+    private let cooldown: ImagePipeline.FailureCooldown
+    private var entries: [String: Entry] = [:]
+
+    var count: Int { entries.count }
+
+    nonisolated init(cooldown: ImagePipeline.FailureCooldown) {
+        self.cooldown = cooldown
+    }
+
+    /// Returns the failure that put the URL of the request in cooldown, or `nil`
+    /// if the request can go ahead.
+    func failure(for request: ImageRequest) -> (any Error)? {
+        // The key is the URL and not the image ID: a custom `imageID` says what
+        // to call the image, and it's the URL that is gone.
+        guard !entries.isEmpty, let key = request.originalImageID, let entry = entries[key] else {
+            return nil
+        }
+        guard !request.options.contains(.skipFailureCooldown), !request.options.contains(.reloadIgnoringCachedData) else {
+            return nil
+        }
+        guard entry.expiresAt > cooldown.now() else {
+            entries[key] = nil
+            return nil
+        }
+        return entry.error
+    }
+
+    /// Records how a download ended: a success takes the URL out of the
+    /// cooldown, and a failure with one of the status codes starts it over.
+    func didFinishLoading(_ request: ImageRequest, error: ImagePipeline.Error?) {
+        guard let key = request.originalImageID else { return }
+        guard let error else {
+            entries[key] = nil
+            return
+        }
+        guard case .statusCodeUnacceptable(let statusCode)? = error.dataLoadingError as? DataLoader.Error,
+              cooldown.statusCodes.contains(statusCode) else {
+            return
+        }
+        if entries.count >= Self.countLimit, entries[key] == nil {
+            makeRoom()
+        }
+        entries[key] = Entry(error: DataLoader.Error.statusCodeUnacceptable(statusCode), expiresAt: cooldown.now() + cooldown.duration)
+    }
+
+    func removeAll() {
+        entries.removeAll()
+    }
+
+    /// Drops what has expired, and failing that, what would have expired first.
+    /// A scan: it only runs after a failed download, and only with a full
+    /// registry, so a sorted structure wouldn't pay for itself.
+    private func makeRoom() {
+        let now = cooldown.now()
+        entries = entries.filter { $0.value.expiresAt > now }
+        if entries.count >= Self.countLimit, let oldest = entries.min(by: { $0.value.expiresAt < $1.value.expiresAt }) {
+            entries[oldest.key] = nil
+        }
+    }
+}

Sources/Nuke/Pipeline/ImagePipeline.swift

@@ -52,6 +52,10 @@ public final class ImagePipeline: Sendable {
     private let tasksFetchOriginalImage: TaskPool<TaskFetchOriginalImageKey, ImageResponse, Error>
     private let tasksFetchOriginalData: TaskPool<TaskFetchOriginalDataKey, (Data, URLResponse?), Error>
 
+    /// The URLs in cooldown. `nil` unless
+    /// ``Configuration-swift.struct/failureCooldown`` is set.
+    let failureRegistry: FailureRegistry?
+
     private(set) var isInvalidated = false
 
     nonisolated private var nextTaskID: UInt64 {
@@ -102,6 +106,11 @@ public final class ImagePipeline: Sendable {
         self.tasksLoadImage = TaskPool(isCoalescingEnabled)
         self.tasksFetchOriginalImage = TaskPool(isCoalescingEnabled)
         self.tasksFetchOriginalData = TaskPool(isCoalescingEnabled)
+        if let cooldown = configuration.failureCooldown {
+            self.failureRegistry = FailureRegistry(cooldown: cooldown)
+        } else {
+            self.failureRegistry = nil
+        }
         let isRecording = configuration.isDiagnosticsEnabled || configuration.isDiagnosticsLoggingEnabled
         self.recorder = isRecording ? Diagnostics.Recorder(pipelineID: id, signpostLog: configuration.signpostLog, isLoggingEnabled: configuration.isDiagnosticsLoggingEnabled) : nil