copy: include backend context in remote read errors

[Rohithmatham12]

Summary

• Carry explicit remote-source context through core.ReadResp and xs.CoiRes for copy/ETL source reads.
• Preserve provider/backend, remote bucket/object, operation, status code when available, and original cause via core.RemoteSourceError.
• Wrap only the actual GetObjReader failure at the backend boundary; later copy/transport errors are reported as copy failures with remote-source context.
• Keep reader behavior unchanged: this version removes the reader wrapper abstraction from the previous push.
• Keep copy transport/retry behavior unchanged; RGET remains only for stats.
• Add focused coverage for structured remote-source errors and for the AIS copy-reader path carrying remote-source context.

Why
This reworks the earlier #321 approach based on maintainer feedback. The previous versions either propagated a coarse RGET boolean or wrapped the reader itself. This version keeps the patch smaller by carrying explicit source context through the existing copy result path and attaching it where errors are reported.

Refs #309.

Testing

• GOCACHE=/private/tmp/aistore-backend-gocache GOMODCACHE=/private/tmp/aistore-backend-gomodcache go test ./core -run 'TestRemoteSource'
• GOCACHE=/private/tmp/aistore-backend-gocache GOMODCACHE=/private/tmp/aistore-backend-gomodcache go test ./ais -run 'TestCopyReaderPreservesRemoteSourceContext'
• GOCACHE=/private/tmp/aistore-backend-gocache GOMODCACHE=/private/tmp/aistore-backend-gomodcache go test ./xact/xs -run '^$'
• git diff --check

Not fully run locally:

• go test ./xact/xs starts but fails in unrelated local macOS sandbox memory-pressure setup (sys/mem: operation not permitted, extreme memory pressure) before reaching this change.

[Nahemah1022]

@Rohithmatham12 thanks, this is closer to the area I was asking about, but I don’t think we should merge it in this form. Propagating an RGET boolean is too coarse: it tells us a remote read was involved, but not which backend operation failed, which remote bucket/object was involved, or whether the failure actually came from the backend versus the target-to-target stream after the reader was opened.
The error context should be produced at the actual failure point and preserve structured details: provider/backend, remote bucket/object, operation, status/code when available, and the original cause. Also, the current test only covers the formatting helper; it does not exercise the copy-listrange remote-backend path in a real/integration setup.

[Rohithmatham12]

Thanks for the review. I reworked the branch to avoid the coarse RGET-boolean wrapping.

What changed in the latest push:

• backend context is now produced in core.GetROC, at the actual GetObjReader boundary
• core.RemoteReadError carries provider, remote bucket/object, operation, status code when available, and the original cause
• the returned remote reader wraps non-graceful stream read errors, so an unexpected EOF from the backend reader keeps backend context
• copy/xaction code no longer relabels errors later based on RGET; that flag remains only for stats
• added coverage for structured backend open/read errors and an AIS copy-reader regression that preserves a remote read error through the copy path

This should distinguish backend reader failures from target-to-target transport failures after the reader is opened, while keeping the copy transport/retry behavior unchanged.

[Nahemah1022]

I don’t think the reader wrapper is the right abstraction here. The goal is to preserve/report remote source context, not to change the reader behavior. Maybe simplify this by carrying explicit remote source context through the existing copy/transport result path and include that context where the error is reported. That should make the patch smaller and easier to reason about, while still satisfying the diagnostic goal.

[Rohithmatham12]

Thanks, that makes sense. I reworked the branch again to remove the reader wrapper abstraction.

Latest push changes:

• core.ReadResp now carries explicit RemoteSrc context for remote backend reads
• xs.CoiRes carries that context through the existing copy/transport result path
• GetObjReader failures are still wrapped at the actual backend boundary with operation/status/cause
• later copy/transport failures are reported as copy failures with remote-source context, rather than changing reader behavior or claiming every later error is a backend read failure
• RGET remains only for stats
• tests now cover the structured RemoteSourceError formatter and the AIS copy-reader path preserving RemoteSrc

This should be closer to the simpler shape you suggested.