feat: backend switching for Mooncake

[AstitvaAggarwal]

Define Mooncake.rrule!! for DI.DifferentiateWith.

[codecov]

Codecov Report

Attention: Patch coverage is 96.82540% with 2 lines in your changes missing coverage. Please review.

Project coverage is 97.87%. Comparing base (eeb4c86) to head (1e85f17).

Files with missing lines	Patch %	Lines
...ationInterface/test/Back/DifferentiateWith/test.jl	88.23%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #768      +/-   ##
==========================================
- Coverage   97.88%   97.87%   -0.01%     
==========================================
  Files         128      129       +1     
  Lines        7694     7754      +60     
==========================================
+ Hits         7531     7589      +58     
- Misses        163      165       +2     

Flag	Coverage Δ
DI	98.72% <96.82%> (-0.03%)	⬇️
DIT	95.61% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[AstitvaAggarwal]

removed the code that piggybacks off the Chainrules wrapper. This is specifically now a Mooncake generic rule which handles backend switching.

[gdalle]

Thanks for this first draft!
I think there are some changes necessary, and most importantly you need to test it, first locally and then during CI (try not to run CI before having tested your changes locally, the process is very expensive since it tests a dozen different backends for like half an hour each).
For the testing, start with manual tests, and then once your code works you can add AutoMooncake() to this line

[AstitvaAggarwal]

sorry i got preoccupied with some other work, hence the incomplete PR. This would be on route now.

[gdalle]

Please keep in mind that every commit costs around 6 hours of CI budget. I suggest you make as many modifications as possible locally and add tests first before pushing

[gdalle]

Thanks, we're getting closer!
Unfortunately I think my existing tests are not enough to capture everything that can go wrong in a Mooncake rule. Perhaps the Mooncake test utilities should be brought in, or more sophisticated tests should be written.

[yebai]

Good work, @AstitvaAggarwal. We are getting there. I have just a few more quality-related comments on the documents, type stability, etc.

[yebai]

Thanks @AstitvaAggarwal for the nice work!

EDIT: I am happy with this PR, but still need @gdalle's blessing before merging.

[gdalle]

Very good work @AstitvaAggarwal, thank you so much for taking the time! My main remaining questions are about:

• support for tuples: do we keep it, do we remove it? It seems a bit arbitrary to have just tuples in addition to the "officially" supported Number and AbstractArray, especially if these can only be tuples of numbers
• testing code, which should not touch on so many Mooncake internals

To lighten the load on you, I'll make the suggested modifications myself, and you can tell me what you think after! Thanks again for your patience.

[gdalle]

The issue is that we're testing DifferentiateWith(f, substitute_backend) with substitute_backend = AutoFiniteDiff(), aka a forward-mode backend. I think it should work with DifferentiateWith(f, AutoEnzyme())?

[gdalle]

Resolved by removing tuples for the time being

[gdalle]

I find it a bit weird to have this union of Number, AbstractArray (two types which are theoretically supported for x inputs in DI) and then just Tuple (which is not officially part of the supported inputs). Why not also NamedTuple for instance? Is it better if we just say Any? Or restrict to Number and AbstractArray for the time being?