Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 2 additions & 5 deletions .JuliaFormatter.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
style = "blue"
align_assignment = true
align_struct_field = true
align_conditional = true
align_pair_arrow = true
align_matrix = true
format_docstrings = true
format_markdown = true
2 changes: 0 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@ repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v3.2.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-files
- id: check-merge-conflict
Expand Down
71 changes: 23 additions & 48 deletions DifferentiationInterface/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,92 +5,67 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]
## [Unreleased](https://github.com/JuliaDiff/DifferentiationInterface.jl/compare/DifferentiationInterface-v0.7.5...main)

## [0.7.5]
## [0.7.5](https://github.com/JuliaDiff/DifferentiationInterface.jl/compare/DifferentiationInterface-v0.7.4...DifferentiationInterface-v0.7.5)

### Added

- Support forward-mode Mooncake with `AutoMooncakeForward` ([#813])
- Support forward-mode Mooncake with `AutoMooncakeForward` ([#813](https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/813))

## [0.7.4]
## [0.7.4](https://github.com/JuliaDiff/DifferentiationInterface.jl/compare/DifferentiationInterface-v0.7.3...DifferentiationInterface-v0.7.4)

### Added

- Make `AutoForwardFromPrimitive` and `AutoReverseFromPrimitive` public ([#825])
- Make `AutoForwardFromPrimitive` and `AutoReverseFromPrimitive` public ([#825](https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/825))

### Fixed

- Replace `one` with `oneunit` in basis computation ([#826])
- Replace `one` with `oneunit` in basis computation ([#826](https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/826))

## [0.7.3]
## [0.7.3](https://github.com/JuliaDiff/DifferentiationInterface.jl/compare/DifferentiationInterface-v0.7.2...DifferentiationInterface-v0.7.3)

### Fixed

- Bump compat for SparseConnectivityTracer v1 ([#823])
- Bump compat for SparseConnectivityTracer v1 ([#823](https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/823))

## [0.7.2]
## [0.7.2](https://github.com/JuliaDiff/DifferentiationInterface.jl/compare/DifferentiationInterface-v0.7.1...DifferentiationInterface-v0.7.2)

### Feat

- Backend switching for Mooncake ([#768])
- Backend switching for Mooncake ([#768](https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/768))

### Fixed

- Speed up sparse preparation for GPU arrays ([#818])
- Speed up sparse preparation for GPU arrays ([#818](https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/818))

## [0.7.1]
## [0.7.1](https://github.com/JuliaDiff/DifferentiationInterface.jl/compare/DifferentiationInterface-v0.7.0...DifferentiationInterface-v0.7.1)

### Feat

- Use Mooncake's internal copy utilities ([#809])
- Use Mooncake's internal copy utilities ([#809](https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/809))

### Fixed

- Take `absstep` into account for FiniteDiff ([#812])
- Make basis work for `CuArray` ([#810])
- Take `absstep` into account for FiniteDiff ([#812](https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/812))
- Make basis work for `CuArray` ([#810](https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/810))

## [0.7.0]
## [0.7.0](https://github.com/JuliaDiff/DifferentiationInterface.jl/compare/DifferentiationInterface-v0.6.54...DifferentiationInterface-v0.7.0)

### Changed

- Preparation is now strict by default ([#799])
- New Arxiv preprint for citation ([#795])
- Preparation is now strict by default ([#799](https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/799))
- New Arxiv preprint for citation ([#795](https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/795))

## [0.6.54] - 2025-05-11
## [0.6.54](https://github.com/JuliaDiff/DifferentiationInterface.jl/compare/DifferentiationInterface-v0.6.53...DifferentiationInterface-v0.6.54) - 2025-05-11

### Added

- Dependency compat bounds for extras ([#790])
- Error hints for Enzyme ([#788])
- Dependency compat bounds for extras ([#790](https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/790))
- Error hints for Enzyme ([#788](https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/788))

## [0.6.53] - 2025-05-07
## [0.6.53](https://github.com/JuliaDiff/DifferentiationInterface.jl/compare/DifferentiationInterface-v0.6.52...DifferentiationInterface-v0.6.53) - 2025-05-07

### Changed

- Allocate Enzyme shadow memory during preparation ([#782])

[unreleased]: https://github.com/JuliaDiff/DifferentiationInterface.jl/compare/DifferentiationInterface-v0.7.5...main
[0.7.5]: https://github.com/JuliaDiff/DifferentiationInterface.jl/compare/DifferentiationInterface-v0.7.4...DifferentiationInterface-v0.7.5
[0.7.4]: https://github.com/JuliaDiff/DifferentiationInterface.jl/compare/DifferentiationInterface-v0.7.3...DifferentiationInterface-v0.7.4
[0.7.3]: https://github.com/JuliaDiff/DifferentiationInterface.jl/compare/DifferentiationInterface-v0.7.2...DifferentiationInterface-v0.7.3
[0.7.2]: https://github.com/JuliaDiff/DifferentiationInterface.jl/compare/DifferentiationInterface-v0.7.1...DifferentiationInterface-v0.7.2
[0.7.1]: https://github.com/JuliaDiff/DifferentiationInterface.jl/compare/DifferentiationInterface-v0.7.0...DifferentiationInterface-v0.7.1
[0.7.0]: https://github.com/JuliaDiff/DifferentiationInterface.jl/compare/DifferentiationInterface-v0.6.54...DifferentiationInterface-v0.7.0
[0.6.54]: https://github.com/JuliaDiff/DifferentiationInterface.jl/compare/DifferentiationInterface-v0.6.53...DifferentiationInterface-v0.6.54
[0.6.53]: https://github.com/JuliaDiff/DifferentiationInterface.jl/compare/DifferentiationInterface-v0.6.52...DifferentiationInterface-v0.6.53

[#826]: https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/826
[#825]: https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/825
[#823]: https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/823
[#818]: https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/818
[#813]: https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/813
[#812]: https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/812
[#810]: https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/810
[#809]: https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/809
[#799]: https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/799
[#795]: https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/795
[#790]: https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/790
[#788]: https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/788
[#782]: https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/782
[#768]: https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/768
- Allocate Enzyme shadow memory during preparation ([#782](https://github.com/JuliaDiff/DifferentiationInterface.jl/pull/782))
64 changes: 33 additions & 31 deletions DifferentiationInterface/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,12 @@
[![Build Status](https://github.com/JuliaDiff/DifferentiationInterface.jl/actions/workflows/Test.yml/badge.svg?branch=main)](https://github.com/JuliaDiff/DifferentiationInterface.jl/actions/workflows/Test.yml?query=branch%3Amain)
[![Coverage](https://codecov.io/gh/JuliaDiff/DifferentiationInterface.jl/branch/main/graph/badge.svg?flag=DI)](https://app.codecov.io/gh/JuliaDiff/DifferentiationInterface.jl)
[![Code Style: Blue](https://img.shields.io/badge/code%20style-blue-4495d1.svg)](https://github.com/JuliaDiff/BlueStyle)
[![ColPrac: Contributor's Guide on Collaborative Practices for Community Packages](https://img.shields.io/badge/ColPrac-Contributor's%20Guide-blueviolet)](https://github.com/SciML/ColPrac)
[![ColPrac: Contributor's Guide on Collaborative Practices for Community Packages](https://img.shields.io/badge/ColPrac-Contributor%27s%20Guide-blueviolet)](https://github.com/SciML/ColPrac)
[![DOI](https://zenodo.org/badge/740973714.svg)](https://zenodo.org/doi/10.5281/zenodo.11092033)

| Package | Docs |
| :--------------------------: | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: |
| DifferentiationInterface | [![Stable](https://img.shields.io/badge/docs-stable-blue.svg)](https://juliadiff.org/DifferentiationInterface.jl/DifferentiationInterface/stable/) [![Dev](https://img.shields.io/badge/docs-dev-blue.svg)](https://juliadiff.org/DifferentiationInterface.jl/DifferentiationInterface/dev/) |
| Package | Docs |
|:----------------------------:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
| DifferentiationInterface | [![Stable](https://img.shields.io/badge/docs-stable-blue.svg)](https://juliadiff.org/DifferentiationInterface.jl/DifferentiationInterface/stable/) [![Dev](https://img.shields.io/badge/docs-dev-blue.svg)](https://juliadiff.org/DifferentiationInterface.jl/DifferentiationInterface/dev/) |
| DifferentiationInterfaceTest | [![Stable](https://img.shields.io/badge/docs-stable-blue.svg)](https://juliadiff.org/DifferentiationInterface.jl/DifferentiationInterfaceTest/stable/) [![Dev](https://img.shields.io/badge/docs-dev-blue.svg)](https://juliadiff.org/DifferentiationInterface.jl/DifferentiationInterfaceTest/dev/) |

An interface to various automatic differentiation (AD) backends in Julia.
Expand All @@ -19,31 +19,31 @@ An interface to various automatic differentiation (AD) backends in Julia.

This package provides a unified syntax to differentiate functions, including:

- First- and second-order operators (gradients, Jacobians, Hessians and more)
- In-place and out-of-place differentiation
- Preparation mechanism (e.g. to pre-allocate a cache or record a tape)
- Built-in sparsity handling
- Thorough validation on standard inputs and outputs (numbers, vectors, matrices)
- Testing and benchmarking utilities accessible to users with [DifferentiationInterfaceTest](https://github.com/JuliaDiff/DifferentiationInterface.jl/tree/main/DifferentiationInterfaceTest)
- First- and second-order operators (gradients, Jacobians, Hessians and more)
- In-place and out-of-place differentiation
- Preparation mechanism (e.g. to pre-allocate a cache or record a tape)
- Built-in sparsity handling
- Thorough validation on standard inputs and outputs (numbers, vectors, matrices)
- Testing and benchmarking utilities accessible to users with [DifferentiationInterfaceTest](https://github.com/JuliaDiff/DifferentiationInterface.jl/tree/main/DifferentiationInterfaceTest)

## Compatibility

We support the following backends defined by [ADTypes.jl](https://github.com/SciML/ADTypes.jl):

- [ChainRulesCore.jl](https://github.com/JuliaDiff/ChainRulesCore.jl)
- [Diffractor.jl](https://github.com/JuliaDiff/Diffractor.jl) (currently broken)
- [Enzyme.jl](https://github.com/EnzymeAD/Enzyme.jl) (see below)
- [FastDifferentiation.jl](https://github.com/brianguenter/FastDifferentiation.jl)
- [FiniteDiff.jl](https://github.com/JuliaDiff/FiniteDiff.jl)
- [FiniteDifferences.jl](https://github.com/JuliaDiff/FiniteDifferences.jl)
- [ForwardDiff.jl](https://github.com/JuliaDiff/ForwardDiff.jl)
- [GTPSA.jl](https://github.com/bmad-sim/GTPSA.jl)
- [Mooncake.jl](https://github.com/chalk-lab/Mooncake.jl)
- [PolyesterForwardDiff.jl](https://github.com/JuliaDiff/PolyesterForwardDiff.jl)
- [ReverseDiff.jl](https://github.com/JuliaDiff/ReverseDiff.jl)
- [Symbolics.jl](https://github.com/JuliaSymbolics/Symbolics.jl)
- [Tracker.jl](https://github.com/FluxML/Tracker.jl)
- [Zygote.jl](https://github.com/FluxML/Zygote.jl)
- [ChainRulesCore.jl](https://github.com/JuliaDiff/ChainRulesCore.jl)
- [Diffractor.jl](https://github.com/JuliaDiff/Diffractor.jl) (currently broken)
- [Enzyme.jl](https://github.com/EnzymeAD/Enzyme.jl) (see below)
- [FastDifferentiation.jl](https://github.com/brianguenter/FastDifferentiation.jl)
- [FiniteDiff.jl](https://github.com/JuliaDiff/FiniteDiff.jl)
- [FiniteDifferences.jl](https://github.com/JuliaDiff/FiniteDifferences.jl)
- [ForwardDiff.jl](https://github.com/JuliaDiff/ForwardDiff.jl)
- [GTPSA.jl](https://github.com/bmad-sim/GTPSA.jl)
- [Mooncake.jl](https://github.com/chalk-lab/Mooncake.jl)
- [PolyesterForwardDiff.jl](https://github.com/JuliaDiff/PolyesterForwardDiff.jl)
- [ReverseDiff.jl](https://github.com/JuliaDiff/ReverseDiff.jl)
- [Symbolics.jl](https://github.com/JuliaSymbolics/Symbolics.jl)
- [Tracker.jl](https://github.com/FluxML/Tracker.jl)
- [Zygote.jl](https://github.com/FluxML/Zygote.jl)

> [!CAUTION]
> Note that in some cases, going through DifferentiationInterface.jl might be slower or cause more errors than a direct call to the backend's API. This is especially true for Enzyme.jl, whose handling of activities and multiple arguments is not fully supported here. We are working on this challenge, and welcome any suggestions or contributions. Meanwhile, if differentiation fails or takes too long, consider using Enzyme.jl through its [native API](https://enzymead.github.io/Enzyme.jl/stable/) instead.
Expand All @@ -63,25 +63,27 @@ To install the development version, run this instead:
```julia
using Pkg

Pkg.add(
Pkg.add(;
url="https://github.com/JuliaDiff/DifferentiationInterface.jl",
subdir="DifferentiationInterface"
subdir="DifferentiationInterface",
)
```

## Example

```julia
using DifferentiationInterface
import ForwardDiff, Enzyme, Zygote # AD backends you want to use
using ForwardDiff: ForwardDiff
using Enzyme: Enzyme
using Zygote: Zygote # AD backends you want to use

f(x) = sum(abs2, x)

x = [1.0, 2.0]

value_and_gradient(f, AutoForwardDiff(), x) # returns (5.0, [2.0, 4.0]) with ForwardDiff.jl
value_and_gradient(f, AutoEnzyme(), x) # returns (5.0, [2.0, 4.0]) with Enzyme.jl
value_and_gradient(f, AutoZygote(), x) # returns (5.0, [2.0, 4.0]) with Zygote.jl
value_and_gradient(f, AutoEnzyme(), x) # returns (5.0, [2.0, 4.0]) with Enzyme.jl
value_and_gradient(f, AutoZygote(), x) # returns (5.0, [2.0, 4.0]) with Zygote.jl
```

To improve your performance by up to several orders of magnitude compared to this example, take a look at the tutorial and its section on operator preparation.
Expand All @@ -90,8 +92,8 @@ To improve your performance by up to several orders of magnitude compared to thi

Whenever you refer to this package or the ideas it contains, please cite:

1. our preprint [*A Common Interface for Automatic Differentiation*](https://arxiv.org/abs/2505.05542);
2. our inspiration [AbstractDifferentiation.jl](https://github.com/JuliaDiff/AbstractDifferentiation.jl).
1. our preprint [*A Common Interface for Automatic Differentiation*](https://arxiv.org/abs/2505.05542);
2. our inspiration [AbstractDifferentiation.jl](https://github.com/JuliaDiff/AbstractDifferentiation.jl).

You can use the provided [`CITATION.cff`](https://github.com/JuliaDiff/DifferentiationInterface.jl/blob/main/CITATION.cff) file or the following BibTeX entries:

Expand Down
10 changes: 5 additions & 5 deletions DifferentiationInterface/docs/src/assets/logo.jl
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,11 @@ end
# ╔═╡ d1b44dac-8487-4581-82fd-ece0ce89f8e2
begin
const purple = Luxor.julia_purple
const red = Luxor.julia_red
const green = Luxor.julia_green
const blue = Luxor.julia_blue
const white = RGB(1, 1, 1)
const black = RGB(0, 0, 0)
const red = Luxor.julia_red
const green = Luxor.julia_green
const blue = Luxor.julia_blue
const white = RGB(1, 1, 1)
const black = RGB(0, 0, 0)
const colors = (purple, red, green)
end;

Expand Down
19 changes: 10 additions & 9 deletions DifferentiationInterface/docs/src/dev_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@ It is not part of the public API and the content below may become outdated, in w

The package is structured around 8 [operators](@ref Operators):

- [`derivative`](@ref)
- [`second_derivative`](@ref)
- [`gradient`](@ref)
- [`jacobian`](@ref)
- [`hessian`](@ref)
- [`pushforward`](@ref)
- [`pullback`](@ref)
- [`hvp`](@ref)
- [`derivative`](@ref)
- [`second_derivative`](@ref)
- [`gradient`](@ref)
- [`jacobian`](@ref)
- [`hessian`](@ref)
- [`pushforward`](@ref)
- [`pullback`](@ref)
- [`hvp`](@ref)

Most operators have 4 variants, which look like this in the first order: `operator`, `operator!`, `value_and_operator`, `value_and_operator!`.

Expand All @@ -39,14 +39,15 @@ In the main package, you should define a new struct `SuperDiffBackend` which sub
You also have to define [`ADTypes.mode`](@extref) and [`DifferentiationInterface.inplace_support`](@ref) on `SuperDiffBackend`.

!!! info

In the end, this backend struct will need to be contributed to [ADTypes.jl](https://github.com/SciML/ADTypes.jl).
However, putting it in the DifferentiationInterface.jl PR is a good first step for debugging.

In a [package extension](https://pkgdocs.julialang.org/v1/creating-packages/#Conditional-loading-of-code-in-packages-(Extensions)) named `DifferentiationInterfaceSuperDiffExt`, you need to implement at least [`pushforward`](@ref) or [`pullback`](@ref) (and their variants).
The exact requirements depend on the differentiation mode you chose:

| backend mode | pushforward necessary | pullback necessary |
| :------------------------------------------------ | :-------------------- | :----------------- |
|:------------------------------------------------- |:--------------------- |:------------------ |
| [`ADTypes.ForwardMode`](@extref ADTypes) | yes | no |
| [`ADTypes.ReverseMode`](@extref ADTypes) | no | yes |
| [`ADTypes.ForwardOrReverseMode`](@extref ADTypes) | yes | yes |
Expand Down
Loading
Loading