Improve docs and document contexts (#469)

* Improve docs and document contexts

* One less section

Author: gdalle

DifferentiationInterface/README.md

@@ -17,20 +17,18 @@ An interface to various automatic differentiation (AD) backends in Julia.
 
 ## Goal
 
-This package provides a unified syntax to differentiate functions.
-
-## Features
+This package provides a unified syntax to differentiate functions, including:
 
 - First- and second-order operators (gradients, Jacobians, Hessians and more)
 - In-place and out-of-place differentiation
-- Preparation mechanism (e.g. to create a config or tape)
+- Preparation mechanism (e.g. to pre-allocate a cache or record a tape)
 - Built-in sparsity handling
 - Thorough validation on standard inputs and outputs (numbers, vectors, matrices)
 - Testing and benchmarking utilities accessible to users with [DifferentiationInterfaceTest](https://github.com/gdalle/DifferentiationInterface.jl/tree/main/DifferentiationInterfaceTest)
 
 ## Compatibility
 
-We support all of the backends defined by [ADTypes.jl](https://github.com/SciML/ADTypes.jl):
+We support the following backends defined by [ADTypes.jl](https://github.com/SciML/ADTypes.jl):
 
 - [ChainRulesCore.jl](https://github.com/JuliaDiff/ChainRulesCore.jl)
 - [Diffractor.jl](https://github.com/JuliaDiff/Diffractor.jl)
@@ -87,7 +85,7 @@ value_and_gradient(f, AutoEnzyme(),      x) # returns (5.0, [2.0, 4.0]) with Enz
 value_and_gradient(f, AutoZygote(),      x) # returns (5.0, [2.0, 4.0]) with Zygote.jl
 ```
 
-To improve your performance by up to several orders of magnitude compared to this example, take a look at the [DifferentiationInterface tutorial](https://gdalle.github.io/DifferentiationInterface.jl/DifferentiationInterface/stable/tutorial1/) and its section on operator preparation.
+To improve your performance by up to several orders of magnitude compared to this example, take a look at the tutorial and its section on operator preparation.
 
 ## Citation
 

DifferentiationInterface/docs/Project.toml

@@ -1,26 +1,16 @@
 [deps]
 ADTypes = "47edcb42-4c32-4615-8424-f2b9edc5f35b"
 BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
-ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
 DifferentiationInterface = "a0c0ee7d-e4b9-4e03-894e-1c5f64a51d63"
-Diffractor = "9f5e2b26-1114-432f-b630-d3fe2085c51c"
 Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
 DocumenterInterLinks = "d12716ef-a0f6-4df4-a9f1-a5a34e75c656"
 DocumenterMermaid = "a078cd44-4d9c-4618-b545-3ab9d77f9177"
 Enzyme = "7da242da-08ed-463a-9acd-ee780be4f1d9"
-FastDifferentiation = "eb9bf01b-bf85-4b60-bf87-ee5de06c00be"
-FiniteDiff = "6a86dc24-6348-571c-b903-95158fe2bd41"
-FiniteDifferences = "26cc04aa-876d-5657-8c51-4c34ba976000"
 ForwardDiff = "f6369f11-7733-5829-9624-2563aa707210"
 Markdown = "d6f4376e-aef5-505a-96c1-9c027394607a"
-PolyesterForwardDiff = "98d1487c-24ca-40b6-b7ab-df2af84e126b"
 PrettyTables = "08abe8d2-0d0c-5749-adfa-8a2ac140af0d"
-ReverseDiff = "37e2e3b7-166d-5795-8a7a-e32c996b4267"
 SparseConnectivityTracer = "9f842d2f-2579-4b1d-911e-f412cf18a3f5"
 SparseMatrixColorings = "0a514795-09f3-496d-8182-132a7b665d35"
-Symbolics = "0c5d862f-8b57-4792-8d23-62f2024744c7"
-Tapir = "07d77754-e150-4737-8c94-cd238a1fb45b"
-Tracker = "9f7883ad-71c0-57eb-9f7f-b5c9e6d3789c"
 Zygote = "e88e6eb3-aa80-5325-afca-941959d7151f"
 
 [compat]

DifferentiationInterface/docs/make.jl

@@ -6,17 +6,8 @@ using DocumenterMermaid
 using DocumenterInterLinks
 
 using ADTypes: ADTypes
-using Diffractor: Diffractor
 using Enzyme: Enzyme
-using FastDifferentiation: FastDifferentiation
-using FiniteDiff: FiniteDiff
-using FiniteDifferences: FiniteDifferences
 using ForwardDiff: ForwardDiff
-using PolyesterForwardDiff: PolyesterForwardDiff
-using ReverseDiff: ReverseDiff
-using Symbolics: Symbolics
-using Tapir: Tapir
-using Tracker: Tracker
 using Zygote: Zygote
 
 links = InterLinks(
@@ -35,9 +26,14 @@ makedocs(;
     format=Documenter.HTML(; assets=["assets/favicon.ico"]),
     pages=[
         "Home" => "index.md",
-        "Tutorials" => ["tutorial1.md", "tutorial2.md"],
-        "Reference" => ["operators.md", "backends.md", "api.md"],
-        "Advanced" => ["dev_guide.md", "implementations.md"],
+        "Tutorials" => ["tutorials/basic.md", "tutorials/advanced.md"],
+        "Explanation" => [
+            "explanation/operators.md",
+            "explanation/backends.md",
+            "explanation/advanced.md",
+        ],
+        "api.md",
+        "dev_guide.md",
     ],
     plugins=[links],
 )

DifferentiationInterface/docs/src/api.md

@@ -8,12 +8,16 @@ CollapsedDocStrings = true
 DifferentiationInterface
 ```
 
-## First order
+## Argument wrappers
 
 ```@docs
+Context
+Constant
 Tangents
 ```
 
+## First order
+
 ### Pushforward
 
 ```@docs

DifferentiationInterface/docs/src/backends.md

@@ -1,10 +1,7 @@
 # Dev guide
 
 This page is important reading if you want to contribute to DifferentiationInterface.jl.
-It is not part of the public API.
-
-!!! warning
-    The content below may become outdated, in which case you should refer to the source code as the ground truth.
+It is not part of the public API and the content below may become outdated, in which case you should refer to the source code as the ground truth.
 
 ## General principles
 

DifferentiationInterface/docs/src/dev_guide.md

@@ -0,0 +1,61 @@
+# Advanced features
+
+## Contexts
+
+### Additional arguments
+
+For all operators provided DifferentiationInterface, there can be only one differentiated (or "active") argument, which we call `x`.
+However, the release v0.6 introduced the possibility of additional "context" arguments, which are not differentiated but still passed to the function after `x`.
+
+Contexts can be useful if you have a function `y = f(x, a, b, c, ...)` or `f!(y, x, a, b, c, ...)` and you want derivatives of `y` with respect to `x` only.
+Another option would be creating a closure, but that is sometimes undesirable.
+
+!!! warning
+    This feature is still experimental, and will likely not be supported by all backends.
+    At the moment, it only works with ForwardDiff.
+
+### Types of contexts
+
+Every context argument must be wrapped in a subtype of [`Context`](@ref) and come after the differentiated input `x`.
+Right now, there is only one kind of context, namely [`Constant`](@ref), but we might add more.
+Semantically, calling
+
+```julia
+gradient(f, backend, x, Constant(c))
+```
+
+computes the partial gradient of `f(x, c)` with respect to `x`, while keeping `c` constant.
+Importantly, one can prepare an operator with an arbitrary value `c'` of the constant (subject to the usual restrictions on preparation).
+
+## Sparsity
+
+When faced with sparse Jacobian or Hessian matrices, one can take advantage of their sparsity pattern to speed up the computation.
+DifferentiationInterface does this automatically if you pass a backend of type [`AutoSparse`](@extref ADTypes.AutoSparse).
+
+!!! tip
+    To know more about sparse AD, read the survey [_What Color Is Your Jacobian? Graph Coloring for Computing Derivatives_](https://epubs.siam.org/doi/10.1137/S0036144504444711) (Gebremedhin et al., 2005).
+
+### `AutoSparse` object
+
+An `AutoSparse` backend must be constructed from three ingredients:
+
+1. An underlying (dense) backend
+2. A sparsity pattern detector like:
+   - [`TracerSparsityDetector`](@extref SparseConnectivityTracer.TracerSparsityDetector) from [SparseConnectivityTracer.jl](https://github.com/adrhill/SparseConnectivityTracer.jl)
+   - [`SymbolicsSparsityDetector`](@extref Symbolics.SymbolicsSparsityDetector) from [Symbolics.jl](https://github.com/JuliaSymbolics/Symbolics.jl)
+   - [`DenseSparsityDetector`](@ref) from DifferentiationInterface.jl (beware that this detector only gives a locally valid pattern)
+3. A coloring algorithm: [`GreedyColoringAlgorithm`](@extref SparseMatrixColorings.GreedyColoringAlgorithm) from [SparseMatrixColorings.jl](https://github.com/gdalle/SparseMatrixColorings.jl) is the only one we support. As a result, sparse AD is now located in a package extension which depends on SparseMatrixColorings.jl.
+
+`AutoSparse` backends only support [`jacobian`](@ref) and [`hessian`](@ref) (as well as their variants), because other operators do not output matrices.
+To obtain sparse Hessians, you need to put the `SecondOrder` backend inside `AutoSparse`, and not the other way around.
+
+!!! note
+    Symbolic backends have built-in sparsity handling, so `AutoSparse(AutoSymbolics())` and `AutoSparse(AutoFastDifferentiation())` do not need additional configuration for pattern detection or coloring.
+
+### Cost of sparse preparation
+
+The preparation step of `jacobian` or `hessian` with an `AutoSparse` backend can be long, because it needs to detect the sparsity pattern and perform a matrix coloring.
+But after preparation, the more zeros are present in the matrix, the greater the speedup will be compared to dense differentiation.
+
+!!! danger
+    The result of preparation for an `AutoSparse` backend cannot be reused if the sparsity pattern changes.