Absstep is not respected in AutoFiniteDiff

[1-Bart-1]

When using the backend = AutoFiniteDiff(absstep=100, relstep=0.01) one would expect the state to be perturbed with a value of max(100.0, 0.01*x), but this doesn't actually happen. It looks like relstep is set to be equal to absstep somewhere.

julia> backend = AutoFiniteDiff(absstep=100, relstep=0.01)
AutoFiniteDiff(relstep=0.01, absstep=100, )

julia> x = collect(1.0:5.0)
5-element Vector{Float64}:
 1.0
 2.0
 3.0
 4.0
 5.0

julia> function f(x)
           println(x)
           sum(abs2, x)
       end
f (generic function with 1 method)

julia> gradient(f, backend, x)
[1.0, 2.0, 3.0, 4.0, 5.0]
[1.0, 2.0, 3.0, 4.0, 5.0]
[1.01, 2.0, 3.0, 4.0, 5.0]
[1.0, 2.0, 3.0, 4.0, 5.0]
[1.0, 2.02, 3.0, 4.0, 5.0]
[1.0, 2.0, 3.0, 4.0, 5.0]
[1.0, 2.0, 3.03, 4.0, 5.0]
[1.0, 2.0, 3.0, 4.0, 5.0]
[1.0, 2.0, 3.0, 4.04, 5.0]
[1.0, 2.0, 3.0, 4.0, 5.0]
[1.0, 2.0, 3.0, 4.0, 5.05]
5-element Vector{Float64}:
  2.009999999999934
  4.019999999999868
  6.030000000000039
  8.040000000000092
 10.049999999999955

[1-Bart-1]

I think it is supposed to work like this, but I don't fully understand why...

[gdalle]

Just to summarize the relevant steps of the pipeline:

1. Inside DI at preparation time, the steps are deduced from backend and stored in prep as follows:

DifferentiationInterface.jl/DifferentiationInterface/ext/DifferentiationInterfaceFiniteDiffExt/onearg.jl

Lines 248 to 276 in d8905f5

	struct FiniteDiffGradientPrep{SIG,C,R,A,D} <: DI.GradientPrep{SIG}
	_sig::Val{SIG}
	cache::C
	relstep::R
	absstep::A
	dir::D
	end
	function DI.prepare_gradient_nokwarg(
	strict::Val, f, backend::AutoFiniteDiff, x, contexts::Vararg{DI.Context,C}
	) where {C}
	_sig = DI.signature(f, backend, x, contexts...; strict)
	fc = DI.fix_tail(f, map(DI.unwrap, contexts)...)
	y = fc(x)
	df = zero(y) .* x
	cache = GradientCache(df, x, fdtype(backend))
	relstep = if isnothing(backend.relstep)
	default_relstep(fdtype(backend), eltype(x))
	else
	backend.relstep
	end
	absstep = if isnothing(backend.absstep)
	relstep
	else
	backend.relstep
	end
	dir = backend.dir
	return FiniteDiffGradientPrep(_sig, cache, relstep, absstep, dir)
	end

2. Inside DI at execution time, the steps are retrieved as-is from prep:

DifferentiationInterface.jl/DifferentiationInterface/ext/DifferentiationInterfaceFiniteDiffExt/onearg.jl

Lines 278 to 289 in d8905f5

	function DI.gradient(
	f,
	prep::FiniteDiffGradientPrep,
	backend::AutoFiniteDiff,
	x::AbstractArray,
	contexts::Vararg{DI.Context,C},
	) where {C}
	DI.check_prep(f, prep, backend, x, contexts...)
	(; relstep, absstep, dir) = prep
	fc = DI.fix_tail(f, map(DI.unwrap, contexts)...)
	return finite_difference_gradient(fc, x, prep.cache; relstep, absstep, dir)
	end

3. Inside FiniteDiff, both steps are combined by compute_epsilon:

https://github.com/JuliaDiff/FiniteDiff.jl/blob/a7eca2d4b73c4de12140d89df7621fcc90d29190/src/gradients.jl#L233-L255

https://github.com/JuliaDiff/FiniteDiff.jl/blob/a7eca2d4b73c4de12140d89df7621fcc90d29190/src/epsilons.jl#L9-L11

I don't know where things go wrong, do you think it is on the DI side?

[gdalle]

Also note that there are spots in FiniteDiff where kwargs are not propagated correctly, so that's definitely a possibility. See e.g. JuliaDiff/FiniteDiff.jl#199

[gdalle]

Okay, I think the following lines are a typo, backend.absstep should be used:

DifferentiationInterface.jl/DifferentiationInterface/ext/DifferentiationInterfaceFiniteDiffExt/onearg.jl

Lines 269 to 273 in d8905f5

	absstep = if isnothing(backend.absstep)
	relstep
	else
	backend.relstep
	end

You were actually the one who introduced them in #693, can you confirm?

[1-Bart-1]

Yes, thanks for fixing this.