Skip to content

Add updated maths and fitting tutorials to the documentation - #331

Closed
markbujehein wants to merge 56 commits into
add-maths-modulefrom
add-maths-tutorials
Closed

markbujehein wants to merge 56 commits into
add-maths-modulefrom
add-maths-tutorials

Conversation

@markbujehein

Copy link
Copy Markdown
Collaborator

Summary

This PR adds a new, learner-friendly tutorial series to the EasyScience documentation, with a clearer path for new users:

  • Basics: working with numbers that carry units, uncertainties, and bounds
  • Maths functions: arithmetic, error propagation, trig and inverse trig, exp/log, hyperbolic functions, and correlation
  • Fitting: fitting workflows and derived quantities from fitted parameters
  • Advanced topics: Bayesian analysis and posterior-based workflows

It also updates the docs landing pages and introduction to direct users into the new tutorial structure.

Why

The current docs were missing a structured onboarding path for users who want to learn EasyScience, so while adding tutorial for the maths module, I'm trying out something that seems more user friendly. These notebooks provide:

  • clear examples of EasyScience applied on various problems
  • inspired by the DMSC 2026 summer school notebooks, exercises with hints & solutions to reinforce learning
  • worked explanations of uncertainty propagation and parameter behaviour
  • a natural progression from basics to fitting and Bayesian analysis

What changed

  • Added a new Basics tutorial section:
    • arithmetic
    • error propagation (with edge cases)
    • trigonometric functions
    • inverse trigonometric functions
    • exponential/logarithm functions
    • hyperbolic functions
    • correlation (with edge cases)
  • Added a Fitting tutorial section:
    • fitting introduction
    • SANS sphere example (ported from Workshops & schools)
    • QENS Lorentzian example (ported from Workshops & schools)
  • Added Advanced Topics:
    • Bayesian analysis
    • Bayesian QENS
  • Updated the docs homepage and introduction page to point new users toward:
    • Installation & Setup
    • Basics tutorials
    • Fitting tutorials
    • Workshops & schools

markbujehein and others added 30 commits October 6, 2026 18:18
…nces as per latest pixi standard (done by `pixi install`)
…ctions

- Update trigonometric functions to accept only the Parameter type.
- Add new inverse trigonometric functions.
- Add TODO comments for inverse trigonometric functions.
Fix inheritance type errors (PEP 673) when invoking from_scipp()
on subclassed objects like Parameter.
- Remove unused imports and commented-out code.
- Standardize function headers and docstrings.
- Add helper functions for bounds derivation to improve error handling.
- Centralize Scipp operations for trig functions (sin, cos, tan, etc.).
- Add a new sinc function with dedicated error handling and docs.
- Protect tangent bounds to prevent variance explosions near asymptotes.
Replace the generic dispatcher with declarative `_MathOp` specs and add
log10, sqrt, erf, erfc and the hyperbolic functions (sinh, cosh, tanh,
arcsinh, arccosh, arctanh).

- Derive exact result bounds from endpoints and critical points instead
  of a sampled grid, which could miss an extremum and make
  Parameter.from_scipp raise
- Convert degrees to radians for trig bounds
- Accept infinite bounds; clip bounds to the function's domain
- Raise ValueError for out-of-domain values and non-finite results
- Propagate variance for erf, erfc and arctan2 via a Jacobian
  (delta method), as Scipp lacks support
- Remove ~500 lines of commented-out code
- Export all math functions from easyscience.variable and make __all__
  a list of strings
- Add unit tests in tests/unit/variable/test_math.py
Give each of the 20 public math functions two examples, using
rad and deg for angle functions and two dimensionless cases (bounds,
variance, clipping or units) for the rest. Outputs were generated by
executing the examples and cross-checked against independent formulas.
Annotate every test in test_math.py with WHEN / THEN / EXPECT comments.
…iance handling

- Replace `lru_cache` with `cache` from `functools` for simpler caching.
- Add constants for mathematical limits and tolerances.
- Enhance error messages for exponential and hyperbolic overflow conditions.
- Enforce checks for finite results during variance propagation in math operations.
- Expand test coverage for `sinc` edge cases and `tan`/`arctan2` variance propagation.
- Mark test for known collapsed bounds bug in `Parameter` (`@pytest.mark.xfail`).
__all__ held the objects themselves (including the value of
__version__) instead of their names. The language reference requires
a sequence of strings, so `from easyscience import *` failed with
AttributeError on the first entry, the version string. Only star
imports were affected, which is why it went unnoticed.

List the names instead, keeping the same set of public objects.

Also import the math submodule and add it to __all__, so that
`es.math` is available.
Remove the math module from easyscience.math to prevent naming conflicts
with Python's built-in math library.
- tan: a bound exactly on an asymptote now makes the result infinite
  on that side only, so [0, 90] deg gives [0, inf) instead of
  (-inf, inf); a lower bound on an asymptote gives -inf
- variable: re-export the math names with `from .math import *` and
  build __all__ from math.__all__ instead of keeping a second list
- math: document how to use es.math inside a model (derive in
  __call__, results are snapshots, inputs are assumed uncorrelated,
  uncertainties depend on the minimizer) and leave NOTE()/TODO()
  remarks for the lead developer (1-sigma guards, non-fixed
  snapshot results, cost per evaluation, hard-coded Para
  tolerance)
- tests: add integration tests that apply es.math to fitted
  parameters (straight line, angle in degrees, quadrature, Gaussian
  peak with a width derived in __call__, across minimizers), a
  strict xfail documenting that correlated parameters are propagated
  as independent, and unit tests for degree variances, tan bounds on
  asymptotes and the package star import not exporting math
Every es.math function now takes a DescriptorNumber or a Parameter (including user
subclasses). The result is a Parameter, with bounds, if any argument is a Parameter,
and a DescriptorNumber otherwise; it is always the base class. A DescriptorNumber
argument counts as the point [value, value] when the bounds are derived.

Unary functions are typed with a TypeVar; arctan2 has three overloads. Tests cover the
new input and result classes, subclasses, mixed arctan2, the unit keyword and the guards.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
claude and others added 12 commits October 15, 2026 15:03
Two special-edition maths notebooks: how the choice of formula changes
the uncertainty of a result (the table-area example), and how correlation
between measurements changes it. Includes exercises with executed
solutions, tests, nav and index entries.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PHwoT3iPrULYWjTrLPuhan
Remove the Jupyter execution metadata and execution counts from the five
older maths notebooks so test_no_execution_state_is_stored passes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PHwoT3iPrULYWjTrLPuhan
…mples

Add a "Good to know" section to the error-propagation notebook and a
bullet to the correlation notebook. The cases are executed and print the
exact result next to what the operators report: the same quantity
reached by two routes, sin² + cos², the bounds of x - x, and the fit
prediction without the covariance.

Drop overly-strict test_no_execution_state_is_stored. Stored execution counts and
timing metadata are written by normal local runs and do no harm, so the
tutorial tests no longer require them to be stripped. Stored outputs are
still rejected for specific tutorial paths.
The printouts of the limits of error propagation now label the two
numbers "correct" and "EasyScience" instead of "exact" and "operators",
and the surrounding text says "EasyScience" for the library's result.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…notebooks

Add a Fitting group to the tutorials, with a hub page, the fitting
introduction and three new notebooks that fit with model classes and units
and derive quantities from the fit with es.math: SANS with a sphere, QENS
with a Lorentzian, and a Bayesian QENS notebook that compares first-order
uncertainties with the posterior.

Add hub pages for Basics, Workshops & Schools and Advanced Topics, move the
Bayesian analysis and Progress Callback notebooks into Advanced Topics, and
list the categories on the tutorials page.

The notebooks that download their data are read by the structure tests but
are not replayed, so the unit tests need no network.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…s page

The introduction now says who EasyScience is for, shows a first example
with its output, and points to where to start. A new Concepts page in the
user guide defines the numbers, uncertainties, bounds, units, models and
fits that the tutorials rely on. The home page no longer promises
terminology and workflow pages that the user guide does not have.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A group of two or more pages in the navigation should open with an
index.md. The check warns about a group that does not, and never fails.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ing introduction into fitting/

The correlation and error propagation notebooks now say "standard
uncertainty of ±0.02 m" instead of "known to 0.02 m", keep "error" for the
`error` attribute and for the deviation of a reading from the true value,
and say "EasyScience" instead of "the operators".

The fitting introduction moves into the fitting/ folder with the other
fitting notebooks; the navigation, the hubs and the tests follow.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Make the hints of the exercises less revealing, print the correlation of
the posterior draws in the Bayesian notebook and refer to it from the
QENS and SANS notes, ask the Bayesian exercise about a probability that is
not 100 %, state when the convergence statistic is acceptable or strict,
and call the resolution time the coherence time.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…have no uncertainty

Bragg's law uses the Bragg angle, half the scattering angle, so the
trigonometric notebooks now say so. "Known exactly" becomes "taken to have
no uncertainty".

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ebooks

The category headings of the tutorials page are now links to the hubs. The
Fitting and Advanced Topics hubs point to each other's Bayesian notebook.
The concepts page no longer calls a DescriptorNumber "named", and the
introduction opens with the foundation role of EasyScience.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

@rozyczko rozyczko left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Putting tutorials in subdirectories broke the Google Colab button
    The template builds the Colab URL from page.nb_url.split('/')[-2] and [-1], i.e. it assumes tutorials/<name>/<name>.ipynb. With use_directory_urls the new notebooks are published at tutorials/basics/<name>/<name>.ipynb, so the generated link drops the basics/ or fitting/ segment.
    Suggested fix (not tested): build the path from the page URL instead of from the last two path segments, e.g.
{% set filename = page.nb_url.split('/')[-1] %}
{% set colab_url =
  base_colab_url ~ config.extra.vars.github_repository ~
  "/blob/gh-pages/" ~ config.extra.vars.docs_version ~
  "/" ~ page.url ~ filename
%}
  1. notebook-strip does not see the sub-folders
    In pixi.toml, task notebook-strip = 'nbstripout docs/docs/tutorials/*.ipynb'.
    The notebook-prepare step that both docs.yml and tutorial-tests.yml run before
    executing notebooks only strips the top-level folder. notebook-exec, notebook-tests and nbqa ruff all take the directory and are recursive, so nothing currently breaks. But the first time someone commits a nested notebook with outputs, the docs build will render stale outputs instead of new ones. Make the glob recursive (e.g. nbstripout $(git ls-files 'docs/docs/tutorials/**/*.ipynb') or a
    find … -exec), and consider the same for the commented-out notebook-convert/-tweak.

- Colab link: build the notebook path from page.url so basics/ and fitting/ are kept, in both directory-URL and file-URL builds.
- notebook-strip, notebook-convert: use recursive ** globs.
- Add tests: a mkdocs build checking the Colab link targets a published notebook, and a run of the notebook-strip command at several depths.
@markbujehein
markbujehein requested a review from rozyczko October 19, 2026 15:32
@markbujehein

Copy link
Copy Markdown
Collaborator Author

@rozyczko

  1. Putting tutorials in subdirectories broke the Google Colab button

Proposed change in docs/overrides/main.html, based on your suggestion but not identical. With page.url ~ filename, the link breaks under --no-directory-urls (used by docs-build-local), because page.url then ends in .html. I used the folder part of page.url plus the file name instead, which works in both modes.

  1. notebook-strip does not see the sub-folders

Proposed: make both globs recursive, e.g. nbstripout docs/docs/tutorials/**/*.ipynb. When I tried the git ls-files '…/**/*.ipynb' form, it listed only the nested notebooks, not the top-level ones, so I went with the pixi glob. I'm happy to switch back if I've misread that.

@markbujehein
markbujehein removed this pull request from stack #332 October 27, 2026 15:24
@markbujehein
markbujehein added this pull request to stack #351 October 27, 2026 15:25
@markbujehein
markbujehein marked this pull request as draft October 27, 2026 17:08
@markbujehein
markbujehein removed this pull request from stack #351 October 29, 2026 22:08
markbujehein added a commit that referenced this pull request Oct 29, 2026
…odule

Files as of origin/add-maths-tutorials (71ea6a2), unchanged. The shared
files keep both sides: the module branch's notebooks, nav entries and
user-guide pages, plus the tutorial navigation, the Concepts page and the
recursive notebook globs. Updates for the current API follow separately.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

[priority] medium Normal/default priority [scope] documentation Documentation only changes (major.minor.patch.POST)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants