LAMMPS Test Status Website

Aggregates the results of the automated LAMMPS test runs into a static website and a rolling GitHub status issue.

How it works

• GitHub Actions workflows in lammps/lammps upload JUnit XML test results as artifacts for post-merge runs on the develop branch (regression tests: merged run.json + JUnit XML; unit tests: one junit-<config> artifact per platform/configuration; the example input check: the JUnit XML of the test harness as it is, see below).
• The full regression tests are no longer run in GitHub Actions but on a dedicated machine with a much more complete LAMMPS configuration, and published on download.lammps.org (tools/fetch_regression.py, see below). The same input decks are run in four configurations: serial (one MPI task), parallel (4 MPI tasks), openmp (2 MPI tasks with 2 OpenMP threads each, through the OPENMP package), and kokkos (the same through KOKKOS/OpenMP).
• The same machine runs the unit tests in its native GCC build of x86_64 Linux and publishes them as a JUnit XML file next to the coverage report (tools/fetch_unittest.py, see below). That build has a far more complete package selection than the GitHub Actions runners compile, so it covers several hundred tests more than any of the configurations ingested from there.
• A second dedicated machine with NVIDIA V100 GPUs runs the unit tests of the GPU builds - the GPU and KOKKOS packages through CUDA, and the GPU package through OpenCL with KOKKOS/OpenMP, each in single, mixed, and double precision - and publishes one JUnit XML file and one JSON summary per build (tools/fetch_unittest.py as well, see below).
• A third dedicated machine with Intel Xeon CPUs runs the unit tests of a build with the Intel oneAPI compilers, Intel MPI, and MKL, including the INTEL package and KOKKOS/OpenMP, and publishes them the same way (tools/fetch_unittest.py, see below).
• The update workflow in this repository ingests new artifacts (tools/ingest_actions.py) and the latest published regression and unit test results (tools/fetch_regression.py, tools/fetch_unittest.py), records when the branches those were run on last moved (tools/fetch_pushes.py), archives one run.json per run under data/<suite>/<runid>/, rebuilds the website (generator/build_site.py), deploys it to GitHub Pages, and updates the rolling status issue (tools/update_issue.py). It runs once a day, at 10:37 UTC: after the nightly runs of the test machines, and before people in the US get to work. Only the latest result of those machines is published, so one that is replaced before the next poll is not archived; that is intended, since test failures are fixed on the latest state.
• The three test machines also run performance benchmarks after their tests and publish them on download.lammps.org (tools/fetch_performance.py, see below); the dashboard shows how they compare with a reference per benchmark, by category.
• The nightly builds of the LAMMPS and LAMMPS-GUI packages publish how each of their tasks went (tools/fetch_nightly.py, see below), which is archived here since only the newest build is published.
• The lines of code of LAMMPS are counted once a week and published as well (tools/fetch_codesize.py, see below).
• Summaries of the server-side reports (code coverage, static analysis) can be ingested as data/external/*.json; the state of the Coverity Scan is collected from two sources, the analysis metrics of the project overview page and the summary of the build that was submitted for scanning (tools/fetch_coverity.py, see below).
• The status of the automated manual builds is collected from the status.json files published with the three manual variants (tools/fetch_docs.py, see below), including the words the spellchecker flagged in the development version.
• The monthly GitHub activity of the members of the core team is swept from the GitHub API (tools/fetch_team.py, see below) and reported on a page of its own, linked from the repository activity card.

Notifications

The status issue body is rewritten in place on every update, which does not notify anybody. A comment is posted only when new failures appear or known failures are fixed; comments notify issue subscribers. Subscribe to the pinned status issue to get emails about regressions - and nothing else.

The same applies to the manual builds: a comment goes out when one of the three variants starts failing or falls behind its branch, and again when it recovers. Each of those is announced once, not on every update, which is tracked per manual with a hidden marker in the comment rather than with the run id used for the test suites.

The nightly package builds work the same way: a comment goes out when the tasks that failed change, when a build is overdue, and when the builds recover.

Local use

Everything only needs the Python standard library (plus the gh CLI for the scripts that talk to GitHub):

python3 generator/build_site.py             # data/ -> _site/
python3 tools/ingest_actions.py --dry-run   # what would be ingested
python3 tools/fetch_regression.py --dry-run # latest regression results
python3 tools/fetch_performance.py --dry-run  # latest benchmark results
python3 tools/perf_reference.py --machine <m> --match <regex> --note <why>
python3 tools/fetch_nightly.py --dry-run    # newest nightly package build
python3 tools/fetch_codesize.py --dry-run   # newest count of the lines of code
python3 tools/fetch_pushes.py               # pushes to the tested branches
python3 tools/fetch_docs.py                 # manual build status
python3 tools/fetch_team.py                 # core developer activity
python3 tools/update_issue.py --repo <owner/repo> --site-url <url> --dry-run

run.json files can also be produced manually from local test runs with tools/regression-tests/merge_results.py (regression tests) in lammps/lammps or tools/junit_to_json.py (any JUnit XML file, e.g. from ctest --output-junit).

Data layout

data/<suite>/<runid>/run.json

<suite> is quick-regression, kokkos-regression, check-examples, full-regression/<config>, or unit-tests/<config>: a suite that is run in more than one configuration keeps them in subdirectories and appears once per configuration. <runid> is <ISO timestamp>_<short sha> and sorts chronologically. The run.json format is documented in tools/rundata.py; a test is recorded there as passed, failed, error, runtest, or skipped, and the website classifies some of them further as it reads them: the errors that hit the time limit of the harness as timeouts, and the skipped tests of the runs archived before the harness had the word runtest as what they are (see below).

Full regression tests

tools/fetch_regression.py archives the results published as https://download.lammps.org/coverage/serial.json and the corresponding parallel.json, openmp.json, and kokkos.json (the -summary.md and -regression.xml files next to them show the same data and are not ingested, since the JSON is a superset of both). The file name is what identifies the configuration: the config_file property does not, three of the four share config.yaml, and only the title of the run spells the difference out - which is why the status issue carries it alongside the suite name where it says more than the name does (rundata.config_label()).

That title is a shorthand, though (MPI+OpenMP, KOKKOS/OpenMP), and it says nothing about the decomposition. The website therefore does not repeat it on the dashboard cards but spells out what each configuration runs on the run page and on the comparison page (rundata.CONFIG_DETAILS), which is also where the order the configurations are listed in comes from (rundata.CONFIG_ORDER: serial, parallel, openmp, kokkos - by what each adds to the one before, not alphabetically).

Only the most recent run is published, so a run that is not picked up before the next one replaces it is lost; the runs are gated by changes in the monitored branch, though, so unchanged results simply stay in place. Since the published files are rewritten even when no new test run happened, ingestion deduplicates on the generation time and commit recorded in the file rather than on its modification time.

Beyond that, the archive keeps one run per commit, since every archived run is a bar of the trend on the dashboard. The test machine only runs when the monitored branch has changed, so a commit that is published twice was run again while the test scripts themselves were being worked on: those results replace the run archived for that commit (rundata.archived_with_commit(), the new run is written before the old one is removed), and where they repeat its every verdict as well they are not archived at all - that is a re-publication rather than a run.

The commit and the branch are read from the commit and branch metadata fields, and recovered from the git_info property where those are missing; that property is also the source of the git describe string kept as version. The website and the status issue read the commit as sha.

The run id is stamped with the generated time where that carries a time zone, and with the publication time from the Last-Modified header where it does not: a generated field without a zone is in the local time of the test machine and cannot be compared with the UTC stamps of the runs ingested from GitHub Actions.

Tests that run out of time

A test that hits the time limit of the test harness is reported as an error like any other, with a message ending in timeout (<n>s expired). Whether it expires depends on the limit in force (180 s serial, 60 s for the others), on how many tests run beside it, and on the machine - so it says nothing about the code. Those runs are classified as timeout (rundata.status_of()), counted apart from the errors, and left out of the broken count that drives the last all OK run and the notification comments. They are not swept under the carpet: they have their own tile, their own band in the trend bars of a card, their own filter and column, a run-to-run comparison lists them as newly out of time, and a test that starts hanging because of a code change shows up there. The limit itself is read back from the messages (rundata.time_limits()), since the run data does not record it.

Runs that complete without a check

Since lammps/lammps#5144 the harness reports a test that ran to completion but could not be checked against anything as runtest: an input without a reference log file (run either shortened to a few steps as a crash test, or in full), a reference log the harness cannot parse, a run without thermo output, or a --preflight-only run in which every input is only parsed and taken one step with -skiprun. Only the run itself was tested, so it is neither passed nor skipped, and the website keeps it apart from both: its own tile and band on the cards, its own column in the status issue, its own filter on the run pages. It is the bulk of what the regression suites report that is not a verdict - 184 of the 811 inputs of the first serial run that carried the word.

Before that, the harness reported the same outcomes as skipped, with a message that starts with completed. rundata.status_of() reads those as runtest as well, so that the archive is one vocabulary from end to end: the band on the trend bars runs through unbroken where the harness changed its wording, and a re-publication of a commit that was archived before the change is not mistaken for a run with hundreds of moved verdicts. In JUnit XML, which has no element for the outcome, the harness writes a runtest as <skipped> with a status="runtest" attribute on the test case, which tools/junit_to_json.py reads back apart.

A test that was broken and now completes as a runtest counts as fixed, in the run-to-run comparison and as the last OK run of a test that breaks again later: the crash is gone, whatever became of the reference log file. This is what merge_results.py upstream reports as fixed too.

What a dashboard card shows

The numbers of the latest run as tiles, the last TREND_RUNS (25) archived runs as one stacked bar each, what changed since the run before, and which branch, commit and time the numbers are of. A bar is as tall as the number of tests of that run and is stacked from the baseline up in the order failed, errors, timed out, skipped, runtest, passed: the outcomes worth watching sit on the baseline, where a change in one of them changes the height of that band rather than shifting everything above it, and the tests that passed float on top, with the runs that completed unchecked right below them, so the top edge of a bar stays the number of tests and the two bands together are what did not break. The bars keep their pitch while the archive fills, with the newest run at the right edge, and they carry the color each outcome has everywhere on the site - which is what makes the tiles above them the legend of the chart.

Reading a regression result

The example inputs were not written to be tests, so a plain pass/fail count is misleading and the run pages group the results the way tools/regression-tests/REPORTING.md in lammps/lammps describes:

• Needs a fix in the examples tree - every test whose attention field names a problem with the input script itself, grouped by kind (rundata.attention_groups()). This is a work list against the repository, not against the code, and it is set independently of the verdict, so a test that passes can carry one. It is also the majority of what the regression suites report: reference log files that match no input, inputs that run a production number of steps, and inputs whose initial velocities depend on the number of MPI processes.
• Worth investigating - the remaining failures, sorted by how early the run deviates from its reference log (rundata.divergence()). A classical MD trajectory is chaotic, so a difference that first appears after a thousand steps says nothing about the code, while one that is there in the very first thermo output cannot be rounding. The late ones are folded away.
• Not really tested - the statuses that are not verdicts, counted per kind, since each implies different work, and grouped by what the harness made of them: the runtests, which ran and only lack a check (no reference log file, a log file the harness cannot parse, no thermo output), the skipped inputs, which were never run (needs a multi-partition run, excluded by the test configuration, a style the binary does not have), and the errors that are not about the code either (package not installed). A wording the site does not know is counted as other rather than dropped (rundata.NOT_TESTED_KINDS), so that a new kind cannot go unnoticed.

compare.html puts the configurations of one commit side by side. It is reached from the run pages of that commit rather than from the dashboard, since it says nothing about a run of any other commit. A test is only counted there where every configuration reaches a verdict on it: inputs that need a fix and inputs that ran out of time are left out, because most of the former cannot match a reference log file that was written with a different number of MPI processes, and they bury everything else.

Because a timeout is the absence of a verdict rather than one, a comparison against a run in which a test timed out falls back to the most recent run before it that did judge that test (rundata.compare_runs() reads older runs lazily, only as far as it needs them). Otherwise a test that keeps failing but flaps through a timeout would be announced as a new failure every time it came back - which is exactly what the archived parallel runs did on 2026-07-27.

Incomplete ingestion

An ingest pass never fails the job it runs in. What it could not take in is written to data/external/ingest.json and to the job log instead. A failed job would publish nothing at all and leave the previous page standing; the point of the report is that a short pass still publishes what it did get and records what is missing.

Three things are reported. Runs that did not come in for a reason another pass may not hit - an artifact that did not download, a zip without the file it should hold, an XML that does not parse - are listed under pending and retried on the following passes by run id, which does not depend on the run listing at all: a run that has meanwhile scrolled out of the window is still reachable that way. After ingest_actions.MAX_ATTEMPTS tries the run is reported as a problem rather than queued forever. Runs that carry nothing this repository can read are listed under problems, but only where they would have advanced a suite: a run older than what the suite already holds is not a gap in its history, which keeps the tail of runs left behind by a corrected workflow out of the report.

The third is the run listing itself. The window is the completed runs on develop created in the last --days days (7 by default), capped at --max-runs; lammps/lammps produces some 25 to 55 of those a day. The date bound is what makes the listing reliable: filtered by branch and status, the runs API answers from a search that is documented to return at most 1,000 results, and without a date the search matches every run develop ever had (some 2500). Which part of them a request then gets is arbitrary - on 2026-09-29, requests a minute apart got listings that began in April, in July, or a week back, and passes kept archiving nothing while the dashboard fell days behind. Bounded by date, the same search comes back complete and current on every request. A listing that comes back out of order - as one did on 2026-08-28, during the GitHub Actions outage of that week - need not reach the newest runs at all, and the pass then finds nothing new and looks exactly like an idle poll, while the results it was meant to pick up quietly scroll out of reach. Each pass therefore checks that the listing is newest-first and that the latest start of a run in it is not earlier than that of a run already archived from GitHub Actions (the runs of the test machines, which share the unit-tests directory, are left out of that comparison).

The stale listings seen before the date bound was added - on 2026-09-24, one request got a listing that began a week back and another one that held nothing newer than July - were the same effect. A listing that fails either check is still fetched again, up to LISTING_ATTEMPTS times and LISTING_PAUSE seconds apart. Where none holds up, the pass does not stop: it ingests what the last listing holds and sets recheck, which makes the next pass examine a window RECHECK_FACTOR times wider (in runs and in days, the latter up to RECHECK_MAX_DAYS, which keeps the search well below its 1,000 results), so that whatever the bad listing skipped is picked up on the next round.

None of this is shown on the dashboard, which is public: the badges of the workflows show directly whether a GitHub Actions run failed, and a gap the ingestion leaves is closed by its next pass. api/summary.json carries the whole report, though, so that whoever gates a nightly run on the snapshot can tell a suite that reported nothing from one whose result never arrived.

Overdue results

What the dashboard does warn about, in a banner it leads with, is data that is out of date in a way that needs somebody's attention: a test machine that has stopped publishing, and a collection that has stopped altogether.

The test machines (the full regression runs, the linux-x86_64-gcc unit test run, the GPU builds, and the Intel oneAPI build: the runs that record the source_url they were fetched from) run only when the branch they test has changed. The age of their newest result says nothing by itself: develop sat still from 2026-09-15 to 2026-09-21, and the results of the 15th were as current as they could be all week. A result is therefore judged against the branch it tested: it is overdue when that branch moved on from the commit it tested and nothing newer has come in for OVERDUE_HOURS (24). Where the branch moved while the run was going, that day is counted from the end of the run: only one instance runs at a time, and a run of the GPU machine takes over six hours (its CPUs are slow, and the KOKKOS builds compile through nvcc), so a push made meanwhile has to wait for the next run. The day covers that run and the time it takes to be launched. The time between two polls of this site does not add to it, since the results are fetched right before the site is built; it only delays the warning.

When the branch moved is taken from the repository activity API of GitHub, which records every push, merge, and force push to a branch with the commit it moved from and the one it moved to. tools/fetch_pushes.py collects the latest 100 of those for each branch the test machines last ran on into data/external/pushes.json, and generator/build_site.py makes the judgment (overdue_state) when the site is built. A branch that is not on GitHub - the machines are pointed at branches of their own while their setup is being worked on - has no pushes, and a result of it is not judged. Replayed over the polls of the month before the check was added, it would have fired for one incident only: from 2026-09-06 on, for the kokkos configuration of the full regression runs, which did not publish the commit the other three did on 2026-09-05 and caught up only on the 8th.

An overdue suite carries a warning sign, and its commit and time turn red. api/summary.json lists the overdue suites with the reason under overdue. An ingest report that has not been rewritten in INGEST_STALE_HOURS (50) also gets the banner: the collectors are polled once a day, and a collection that stopped leaves the same data behind as one that found nothing. The scheduled job rewrites the report right before it builds the site, so this is only ever seen on a site rebuilt after a push to this repository.

Example input check

check-examples.yml in lammps/lammps (Check example inputs /w -skiprun, added with the same pull request as the runtest status) runs every input script of the examples tree with -skiprun appended, on 2 MPI tasks of a GitHub Actions build: the whole script is parsed, every style instantiated, the system set up, and one step taken - the loops of all run and minimize commands end after one step. That finds inputs that no longer parse or crash in setup at a small fraction of the cost of the regression runs, and on every push rather than once a day; it checks nothing numerically. The workflow's own verdict is the failure.yaml it stops on; what is archived here is the JUnit XML it uploads in check-examples-results, converted by tools/junit_to_json.py into data/check-examples/<runid>/run.json (ingest_actions.JUNIT_WORKFLOWS; the commit, the branch and the time are those of the workflow run, since the git_info the harness records is the detached HEAD of the checkout).

By design, every input that completes is a runtest (only a -skiprun check) and none passes, so the card of the suite shows zero passed and zero failed and the suite is all OK exactly when no input errors out. The inputs it cannot judge are skipped and counted per kind on the run page: the ones excluded in config_skiprun.yaml, the ones whose post-run analysis cannot work with a run cut short (cannot be checked with -skiprun, an error message that names a variable or a tally that needs the full run), the multi-partition inputs, and the ones that use a package or feature the Actions build lacks. The dashboard sorts the suite after the full regression cards (rundata.SUITE_ORDER), names it Example Input Check (rundata.SUITE_TITLES), and spells out on its run pages what it runs (rundata.SUITE_DETAILS), since the name of the suite says neither.

The same pull request adds unittest-fftw.yaml (Unittest for Linux, FFTW3 and KOKKOS OpenMP): the unit tests in the build configuration the example check uses - MPI enabled, FFTW3 instead of KISS FFT, the KOKKOS package with its OpenMP backend, and the default (smallbig) integer sizes, which no other Actions configuration has. Its results are ingested like those of the other unit test workflows, as unit-tests/linux-x86_64-fftw.

Published unit test run

tools/fetch_unittest.py archives https://download.lammps.org/coverage/junit.xml under unit-tests/linux-x86_64-gcc. This is the unit test suite run in the same pass as the coverage report, in the machine's native GCC build of x86_64 Linux: a package selection far more complete than the GitHub Actions runners compile, and the only configuration in the matrix that is not ingested from there. It is published in the ctest --output-junit format, which the converter for the GitHub Actions artifacts already reads (tools/junit_to_json.py).

Publication works as for the regression results - a single file rewritten in place - so the same rules apply, and they are implemented by the same helpers in tools/rundata.py: deduplication on the generation time and commit recorded in the file, and one run per commit.

What the JUnit format does not carry is the commit and the branch; it stamps the run in the local time of the test machine, which cannot be compared with the UTC stamps of the other suites, and it says nothing about the build beyond the host name. All of that comes from https://download.lammps.org/coverage/summary.json instead - the second set of data published by this same run, already fetched for the coverage numbers on the dashboard (tools/fetch_external.py). It records the commit in full, the branch, the date of the run as UTC (which is what the run id is stamped with), and the compiler and operating system, kept as properties of the run under the names the full regression runs report the same two in (fetch_unittest.SUMMARY_PROPERTIES).

The two files are the two halves of one run only once it has finished publishing: a run wipes the webroot and fills it again, so both are absent for the length of that (which is why all four regression results can read as 404 for a while), and a summary fetched in between can still be the one of the run before. The abbreviated commit in the test output is of the binary those very tests ran, so a summary that disagrees with it is not the other half of this run: that read is dropped with a warning and retried on the next poll, since what is published stays in place until the next run replaces it.

The git describe string kept as version is read from a version field of the summary where it carries one, and otherwise from the output of the tests themselves: lmp -h prints a Git info (<branch> / <describe>) banner, and the JUnit file quotes what each test printed (fetch_unittest.git_info_of()). That banner also names the branch and the abbreviated commit, which is what carries a run whose summary could not be fetched - with the Last-Modified header of the JUnit file for a stamp, so that a fetch that fails on the summary alone still archives the run instead of dropping it.

That fallback is thin, and deliberately not relied on for anything else: ctest cuts the output it quotes off at 1024 bytes per test (719 of the 991 tests of the run archived on 2026-07-28 are truncated), so the banner survives only because it is printed near the top of the help text. The compiler and the operating system, printed at the end of the same help text, do not survive at all - which is why the summary is the only source for those two.

The name of the ctest suite (e.g. Linux-g++-15) is kept as a property beside them: it names the compiler in short, and it is the only such record for a run archived before the summary reported one.

GPU unit test runs

The same script archives the unit test runs of the GPU machine, one per build, from https://download.lammps.org/gpu-results/cuda-single-junit.xml and the corresponding cuda-mixed, cuda-double, opencl-single, opencl-mixed, and opencl-double files, under unit-tests/<build> with the name of the file as the name of the configuration (fetch_unittest.SOURCES). The cuda builds run the GPU and KOKKOS packages through CUDA, the opencl builds the GPU package through OpenCL and KOKKOS on its OpenMP backend; the precision is that of the GPU package (and of KOKKOS, for CUDA). The dashboard spells that out for each of them (rundata.CONFIG_DETAILS).

Next to each JUnit file that machine publishes a summary of the build with the same fields as the summary of the coverage report (https://download.lammps.org/gpu-results/cuda-single-summary.json and so on), so these runs are archived just like the linux-x86_64-gcc run: the commit in full, the branch, the version, and the UTC date from the summary, the compiler and operating system as properties of the run, and the same check of the summary against the Git info banner of the test output, with the same fallback to that banner where a summary cannot be fetched. They are run when develop has changed, like the linux-x86_64-gcc run, and deduplicated by the same rules. All six builds are uploaded together once the last one is tested, but each summary is dated when the tests of its own build finished, so the six runs of one commit carry somewhat different stamps.

Since the opencl builds test KOKKOS on its OpenMP backend, the kokkos-openmp-<precision> configurations of unittest-kokkos.yml in GitHub Actions were retired (lammps/lammps#5202) and their archived runs removed; only the kokkos-serial-<precision> ones are still run there. The workflow runs from before that change stay in the listing window for days, so ingest_actions.RETIRED_CONFIGS names the configurations that are no longer taken in - without it, the next pass would bring them back.

Intel oneAPI unit test run

A third machine, with Intel Xeon CPUs, compiles LAMMPS with the Intel oneAPI compilers (icx, icpx, and ifx), Intel MPI, and MKL, with the package selection of the coverage build plus the INTEL package and KOKKOS on its OpenMP backend, and runs the unit tests with it. The same script archives that run from https://download.lammps.org/intel-results/intel-junit.xml and the summary next to it (https://download.lammps.org/intel-results/intel-summary.json), which has the same fields as the other summaries, under unit-tests/linux-x86_64-oneapi: named like linux-x86_64-gcc by its platform and compiler, since "intel" alone would read as the INTEL package. It is checked, deduplicated, and archived by the same rules as the GPU builds.

Like the other test machines, it tests develop, so its results are judged as overdue by the same rule (see above).

Performance benchmarks

After their tests, the three test machines run performance benchmarks with the binaries they have just built, whenever the branch they test has changed, and publish them under https://download.lammps.org/performance/, each under a prefix of its own: gpu (the machine with the V100 GPUs: the GPU package through CUDA and OpenCL, KOKKOS through CUDA), cpu (the machine of the full regression tests: the CPU styles, built with GCC and MPICH), and intel (the machine with the Intel Xeon CPUs: the INTEL package with the oneAPI compilers, and GCC builds of the plain, OPENMP, and KOKKOS/OpenMP styles on one core, on both sockets with MPI, with MPI and OpenMP threads, and with two threads per core). The prefixes name a role rather than a host, since the machine behind one can change. tools/fetch_performance.py reads the <prefix>-summary.json of each and archives it as data/performance/<machine>/<runid>/run.json; the table of samples and the logs published beside it are not ingested, the summary carries every sample as well.

A benchmark is measured in atom-steps per second. The GPU machine runs every benchmark on each of its four GPUs at the same time, and the best of those copies is what counts: what sets one copy apart from the others is what disturbed it, and over the clean nights of the calibration runs the best copy moved by 0.7% at most from night to night, where the median moved by up to 2.2%. The archive keeps one run per commit and machine, as for the unit tests: a commit is only benchmarked twice when a run was repeated by hand because the first one was disturbed or broke off, and the repeat replaces it.

Every benchmark is compared with a reference, recorded per machine and benchmark in data/performance/references.json (tools/perfdata.py documents the layout). A benchmark gets its first reference from the first run it succeeds in. A change that makes it faster or slower for good - of the code, the compilers, or the machine - is acknowledged with a new reference, set by hand with tools/perf_reference.py from one run, or from the median of the runs since one. A reference holds from the run it was taken from on, so the runs before it are still measured against the one they had, and the step where the change came in stays in the history.

The benchmarks are reported in six categories, which the summaries name per benchmark: the GPU package, KOKKOS/CUDA, KOKKOS/OpenMP, CPU/MPI (the plain, OPENMP, and OPT styles, from one core to a full node), CPU only (decks of styles only the CPU machines run, of any package), and the INTEL package. A category is the geometric mean of the ratios of its benchmarks to their references - the mean of ratios, so that twice as fast and half as fast cancel out - over whichever machines run benchmarks of it: CPU/MPI and KOKKOS/OpenMP merge the cpu machine with the GCC builds of the intel one. A benchmark that failed is left out of the mean and counted apart.

The dashboard card shows the last TREND_RUNS (25) nights in which any machine ran, by UTC day: the machines run one after the other between about 04:00 and 10:00 UTC, so the runs of one night share their day. A night in which no machine ran would only repeat the one before, and is left out. A machine without a run in a night keeps the results of its last one, since the code it tests did not change; those are drawn faded, so that they do not read as a confirmation. The bars keep their width while the archive fills, with the newest night at the right edge, as on the cards of the test suites. The card has two charts on the same columns. Above, one bar per night stacked from one segment per category, each as tall as the percentage of its references it reached, so that the height of a bar is the performance of all benchmarks together and its course over the weeks the overall trend. A bar standing on zero cannot show a change of a few percent in one category, though; that is what the strips below it are for, one per category, with a bar up from the line of its references for a night it ran faster and down for a night it ran slower, 1.0% to the edge (PERF_SCALE): a category is the mean of about 20 to 70 benchmarks, which moves by 0.05-0.2% from one night to the next, and by only about 0.3% when one benchmark in 30 changes by 10%. The strips carry the colors and names of the segments, and are the legend of the bars above them. performance.html draws the same charts at a readable size, lists the latest run of each machine with its build and hardware, and every benchmark with its latest result, its reference, the change since the run before, and its own history, in a strip that goes to 10% (PERF_ENTRY_SCALE), the failures and the slowest first. api/summary.json carries the newest night under performance.

No threshold is applied yet, and nothing is reported in the status issue: which change is worth a notification is to be decided once the benchmarks have run unchanged for a while.

Nightly package builds

The scripts of the lammps-release repository build the packages of LAMMPS and LAMMPS-GUI every night, whenever develop has changed since the last build, and upload them to a dated folder under https://download.lammps.org/nightly/. The build is made of tasks: boot and shutdown start and stop the containers and the virtual machine it runs in, and macos, linux, static, flatpak, and windows each build one kind of package. How long each took and whether it succeeded is published as https://download.lammps.org/coverage/nightly.json, together with the branch, commit, and version of LAMMPS that was built.

Only the status of the newest build is published, so tools/fetch_nightly.py archives it as data/nightly/<folder>.json (the layout is documented in tools/nightlydata.py). A folder is replaced when a build is repeated on the same day, and so is its file. A night without a build adds nothing, since there is nothing new to report, and a status file that is unreachable or not usable is skipped with a warning.

The dashboard card shows how the newest build went, and one strip per task over the last 25 builds: a bar as tall as the time the task took against the longest of them, since the tasks take anything from seconds to more than an hour. A task that failed usually stopped early, so its time says nothing and a short bar would hide the failure: its bar fills the strip, in red, with the time as its tooltip. A task added on the build machine shows up by itself, after the ones tools/nightlydata.py knows the order and the name of.

The builds are judged for being overdue like the results of the test machines (see above), by the branch and commit in the status file. api/summary.json carries the newest build under nightly, and lists it as nightly-builds under overdue. The status issue shows the newest build, and a comment goes out when the tasks that failed change, when a build is overdue, and when the builds recover (see Notifications). Being overdue is also the only way a build that fails before it gets to publish its status can show up.

Code size

codesize.py in the lammps-analyze repository counts the lines of code of the develop branch once a week, per folder below src/ and lib/, and publishes them as https://download.lammps.org/analysis/codesize.json. It reports on the last full week, from Sunday to Saturday as the weekly commits of the activity card. A line counts unless it is blank or holds nothing but a comment, as the "code" column of cloc.

Only the newest week is published, so tools/fetch_codesize.py archives it as data/codesize/<week>.json (the layout is documented in tools/codesizedata.py). The weeks before the first run of the job were filled in once from a local clone, as described in codesize.py.

The dashboard card puts the folders into groups, with the presets of the newest week for all of the history, so that a change of the groups in tools/codesizedata.py needs no new count. It shows one stacked bar per week for the last 26 weeks, and below it one strip per group with the lines gained or lost in each week. The size of the code changes too slowly for the bars to show more than the larger steps, and the groups change by very different amounts, so that each strip is scaled to its own largest change. The footer turns red when no new week has come in for 14 days. Nothing is reported in the status issue.

Documentation build status

The three published variants of the manual - develop (https://docs.lammps.org/latest/), release (https://docs.lammps.org/), and stable (https://docs.lammps.org/stable/) - each carry a status.json in their document root with the documented commit, the build time, and the outcome and duration of the build steps (html, pdf, publish, and, on develop, spelling; see below). tools/fetch_docs.py collects those into data/external/docs.json, together with the current head commit of each branch as queried from GitHub.

The manual is rebuilt hourly but only once per commit hash, so an unchanged status.json normally means there was nothing to do and its age says nothing about the health of the build machine. Freshness is therefore judged by comparing the documented commit against the branch head: a newer head commit shows as pending and, once it has gone unbuilt for more than STALE_HOURS (6 h), as stale. A status.json that cannot be fetched leaves the last known values in place; if it stays unreachable for longer than the same interval, the entry is shown as unknown rather than as a stale success.

Those rules and the docs.json layout live in tools/docsdata.py (the counterpart of tools/rundata.py), so the website and the status issue judge a build the same way and report the same state.

Spellchecker

The build of the development version also runs the spellchecker (make spelling in the doc directory) and lists every word it did not recognize in status.json, one line of <file>:<line>: (<word>) <context> per hit, with their number in the spelling build step. The other two manuals do not run it: a typo can only be fixed on develop.

The list is carried into docs.json as it stands and rendered on a page of its own, spelling.html, which links each hit to the documentation source on GitHub at the commit that was built. The dashboard card and the status issue only say how many words were flagged and link to that page.

A flagged word is not necessarily a misspelling - technical terms, author names, and syntax the checker cannot know belong in doc/utils/sphinx-config/false_positives.txt in the LAMMPS repository - so the number is reported but does not enter the verdict on the build. What the spelling step reports is whether the checker ran, the same as for the html, pdf, and publish steps.

Coverity Scan

tools/fetch_coverity.py collects data/external/coverity.json from the two ends of the scan, which know different things about it:

• metrics - the publicly visible Analysis Metrics of https://scan.coverity.com/projects/lammps-lammps, scraped from the project overview page: outstanding, newly detected and fixed defects, defect density, lines analyzed, and the day of the last analysis. This is the outcome of an analysis, and the page does not say which state of the source tree it was run on.
• build - https://download.lammps.org/analysis/coverity.json, written by the script that builds and submits LAMMPS for scanning (coverity.sh in the lammps-analyze repository, run twice a week and only when the monitored branch has changed). It records the branch, commit, and version that were built, when, and with which compiler on which operating system - the input of an analysis, and none of it visible on the Coverity side.

The two halves are not two views of one run and are not merged into one: the build is submitted from the LAMMPS side, the analysis runs on the Coverity servers and can be delayed considerably, so the metrics usually describe an earlier submission than the last one recorded. The dashboard card states them as two things, and where the last submission postdates the day of the last analysis - which is as precisely as the project page dates it - it is marked analysis pending rather than compared against numbers it cannot be part of.

Whichever half cannot be read keeps the values it had, so a Coverity page that refuses the scrape does not take the recorded submission down with it, and an unreachable summary file does not blank out the metrics.

Core developer activity

tools/fetch_team.py collects data/external/team.json: seven counts per member of the core team and calendar month, over the last 13 months. The site renders them on team.html, linked from the repository activity card of the dashboard - a table of the sums over the twelve completed months, and one line chart per member and count below it. The thirteenth month is the one still running: it is drawn, with its last point left hollow, but left out of every sum, so that a partial month cannot read as a drop in activity.

GitHub has no single activity endpoint, so each count is swept separately and bucketed here:

count	where it comes from
commits	the GraphQL commit history of the default branch
PRs opened, issues opened	one GraphQL search per month, by creation date
PRs merged	a second search per month; there is no merged-by: qualifier, so the merged pull requests are listed and their mergedBy counted
reviews, approvals	the reviews themselves, read off the pull requests each member reviewed
comments	the two repository-wide comment listings, conversation and inline

Three of those need care when reading them:

• commits count for whoever wrote them, PRs merged for whoever merged them. In a repository where one maintainer does most of the merging, the merge column says so and is not a measure of the others' activity.
• approvals are the subset of reviews that approved; a review that requested changes or only commented is in the first number and not in the second. The summary text of a review counts as a review, not as a comment.
• the repository, all contributors row of the table has no entry for reviews and approvals: those are collected one member at a time, from that member's own contributions, so there is no repository-wide number to put beside them.

Every count is bucketed by the UTC month of the event itself rather than read off an aggregate endpoint. That is deliberate, and it is worth saying why: the two aggregates GitHub offers for this are both wrong for a monthly breakdown, neither of them noisily.

• The totals of a GraphQL contributions collection are bucketed by day in the contributor's own profile timezone, and a query window snaps out to whole days in that timezone. For a US/Eastern member, one boundary day alone moved 39 commits across a month queried as UTC.
• Its pullRequestReviewContributions report one contribution per pull request rather than one per review, deduplicated across the whole window asked for. A pull request reviewed in two months is credited to one of them, so the count of a fixed month falls as the surrounding window grows. One member's November came out as 14 reviews read from the pull requests, 10 from a one-month collection, and 7 from an eleven-month one.

Neither is used. Every number is counted from the individual events and their own timestamps, so the same month gives the same answer however wide a window it was asked for - which is checked by sweeping a short window and a long one and comparing the months they share.

A full sweep is about 150 requests and takes a couple of minutes, well inside both rate budgets; the REST search API and its 30 requests per minute are avoided throughout in favour of GraphQL search. Nothing is collected incrementally - the whole window is swept each time, so a month that gains a late edit or loses a deleted comment stays correct.

Reading team membership needs a token with the read:org scope, which the repository GITHUB_TOKEN of Actions does not have; the workflow takes one from the TEAM_TOKEN secret (falling back to ISSUE_TOKEN). As in tools/fetch_activity.py, an incomplete sweep is never published: any failure warns and leaves an existing file untouched, so the page keeps showing the last complete set of counts rather than one in which some members silently dropped to zero. Without the file the page is not built and the dashboard does not link it.