feat(analyze): add log viewer (.bin/.tlog) and onboard log UX improvements for ardupilot#14301
Open
AhmWael wants to merge 13 commits intomavlink:masterfrom
Open
feat(analyze): add log viewer (.bin/.tlog) and onboard log UX improvements for ardupilot#14301AhmWael wants to merge 13 commits intomavlink:masterfrom
AhmWael wants to merge 13 commits intomavlink:masterfrom
Conversation
Introduce a unified Analyze -> Log Viewer workflow and supporting code: add QML pages (LogViewerPage.qml, LoggingSetupPage.qml), a LogViewerController, and a DataFlashLogParser (cc/h) to parse .bin DataFlash logs (signals, parameters, events, mode segments, timestamps). Integrate the module into AnalyzeView (CMake + QGCCorePlugin entries), add docs (index, log_viewer, logging_setup) and expand Log Download docs, and include unit tests for the DataFlash parser. Provides plotting, event markers, zooming, and QML APIs for replay/telemetry integration.
…vert Extract common log-clearing steps into clearLoadedLogState() in LogViewerPage.qml and use it when loading/replacing logs or pressing Clear to avoid duplicated logic and ensure consistent state. Update OnboardLogController::_sortEntriesByTimestamp to group entries with valid timestamps first, sort by time (and id for ties) respecting _sortAscending, and fall back to id ordering for entries without valid time. Remove the "Invert Selection" button from OnboardLogPage.qml to simplify the UI.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds an Analyze View “Log Viewer” workflow to open ArduPilot DataFlash (.bin/.log) logs for basic plotting/parameters/events and integrates telemetry (.tlog) replay access, while also improving onboard log list selection and sorting UX.
Changes:
- Introduces new Analyze pages: Log Viewer (DataFlash + tlog replay entrypoint) and Log Setup (parameter guidance UI).
- Adds a new DataFlash parsing pipeline (signals/parameters/events/mode segments) plus minimal unit tests.
- Improves Onboard Logs UX with select-all tri-state header and timestamp-based sorting toggle.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| test/AnalyzeView/DataFlashLogParserTest.h | Declares new unit test fixture for DataFlash parser. |
| test/AnalyzeView/DataFlashLogParserTest.cc | Adds minimal valid/invalid DataFlash parsing tests. |
| test/AnalyzeView/CMakeLists.txt | Registers new DataFlash parser tests in test build. |
| src/AnalyzeView/OnboardLogs/OnboardLogPage.qml | Enables tri-state select-all UI and adds sort toggle button. |
| src/AnalyzeView/OnboardLogs/OnboardLogController.h | Exposes selection check state and sort order to QML; adds selection/sort invokables. |
| src/AnalyzeView/OnboardLogs/OnboardLogController.cc | Implements select-all/invert selection, selection state computation, and timestamp sorting. |
| src/AnalyzeView/LogViewer/LoggingSetupPage.qml | New page to surface key ArduPilot logging params via Facts plus guidance text. |
| src/AnalyzeView/LogViewer/LogViewerPage.qml | New unified viewer page: open .bin (parse + plot + timeline) and .tlog (replay controls + inspector). |
| src/AnalyzeView/LogViewer/LogViewerController.h | New QML-facing controller API for selected signals, grouping, and colors. |
| src/AnalyzeView/LogViewer/LogViewerController.cc | Implements grouping/selection state and deterministic color assignment. |
| src/AnalyzeView/LogViewer/DataFlash/DataFlashLogParser.h | New QML-exposed DataFlash parser interface (signals/params/events/modes). |
| src/AnalyzeView/LogViewer/DataFlash/DataFlashLogParser.cc | Implements FMT-driven decode and extracts samples/events/mode segments. |
| src/AnalyzeView/LogViewer/CMakeLists.txt | Adds LogViewer C++ sources/includes to build. |
| src/AnalyzeView/CMakeLists.txt | Adds LogViewer subdir and includes new QML pages in AnalyzeView QML module. |
| src/API/QGCCorePlugin.cc | Adds Analyze page entries for Log Viewer and Log Setup. |
| docs/en/qgc-user-guide/analyze_view/logging_setup.md | New user guide page for Log Setup. |
| docs/en/qgc-user-guide/analyze_view/log_viewer.md | New user guide page for Log Viewer. |
| docs/en/qgc-user-guide/analyze_view/log_download.md | Documents new selection/sort capabilities on log download page. |
| docs/en/qgc-user-guide/analyze_view/index.md | Adds Log Viewer/Log Setup links to Analyze View index. |
Codecov Report❌ Patch coverage is ❌ Your patch check has failed because the patch coverage (0.80%) is below the target coverage (30.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## master #14301 +/- ##
=========================================
Coverage ? 24.57%
=========================================
Files ? 734
Lines ? 62165
Branches ? 28705
=========================================
Hits ? 15277
Misses ? 36046
Partials ? 10842
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
Contributor
Build ResultsPlatform Status
All builds passed. Pre-commit
Pre-commit hooks: 4 passed, 46 failed, 7 skipped. Test Resultslinux-coverage: 85 passed, 0 skipped Code CoverageCoverage: 58.1% No baseline available for comparison Artifact Sizes
Updated: 2026-04-30 16:55:40 UTC • Triggered by: Android |
Refactor DataFlash parsing into a ParseResult-based _parseDataFlashFile, add parseFileAsync using QtConcurrent/QFutureWatcher and a parseFileFinished signal. Improve robustness by memory-mapping files, adding vehicle-type detection from MSG text, vehicle-specific ArduPilot mode name mapping, safer timestamp extraction, and better error reporting. Optimize sample handling: store signal samples in QHash, make signalValueAt use binary search, decimate chart points with guaranteed final sample, and maintain min/max timestamps and sample counts. Expose detectedVehicleType property and notify changes. QML: add signal/parameter search UI, debounced timers, filtered models, async parse handling, and UI tweaks (max chart points, ensure final point after decimation). Misc: use unsigned hash and safer modulo in LogViewerController color assignment; remove redundant selection-changed bookkeeping in OnboardLogController.selectAll/invertSelection. Emits and property change notifications updated accordingly.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a new Analyze View workflow for opening and inspecting ArduPilot logs inside QGroundControl (DataFlash .bin/.log and telemetry .tlog replay), and improves Onboard Logs selection/sorting UX to better handle large log lists.
Changes:
- Added Log Viewer and Log Setup pages to Analyze View, including a new DataFlash parsing pipeline and plotting/timeline UI.
- Improved Onboard Logs UX with tri-state select-all, selection helpers, and timestamp sort toggle (newest/oldest with fallback behavior).
- Added user-guide docs and unit tests for minimal/invalid DataFlash parsing.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| test/AnalyzeView/DataFlashLogParserTest.h | Declares unit tests for DataFlash log parsing. |
| test/AnalyzeView/DataFlashLogParserTest.cc | Implements minimal-valid and invalid-input parser tests. |
| test/AnalyzeView/CMakeLists.txt | Registers the new DataFlash parser tests with the AnalyzeView test target. |
| src/AnalyzeView/OnboardLogs/OnboardLogPage.qml | Adds tri-state select-all checkbox and a sort toggle button to the UI. |
| src/AnalyzeView/OnboardLogs/OnboardLogController.h | Exposes selection check state and sort state to QML; adds selection/sort invokables. |
| src/AnalyzeView/OnboardLogs/OnboardLogController.cc | Implements select-all/invert selection, selected count, and timestamp-based sorting. |
| src/AnalyzeView/LogViewer/LoggingSetupPage.qml | New Analyze page with ArduPilot logging parameter guidance via Facts. |
| src/AnalyzeView/LogViewer/LogViewerPage.qml | New unified log viewer UI for .bin parsing/plotting and .tlog replay controls. |
| src/AnalyzeView/LogViewer/LogViewerController.h | New QML-facing controller for log viewer state, signal grouping/selection, and color mapping. |
| src/AnalyzeView/LogViewer/LogViewerController.cc | Implements signal grouping/selection logic and deterministic color assignment. |
| src/AnalyzeView/LogViewer/DataFlash/DataFlashLogParser.h | New QML-facing DataFlash parser interface and result properties. |
| src/AnalyzeView/LogViewer/DataFlash/DataFlashLogParser.cc | Implements memory-mapped parsing, FMT-driven decode, event/mode extraction, and async parsing. |
| src/AnalyzeView/LogViewer/CMakeLists.txt | Adds the LogViewer module sources/includes to the build. |
| src/AnalyzeView/CMakeLists.txt | Includes the new LogViewer subdirectory and QML pages in the AnalyzeView QML module. |
| src/API/QGCCorePlugin.cc | Registers new Analyze pages (“Log Viewer”, “Log Setup”) into the Analyze sidebar list. |
| docs/en/qgc-user-guide/analyze_view/logging_setup.md | New user guide page describing Log Setup usage and troubleshooting. |
| docs/en/qgc-user-guide/analyze_view/log_viewer.md | New user guide page describing Log Viewer workflows and performance notes. |
| docs/en/qgc-user-guide/analyze_view/log_download.md | Updates docs to mention select-all/invert/sort capabilities. |
| docs/en/qgc-user-guide/analyze_view/index.md | Adds Log Viewer and Log Setup entries to the Analyze View index. |
Contributor
Author
|
I hit two Windows/MSVC build issues while testing this branch after syncing with latest 1) MSVC object section limit (
|
Change DataFlashLogParser signal storage from QVariantList to QVector<QPointF> for more efficient storage and binary-search lookups, and update parsing to append QPointF points. Convert signalSamples back to a QVariantList for QML consumers. Add a _parseRequestId to invalidate superseded async parse results and increment it on sync parses. Use tr() for event/error text, remove unused helper methods, and adjust includes/members accordingly. QML: show the signal row only when signals are selected and handle failed .tlog replay startup by showing an error and aborting. OnboardLogController: fix sorting comparator to handle null lhs/rhs correctly to avoid incorrect ordering/crashes.
Collaborator
|
For the Windows things, how come Windiows CI isn't failing? |
Contributor
Author
|
@DonLakeFlyer It's a weird thing after i finished all my changes and everything was working, I pushed everything, then I pulled the latest commits from remote which where about two new commits, build started failing. |
Collaborator
This, makes sense you should do that.
I'd rather hold off on this to make sure it isn't a fluke and see CI fail. Adding it will slow things down a bit. |
Contributor
Author
|
@DonLakeFlyer Ok should i add it to this PR? |
Collaborator
yes |
Introduce an inline _qCompareCoords function to compare QGeoCoordinate values with a meter tolerance (default 1.0m) and emit a QFAIL with a formatted message when the distance exceeds tolerance. The old complex varargs macro implementation was removed and QCOMPARE_COORDS now forwards to _qCompareCoords for simpler, type-safe usage and clearer formatting (using QStringLiteral). Existing QVERIFY_COORDS_NEAR is left unchanged.
Contributor
Author
|
@DonLakeFlyer Done |
Add detailed ArduPilot error/event descriptions and improve LogViewer UI. DataFlashLogParser: introduce _ardupilotErrDescription and _ardupilotEventDescription helpers, parse Subsys/ECode/Id as integers and use the new human-readable mappings for ERR/EV messages. LogViewerPage.qml: style search fields' text and placeholder colors, add a "Clear Selected" button, fix signal checkbox binding to use selectedSignals, and add section labels for Modes/Signals/Events. LoggingSetupPage.qml: add FactComboBox and FactBitmask support for enum/bitmask params, show combined bitmask value, and restrict FactTextField visibility to non-enum/non-bitmask facts.
Contributor
Author
|
@DonLakeFlyer
|
Contributor
Author
|
@DonLakeFlyer |
Collaborator
|
Gonna take me a bit to get to this in detail. Still digging out from being gone for a month. |
Collaborator
|
So one fundamental problem here is that we are mid-transtion from Qt Charts to the new Qt Graphs. I'm working on it here #14234. This will need to move to that, but it will need to wait on my pull before to go in to set up the build infrastructure for using Qt Graphs. Sorry about that. Depending on the usage its not a massive change to switch. But it also not simple. |
Contributor
Author
|
@DonLakeFlyer That's completely understandable. I noticed the transition work moved over to #14315, so I'll keep an eye out for that to get merged. Once the Qt Graphs build infrastructure is in place, I will make the switch here. |
Extract message rows from DataFlash logs and improve parsing performance, while migrating the bin chart UI to QtGraphs with rendering/decimation fixes. - DataFlashLogParser: precompute fully-qualified signal names per format to avoid per-message QString::arg() overhead; collect numeric samples in a single pass; add support for MSG rows (result.messages) and expose messages as a Q_PROPERTY with change notifications; ensure clear() and async parse flow emit messagesChanged. - LogViewerController: add setSignalSelected(invokable) to explicitly set selection state; replace mode color hashing with a small deterministic palette to produce visually distinct mode colors. - LogViewerPage.qml: replace QtCharts import/ChartView with QtGraphs/GraphsView; implement two-pass decimation (compute axis ranges first, then create series) to stabilize layout; add selected-signal categorical color palette; add Messages tab and message list UI; various UI/UX fixes (reduced signals pane height, improved cursor popup positioning, chart nudging to force full-series render, use components for line/scatter series). These changes improve parse speed, expose textual messages to the UI, and fix rendering/interaction issues in the log bin chart.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new Analyze View workflow for loading/inspecting ArduPilot DataFlash (.bin/.log) logs and telemetry (.tlog) logs, and improves onboard log selection/sorting UX, along with initial unit tests and user documentation.
Changes:
- Introduces a new Log Viewer Analyze page (QML + controller) and a Log Setup page for ArduPilot logging parameters.
- Adds a new DataFlashLogParser pipeline (sync + async parse) and a minimal unit test suite.
- Improves Onboard Logs UX with select-all (tri-state) and timestamp sorting controls, plus related docs updates.
Reviewed changes
Copilot reviewed 20 out of 20 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| test/UnitTestFramework/UnitTestCoords.h | Refactors coordinate comparison helper into an inline function and simplifies the macro. |
| test/AnalyzeView/DataFlashLogParserTest.h | Adds a new unit test fixture for the DataFlash parser. |
| test/AnalyzeView/DataFlashLogParserTest.cc | Implements minimal valid/invalid DataFlash log parsing tests. |
| test/AnalyzeView/CMakeLists.txt | Registers the new AnalyzeView unit tests with CMake. |
| src/AnalyzeView/OnboardLogs/OnboardLogPage.qml | Adds tri-state select-all checkbox and a sort toggle button to the Onboard Logs UI. |
| src/AnalyzeView/OnboardLogs/OnboardLogController.h | Exposes selection state + sorting properties and new QML-invokable actions. |
| src/AnalyzeView/OnboardLogs/OnboardLogController.cc | Implements select-all/invert selection, selection state computation, and timestamp sorting. |
| src/AnalyzeView/LogViewer/LoggingSetupPage.qml | Adds a new Analyze page to guide/configure common ArduPilot logging parameters via Facts. |
| src/AnalyzeView/LogViewer/LogViewerPage.qml | Adds the new Log Viewer UI: open .bin/.tlog, plot signals, show parameters/events, and replay controls. |
| src/AnalyzeView/LogViewer/LogViewerController.h | Adds a QML-exposed controller for log viewer UI state (selected signals, grouping, status, etc.). |
| src/AnalyzeView/LogViewer/LogViewerController.cc | Implements signal grouping/selection state and color assignment helpers. |
| src/AnalyzeView/LogViewer/DataFlash/DataFlashLogParser.h | Declares the new QML-exposed DataFlash parser API and properties. |
| src/AnalyzeView/LogViewer/DataFlash/DataFlashLogParser.cc | Implements FMT-driven DataFlash parsing, extraction of samples/params/events/modes, and async parsing. |
| src/AnalyzeView/LogViewer/CMakeLists.txt | Adds the new LogViewer module sources and include paths to the build. |
| src/AnalyzeView/CMakeLists.txt | Includes the new LogViewer subdirectory and adds new QML pages to the AnalyzeView module. |
| src/API/QGCCorePlugin.cc | Registers the new Analyze pages (“Log Viewer”, “Log Setup”) in the Analyze view page list. |
| docs/en/qgc-user-guide/analyze_view/logging_setup.md | Adds user guide content for the new Log Setup page. |
| docs/en/qgc-user-guide/analyze_view/log_viewer.md | Adds user guide content for the new Log Viewer page. |
| docs/en/qgc-user-guide/analyze_view/log_download.md | Updates Log Download docs to reflect new selection/sorting capabilities. |
| docs/en/qgc-user-guide/analyze_view/index.md | Adds Analyze View index links for Log Viewer and Log Setup pages. |
Comment on lines
+40
to
+54
| // Visually distinct categorical palette for selected-signal series. | ||
| // Position-based picks avoid hash collisions producing similar colors. | ||
| readonly property var _signalChartColors: [ | ||
| "#1E88E5", // blue | ||
| "#E53935", // red | ||
| "#43A047", // green | ||
| "#FB8C00", // orange | ||
| "#8E24AA", // purple | ||
| "#00ACC1", // cyan | ||
| "#FDD835", // yellow | ||
| "#D81B60", // pink | ||
| "#6D4C41", // brown | ||
| "#546E7A", // blue grey | ||
| "#3949AB", // indigo | ||
| "#00897B", // teal |
Comment on lines
+9
to
+10
| inline void _qCompareCoords(const QGeoCoordinate &actual, const QGeoCoordinate &expected, double toleranceMeters = 1.0) | ||
| { |
Comment on lines
+24
to
+27
| .arg(expected.longitude(), 0, 'f', 7) | ||
| .arg(expected.altitude(), 0, 'f', 2); | ||
| QFAIL(qPrintable(message)); | ||
| } |
Localize log viewer strings and tidy up UI bindings. Replace hardcoded "Unknown" and "Other" with tr()/DataFlashLogParser::tr() for translation; add eventTypeLabel in LogViewerPage.qml to show localized event type names (Mode, Event, Error) and use it for event labels; change OnboardLogPage.qml to use a Binding for checkState to preserve reactivity; remove an outdated "Inverting current selection" line from the log download docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds a new Analyze View Log Viewer workflow for ArduPilot logs and improves onboard log management UX.
What this solves
QGroundControl previously lacked an integrated DataFlash (
.bin) log viewer and had limited telemetry replay analysis, forcing users into external tools for post-flight debugging/tuning. Onboard log list management was also limited.What’s included
.bin,.log).tlog) through replay flowType of Change
Testing
Platforms Tested
Flight Stacks Tested
Screenshots
.binloaded): signal list + chart + timeline markers + cursor popup showing synchronized values/mode/eventsChecklist
Related Issues
By submitting this pull request, I confirm that my contribution is made under the terms of the project's dual license (Apache 2.0 and GPL v3).