Fix: Expand multi-value template variables in prefixPath into multiple paths#109
Open
jixuan1989 wants to merge 2 commits into
Open
Fix: Expand multi-value template variables in prefixPath into multiple paths#109jixuan1989 wants to merge 2 commits into
jixuan1989 wants to merge 2 commits into
Conversation
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Pull request overview
This PR aims to support a common Grafana dashboard pattern by expanding multi-value template variables used inside prefixPath into multiple valid IoTDB prefix paths (instead of producing a single invalid concatenated path).
Changes:
- Updated
applyTemplateVariablesto attempt multi-value expansion forprefixPathusing Grafana’s template variable replacement. - Added Jest unit tests for literal paths, single-value variables, and multi-value expansion behavior.
- Documented the new behavior in the Grafana plugin README.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
connectors/grafana-plugin/src/datasource.ts |
Adds prefixPath expansion logic for template variables (currently implemented via splitting the replaced string). |
connectors/grafana-plugin/src/datasource.test.ts |
Introduces unit tests for the new prefixPath expansion feature (mocks currently don’t reflect typical Grafana pipe formatting). |
connectors/grafana-plugin/README.md |
Documents multi-value variable expansion behavior for the FROM/prefixPath input. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+82
to
+93
| it('should handle mixed literal and template paths', () => { | ||
| mockContainsTemplate.mockImplementation((path: string) => path.includes('${')); | ||
| mockReplace.mockReturnValue('root.app.device1|root.app.device2'); | ||
| const query = { | ||
| ...baseQuery, | ||
| prefixPath: ['root.static.path', 'root.app.${device}'], | ||
| } as IoTDBQuery; | ||
|
|
||
| const result = ds.applyTemplateVariables(query, scopedVars); | ||
|
|
||
| expect(result.prefixPath).toEqual(['root.static.path', 'root.app.device1', 'root.app.device2']); | ||
| }); |
Member
Author
There was a problem hiding this comment.
Same as above — fixed in the force-push. Tests now use mockGetVariables() to provide variable values directly, matching the actual implementation.
Fix two issues with variable expansion in Grafana 11.4 (Scenes architecture):
1. Replace templateSrv.containsTemplate() with regex-based detection.
In Grafana 11.4's Scenes mode, containsTemplate() returns false for
${var} syntax even when the variable exists. Using a regex pattern
ensures reliable detection regardless of Grafana version.
2. Process prefixPath expansion for queries without sqlType field.
Previously only queries with sqlType === 'SQL: Full Customized' were
processed. Queries provisioned via JSON (without explicit sqlType)
were silently skipped.
The expansion logic now:
- Detects variables via regex pattern /\$\{(\w+)(?::[^}]*)?\}|\\b/
- Resolves values from scopedVars first, then falls back to getVariables()
- Handles $__all by expanding to all non-$__all options
- Supports both single-value and multi-value (array) variables
- Fully backward-compatible: literal paths pass through unchanged
Closes #108
jixuan1989
force-pushed
the
fix/grafana-expand-multi-value-prefixpath
branch
from
c977150 to
811dba6
Compare
jixuan1989
changed the title
Comment on lines
+79
to
+81
| if (values.length === 0) { | ||
| values = [templateSrv.replace(varMatch[0], scopedVars)]; | ||
| } |
Member
Author
There was a problem hiding this comment.
Fixed in commit 04db426. The fallback now uses templateSrv.replace(path, scopedVars) on the whole path (not just the token), producing a single entry that preserves prior behavior when the variable cannot be resolved.
- When scopedVars provides $__all, expand using options list instead of treating it as a literal path segment - When variable cannot be resolved, fallback to replace() on the whole path (preserving prior behavior) instead of only replacing the token - Add tests for both cases
Comment on lines
+52
to
+65
| if (scopedVars && scopedVars[varName]) { | ||
| const val = scopedVars[varName].value; | ||
| if (val === '$__all') { | ||
| const allVars = templateSrv.getVariables() as any[]; | ||
| const found = allVars.find((v: any) => v.name === varName); | ||
| if (found && found.options) { | ||
| values = found.options | ||
| .filter((o: any) => o.value !== '$__all') | ||
| .map((o: any) => o.value); | ||
| } | ||
| } else { | ||
| values = Array.isArray(val) ? val : [String(val)]; | ||
| } | ||
| } else { |
Comment on lines
+44
to
+48
| if (varPattern.test(path)) { | ||
| const varMatch = path.match(/\$\{(\w+)(?::[^}]*)?\}|\$(\w+)\b/); | ||
| if (varMatch) { | ||
| const varName = varMatch[1] || varMatch[2]; | ||
| const idx = varMatch.index!; |
|
|
||
| expect(result.prefixPath).toEqual(['root.static.path', 'root.app.device1', 'root.app.device2']); | ||
| }); | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When a multi-value Grafana template variable is used in
prefixPath(e.g.root.application.${device}), the plugin now expands it into multiple valid IoTDB paths instead of producing an invalid concatenated path.Closes #108
Before (Original Plugin)
To achieve "one dropdown filters all panels", users had to:
1. Define a multi-value variable
{ "templating": { "list": [{ "name": "device", "type": "custom", "multi": true, "includeAll": true, "query": "Device 1 : device1,Device 2 : device2,Device 3 : device3,Device 4 : device4,Device 5 : device5,Device 6 : device6,Device 7 : device7,Device 8 : device8", "current": { "text": "All", "value": "$__all" } }] } }2. Hardcode all device paths in every panel + add a transformation
{ "targets": [{ "refId": "A", "sqlType": "SQL: Full Customized", "expression": ["m1"], "prefixPath": [ "root.application.device1", "root.application.device2", "root.application.device3", "root.application.device4", "root.application.device5", "root.application.device6", "root.application.device7", "root.application.device8" ] }], "transformations": [{ "id": "filterFieldsByName", "options": { "include": { "pattern": "time|.*(${device:pipe}).*" } } }] }The variable definition alone is not enough — the plugin cannot expand
${device}in prefixPath, so you must:filterFieldsByNamewith${device:pipe}(which Grafana expands todevice1|device2|...regex) to hide unselected devices client-sideProblems:
time|.*(${device:pipe}).*regex pattern is confusing to new usersAfter (This PR)
1. Define a multi-value variable (same as before)
{ "templating": { "list": [{ "name": "device", "type": "custom", "multi": true, "includeAll": true, "query": "Device 1 : device1,Device 2 : device2,Device 3 : device3,Device 4 : device4,Device 5 : device5,Device 6 : device6,Device 7 : device7,Device 8 : device8", "current": { "text": "All", "value": "$__all" } }] } }2. Use the variable directly in prefixPath (no transformation needed)
{ "targets": [{ "refId": "A", "sqlType": "SQL: Full Customized", "expression": ["m1"], "prefixPath": ["root.application.${device}"] }], "transformations": [] }Key changes:
prefixPath: from N hardcoded paths → 1 template path with${variable}transformations: completely removed — no longer needed. The plugin now expands${device}into multiple paths at query time, so only the selected devices are queried from IoTDB. No extra data is fetched, no post-filtering is required.When user selects
device1anddevice2, the plugin internally expands to:Comparison
${variable}filterFieldsByNameon every panel)Fully Backward-Compatible
This change does not break any existing dashboard configuration:
root.app.device1): no template syntax detected, passed through unchangedfilterFieldsByNamein your panels, it still works — it just becomes redundantexpression,condition,controlfields: continue using simplereplace()as beforeTechnical Details
Two fixes for Grafana 11.4 (Scenes architecture) compatibility:
Replace
templateSrv.containsTemplate()with regex detectionIn Grafana 11.4 Scenes mode,
containsTemplate()returnsfalsefor${var}syntax even when the variable exists and has values. We now use regex/\$\{(\w+)(?::[^}]*)?\}|\$(\w+)\b/for reliable detection across all Grafana versions (9.3+).Process queries without explicit
sqlTypefieldPreviously only
sqlType === "SQL: Full Customized"was processed. Queries provisioned via JSON API (without explicitsqlType) were silently skipped. Now!query.sqlType || query.sqlType === "SQL: Full Customized"covers both cases.