Skip to content

Release/0.4.0 - #156

Open
olivhoenen wants to merge 71 commits into
mainfrom
release/0.4.0
Open

olivhoenen wants to merge 71 commits into
mainfrom
release/0.4.0

Conversation

@olivhoenen

Copy link
Copy Markdown
Contributor

Notes for release 0.4.0

Frontend

Backend

  • Remove limitation on interpolation from same IDS/same URI
  • Improved UDA backend compatibility by automatically appending cache_mode=none to UDA URIs unless already specified
  • Increasing robustness when using different IMAS backends.
  • Fixed a bug in recursive metadata serialization to ensure the show_error_bars flag is correctly propagated to child nodes

Internal reactivity improvements

  • Split configuration and data store object
  • Reduce amount of redraw and backend requests from frontend

Measuring first, so every later reactivity change has a before/after number
instead of an assertion.

The benchmark builds the canvas from issue #121: a 2D
equilibrium/time_slice/profiles_2d/psi heatmap plus a 1D grid with two traces.
It asserts on counts - backend requests and Plotly redraws - and never on
wall-clock times, which are noise on a shared runner; timings are still
recorded and printed.

Renderer-side counters live in utils/perf.ts and are installed only when
E2E_TEST=true, matching the convention already used to disable Mantine
transitions. Plotly redraws are counted through react-plotly's public
onAfterPlot event, so nothing depends on the bundled plotly instance.

The spec sits in src/tests/perf/, which keeps it out of `npm run test:e2e`
(whose glob is not recursive) and behind the new `npm run test:perf`.

Three of its four guards fail today, which is the point of committing them:
toggling one panel's edit flag redraws the untouched panel 6 times, two slider
steps cost 12 redraws (3 on an unrelated panel), and reopening a metadata tab
re-downloads 2 plot_data payloads. BASELINE.md records the numbers.

The disruption dataset is used rather than the scenario one because it carries
several time slices, so the coordinate sliders are actually operable.

Assisted-by: Claude/opus-5
A data entry does not change while IBEX is open, so a response stays valid for
the whole session. fetchFromApi is the single place every renderer->backend GET
goes through, so the cache sits there.

The cache stores the response body TEXT and each caller parses its own copy.
That is a correctness requirement, not an optimisation: fetchDataPlot
post-processes the parsed response in place (it renames data.name, rewrites
coord.target, tensorizes irregular data, runs the non-idempotent
transformComplexData, then replaceNullsWithNaN) and five call sites then alias
the result straight into the store with `plot.yData = response.data.value`,
where transposeAxis and applyRange go on to mutate it in place. Sharing one
parsed object would alias one grid's data to another's and corrupt the cached
entry with it.

Also: in-flight de-duplication so concurrent identical requests share one round
trip, an LRU bounded by total and per-entry bytes so a large 2D payload cannot
evict the session (it is de-duplicated but not retained), and getConfig()
resolved once instead of crossing the Electron IPC boundary on every request.
/info/version opts out - the header polls it as a liveness probe.

Measured on the benchmark canvas: reopening a metadata tab drops from 6
requests to 1 and no longer re-downloads plot_data at all, which is the third
benchmark guard now passing. First open drops from 4 requests to 2.

resetAppState() clears the cache between specs. Without it an earlier spec
warms the cache and a later one exercises a different path - which is exactly
how two data-manipulation specs failed while passing in isolation.

Assisted-by: Claude/opus-5
react-plotly.js decides whether to redraw by comparing data, layout and config
by reference, so an object literal passed inline forces a full Plotly.react on
every render however little changed. Both plot components did exactly that, and
the heatmap also rebuilt its trace array with a full copy of the z matrix each
time.

- config moves to two shared frozen constants (plotConfig.ts); it only ever
  varied by staticPlot, which has two values.
- the heatmap trace array becomes a useMemo, and x/y/z are copied once where
  they are computed - an effect that only runs when the data or coordinates
  change - instead of on every render. Plotly keeps and mutates what it is
  given and those vectors index into the store, so exactly one copy is kept.
- isMatrixPlottable built a tf.tensor over the whole array, never disposed,
  purely to read whether the last dimension was non-zero, and ran in the JSX
  body of both plot components several times per render. A plain scan computes
  the same thing allocating nothing, keeping the old semantics including the
  cases where tf.tensor used to throw (ragged, mixed depth, mixed scalar types,
  complex pairs). Heatmap2D no longer imports tensorflow at all.
- getVectorData deep-cloned every coordinate, including its full data array,
  only to sort by axeIndex and read valueIndex. It now projects onto those two
  numbers first. This runs on every slider tick and on every plot's render path.

Also fixes two error-reporting bugs that in-flight de-duplication exposed: the
HTTP status now travels on the error object rather than in a closure the joining
caller never runs - without it an expected 464 on an error-band node lost its
status and was reported to the user as "Unable to contact the server" - and
handleError no longer notifies twice for one shared rejection.

resetAppState dismisses leftover notifications: they expire on a timer, so the
suite was relying on being slow enough for that to happen between tests.

Measured: toggling a UI flag drops from 11 redraws to 2, two slider steps from
12 to 6, reopening the metadata panel from 19 redraws and 6 requests to 9 and 0.
The slider no longer redraws unrelated panels at all, so that guard now passes.

Assisted-by: Claude/opus-5
The plot components and the grid panel only ever wrote to the store, yet each
subscribed to all of it, so any change anywhere re-rendered every panel and
handed Plotly new props.

- SimplePlotly and Heatmap2D no longer subscribe at all: both read
  useIbexStore.getState() inside the handlers and effects that write, which is
  the pattern handleDeleteGrid already used. This also removes a
  structuredClone of every plot's data that ran just to change a title.
- GridLayoutPlot subscribes only to whether any URI is selected - the one value
  it reads while rendering - and reads the rest at call time. Its three panel
  handlers lose their [active] dependency and become stable for the component's
  lifetime.
- GridLayoutPlot is memoized. That only works because handleEditGrid now keeps
  the identity of grids it is not changing, instead of rebuilding every grid
  object to set two booleans that were already false.

Measured: toggling a UI flag drops from 42 component renders to 14, and two
slider steps from 40 to 28.

The untouched panel still redraws once per toggle, down from 6. Closing that
needs the data/config store split: the configuration is still replaced whole on
every write, so VisualizationPlot re-renders and react-grid-layout clones every
child on the way through.

Assisted-by: Claude/opus-5
Entering edit mode sets `static` on one grid, which changes the layout
react-grid-layout derives from its children, so RGL reports onLayoutChange.
handleUpdateLayout then rebuilt every grid object from that report - including
the ones that had not moved - which handed the memoized panels new props and
redrew the untouched heatmap.

- Grids whose x/y/w/h/static match the report keep their identity.
- A report that changes nothing writes nothing, so a layout event no longer
  flags the configuration as unsaved on its own.
- minH/minW are derived the same way the data-grid prop derives them, instead
  of being hard-coded to a value that only suited grids with coordinates.
- handleUpdateLayout reads the store at call time and loses its [active]
  dependency, the pattern the panel handlers already use.

This closes the last benchmark guard: toggling one panel's edit flag now
redraws only that panel (the untouched heatmap goes 6 -> 0 against the original
baseline) and costs 4 component renders instead of 42. All four scenarios in
`npm run test:perf` pass, and the e2e suite is green.

The container deliberately subscribes to `active` rather than `active.dataPlot`:
handleNewPlot pushes a new grid into that array in place, so its identity does
not change when a panel is added.

Assisted-by: Claude/opus-5
node_info passed `show_error_bars` positionally into get_node_info, whose
second parameter is `recursive`. With the "see error bars" preference on, every
call therefore walked and serialized the metadata of the entire subtree - which
NodeInfoResponse then discarded, since NodeInfoChildModel has no `children`
field - and the error bar filter never ran at all.

The filter being skipped had no visible effect (the recursive branch returns
every child, which is what the flag asks for), so the cost was the only symptom:
on the disruption fixture, node_info on summary:0 takes 0.20 s with the flag on
against 0.13 s with it off, while on equilibrium:0 the difference is lost in the
filled-path scan that dominates there.

- Pass the flag by keyword.
- Forward it through _jsonify_metadata's recursive branch, which dropped it.
- Guard the delegation in a test: the response looks identical either way, so
  assert on the arguments the endpoint hands the service.

Assisted-by: Claude/opus-5
react-plotly.js compares `layout` by reference, so each of the fifteen effects
that called setLayoutPlot handed it a new identity and cost a redraw as a panel
appeared: seven in SimplePlotly, five in usePlotLayout, six in Heatmap2D.

Each component now derives its whole layout in one useMemo:

- usePlotLayout returns a memoized {xaxis, yaxis, yaxis2} fragment (grid display
  and axis types) instead of taking a setter. Its rule that string x data forces
  a category axis, and that a category axis returns to linear once the data is
  numeric, is unchanged - but it no longer assigns to itemDataGrid.xAxisData in
  place. The configured type is persisted and read back by the customization
  panel, so it is written through updatedConfiguration, guarded so it writes
  only on the two transitions the effect handled.
- SimplePlotly derives title, height, width, the axis titles and the whole
  y2 block. Its `dataEntries` state existed only to gate those effects and is
  gone; the titles derive from itemDataGrid.plot directly.
- Heatmap2D derives the same, plus the 1:1 ratio (which no longer needs a state
  mirror of forceXyRatio nor a structuredClone of the layout to patch two
  fields) and the category y axis that init3DAxis used to push in.
- What the user does with the mode bar cannot be derived, so it stays in state
  and is merged last: rebuilding the layout never discards a zoom or pan.

Also fixes a stale width: the effect depended on [width] but read
layoutPlotWidth, which also depends on how many coordinate sliders are shown.

Redraws when a panel appears: 9 -> 6 on a metadata revisit, 3 -> 2 on first
open, i.e. 19 -> 6 against the original baseline. The four benchmark guards and
the 20 e2e specs pass, and the rendered layouts were checked against Plotly's
_fullLayout (axis titles, types, grid, forced ratio, y2 side).

Assisted-by: Claude/opus-5
Two kinds of waste, both on the paths that run per interaction rather than per
fetch.

Round trips that were taken one at a time:
- The geometry overlays fetched r and z (outline), r/z/width/height
  (rectangle) and r/z/length_alpha/length_beta/alpha/beta (oblique) one after
  the other, although the nodes are independent: up to six sequential round
  trips per overlay, now one Promise.all.
- Error bands fetched the upper band, waited, then fetched the lower one; same
  fix, in both the interpolated and the plain branch.
- The interpolation <Select> stayed enabled while a method was in flight, so
  changing it again started a second grid-wide refetch and the two results
  raced to write the grid. It is now disabled while loading, and a response
  whose request has been superseded is dropped.

Copies made to change a field:
- Toggling "error bands" and switching plot type cloned the whole
  configuration - every fetched array - to set one field on one grid. They now
  copy that grid only and keep the identity of the others, so the memoized
  panels are not re-rendered either. Closing the customization panel likewise.
- SimplePlotly cloned itemDataGrid.plot and itemDataGrid.coordinates on every
  change of either, i.e. on every slider tick. getErrorsAreaToPlot only writes
  connectgaps, customdata and hovertemplate onto each plot, and reads the
  coordinates, so the copy is now the plot objects plus the two vectors handed
  to Plotly; yData and the coordinates are shared.
- Deciding whether two grids share a coordinate compared
  JSON.stringify(a) === JSON.stringify(b), serializing both coordinate arrays
  of every grid on every slider tick. isSameAxisData walks them with an early
  exit instead, keeping the old semantics (NaN equals NaN, null equals
  undefined) because JSON.stringify wrote both as null.

The benchmark counts are unchanged - these are cost-per-operation, not
redraw-count, fixes - and the fixtures are too small for the timings to show
it. The e2e suite passes, but note it does not cover error bands: the spec that
did needs a WEST pulse file and is commented out upstream, and no node in the
local fixtures carries error bar data.

Assisted-by: Claude/opus-5
The committed benchmark runs on the e2e fixtures, which are small enough that
the timings say little. The same scenarios were run by hand against a
production ITER entry (134173/106: an equilibrium with 720 time slices,
plotting profiles_2d/psi as a heatmap - the case in issue #121) and compared
with commit 5614230, the benchmark before any fix.

Plotting the heatmap: 8 -> 4 redraws, 36 -> 26 renders, 6.4 s -> 4.7 s.
Stepping a coordinate slider: 6 -> 4 redraws per step. Toggling a UI flag:
3 -> 1 redraw. No change in the number of backend requests, which was already
minimal for this canvas.

Also records why the run has to be driven from the DOM rather than through
getTestState, and that what is left of the plotting cost is the payload: the
backend does not downsample 2-D nodes.

Assisted-by: Claude/opus-5
Three defects on the error-band path, none of them covered by a test: the CI
fixtures carry no error data, so this was checked by hand against the WEST
entry 58463/0 and a fixture written for the symmetric case.

A band trace never copied its main trace's `yaxis`, so a band belonging to a
y2 trace was drawn against y1 - Plotly's default - and dragged y1's autorange
onto y2's scale. On a temperature/density panel, switching bands on flattened
temperature (~1e2 eV) against the axis floor of a density band (~1e19 m^-3).

The two band nodes were fetched together and neither was formatted until both
had arrived, so one missing node threw away the other. A quantity that stores
only `_error_upper` - the IMAS convention for a symmetric error - therefore
got no band at all, although `getErrorsAreaToPlot` has always known how to
mirror a single band around its trace. Each node is now fetched on its own and
whichever exists is drawn; only when both are absent is that reported, as a
warning naming the node, since it is the expected case for most quantities.
The warn and error branches were also the wrong way round.

Finally `userRelayout`, which carries what the user did with the mode bar
across layout rebuilds, accumulated the raw `plotly_relayout` payload and was
never pruned: the ranges of an old zoom outlived the autoscale meant to clear
them, and both outlived the panel being pointed at other data. It is folded
per axis now, `range` and `autorange` replacing one another, and merged inside
each axis rather than at the top level, which is what makes a saved zoom
actually restore. It resets when the plotted nodes change, with indices
normalised away so that stepping a coordinate slider keeps the zoom.

Assisted-by: Claude/opus-5
`getStateHandler` handed Electron's structured clone - and then the WebDriver
JSON bridge - the whole store on every `getTestState()` poll. The store owns the
fetched matrices, so on a production entry that is the psi matrix, per poll, in a
`waitForValue` loop. BASELINE.md recorded the consequence and told the next
author to drive the UI from the DOM instead; measured again here, on
imas:hdf5?path=/work/imas/shared/imasdb/ITER/3/134173/106 with coordinate shapes
[871,1,65] [871,1,129] [871,1] [871], WebDriver gives up with ScriptTimeoutError
before the renderer finishes serializing. With the projection the same poll costs
52 ms once and 5-6 ms after that, for a 17 kB snapshot.

utils/testState.ts drops four families on the way out - plot[].yData,
plot[].error_bands[].yData, coordinates[].data and geometries[].x/.y - and
nothing else. It is a denylist rather than an allowlist on purpose: a spec
reading a field nobody thought to enumerate keeps working, and only those four
can go missing. The derived vectors still cross, because they are a single row
whatever the entry's size and because plot-ui.spec.ts asserts exact floats on
plot[0].y.

The inverse is not optional and ships in the same commit. Three sites
(dataManipulation.ts, reactivity.perf.spec.ts twice) read the whole state and
write it straight back to flip one field, and `setState` replaces
configurations/active wholesale - so a projection without `mergeTestState` would
have the first such round trip silently empty every plot in the store, and the
spec that failed would not be the one that caused it. Grids carry a
`__payloadsOmitted` marker so the merge knows which ones to rehydrate; the
fixtures in tests/utils/state.ts have no marker and stay authoritative.

Also: `active` and its entry in `configurations` are re-linked to one object
after a merge, which is the invariant `updatedConfiguration` maintains and which
`setState` could previously break; and App stops subscribing to the store. It is
the root component, so that subscription re-rendered the whole tree on every
write, for a bridge only the e2e suite uses - the handlers read at call time
instead.

No renderer behaviour changes and the benchmark counts are identical to stage 5
across all five scenarios. Full e2e suite green, 20 passing.

Assisted-by: Claude/opus-5
`isEditing` was a boolean on every grid that only one grid could ever hold:
handleEditGrid cleared it on all the others on its way through, and stage 4 had
to add explicit identity preservation so that clearing it did not re-render
every panel. Expressing the singleton as a singleton - `editingGridId` in a new
`ui` slice - makes entering or leaving edit mode write one string and touch no
grid object at all. The panels read it through a boolean selector, so only the
two whose flag actually flips re-render, and the plot components take it as a
prop instead of reading it off the grid.

`static` went with it, and that one is the interesting half. It was a grid field
always equal to `isEditing`, and it is what made the stage-4 write cycle
possible: setting it changed the layout react-grid-layout derives from its
children, so RGL reported onLayoutChange, and handleUpdateLayout wrote the
report back. It is now derived where the `data-grid` prop is built and dropped
from what RGL reports, so the cycle cannot form rather than being unpicked after
the fact. Nothing was persisting it anyway - DataGridPlotToSave never had it.

Two consumers needed more than a rename:

- fetchErrorBandsInConfig located its grid with `find((d) => d.isEditing)`, so
  it returned early whenever nothing was being edited. Giving each panel its own
  id looks equivalent and is not: HoverButtons' effect runs on mount for every
  panel with displayErrorBand defaulting to true, so every panel started
  fetching error bands as it appeared. The benchmark caught it - metadata panel
  revisit went 0 requests to 12 - and it now takes the editing grid's id,
  whichever grid that is.
- SimplePlotly and Heatmap2D guarded their title write on the grid's flag, but
  the customization and metadata panels render a detached copy of a grid whose
  flag was copied with it, so a preview could write a title into the store.
  `isEditing` is optional there and defaults to false, which is what the comment
  in Heatmap2D ("to prevent from updating in customization") always said the
  guard was for.

The e2e bridge grows an `editingGridId` field, typed as its own `TestState`
rather than the whole store, and the perf spec's setGridEditing stops rewriting
every grid to flip one flag.

All five benchmark scenarios are identical to stage 6 - the counts were already
at their floor here; what changed is what produces them. The remaining 1 redraw
on the toggle is the toggled panel itself, whose Plotly config genuinely
changes. Full e2e suite green, 20 passing.

Assisted-by: Claude/opus-5
`metadataGridLayout` and `customizedGridLayout` follow `isEditing` into the `ui`
slice. Which panel is open is not part of the document - nothing saved it, and
`ConfigurationToSave` never had either field - so opening or closing one now
writes a single field instead of replacing the configuration and every grid
reference in it. `TreeLibrariesAccordion` loses the two
`useMemo(..., [JSON.stringify(x)])` stabilisations it needed, because both
selectors now return plain ids.

Moving them exposed an invariant the old model held by accident: while the ids
lived on the configuration, clearing the configuration took them with it. As
session state they outlive it, and a stale `customizing` is not harmless -
TreeLibrary.handleDisableTree disables the whole node tree whenever a panel is
open, so an id belonging to a configuration that no longer exists leaves the
tree dead with no panel on screen to explain why. Eight e2e specs failed on
exactly that, every one of them reporting that checking a node plotted nothing,
and none of them pointing anywhere near the cause.

So the invariant is now the store's, not the callers': `pruneUiState` drops any
ui id that does not name a grid of the active configuration, and `setActive`,
`removeConfiguration` and `setState` apply it. Both slices are typed against the
whole store so the configuration slice can enforce it. A field added to `ui`
that references a grid has to be added there too.

Benchmark counts identical to stage 6 across all five scenarios. Full e2e suite
green, 20 passing.

Assisted-by: Claude/opus-5
First half of the data/configuration split: the bulk arrays get a name, a
lifetime and somewhere to live that is not the configuration. Nothing reads
through the registry yet - the refs sit alongside the arrays they name - so this
commit cannot change behaviour, and the benchmark counts confirm it did not.

stores/payloadRegistry.ts keys a payload by the request that produced it, in the
same canonical form requestCacheKey computes, so a payload and its cached
response body are one identity rather than two - which is what later lets the
two caches merge. Registration happens inside fetchDataPlot and fetchFieldValue,
where the request is known and the post-processing has finished, rather than at
the twenty-odd call sites that store the result. requestCacheKey now
canonicalises a relative endpoint to the same key as the absolute URL it
becomes, so the fetch layer and the cache cannot disagree.

It is deliberately a module-level Map and not a zustand slice. Nothing renders a
payload directly, so one arriving must not notify a subscriber; and keeping it
out of the store keeps it out of getState(), which the e2e bridge serialises,
and out of every structuredClone of a configuration.

Lifetime is mark-and-sweep from a store subscription, debounced onto an idle
callback. Reference counting would need a matching release at all 24
configuration writers and would leak, or free too early, the first time one was
missed; sweeping from what the store can reach means deleting a grid or a
configuration needs no code at all. DataplotCustomization holds a detached copy
of a grid the store cannot see, so pin/unpin exist for it and pins are roots.

It sweeps for real rather than in audit mode, which is the opposite of the
cautious choice and is the right one here: the registry holds a second reference
to every payload, so a sweep that frees nothing is a memory leak, while an
over-eager sweep has no observable effect as long as nothing reads through the
registry. Audit mode stays for the stages that will.

Where a transform still replaces a payload - transposeAxis, the applyRange
family, tensorising a coordinate - the ref is cleared rather than re-derived.
Those are genuinely different payloads and deriving keys for them is what the
transposition and range stages do.

Also adds `npm run test:unit`: mocha, ts-node and chai are already
devDependencies, so this is a script and a CI workflow, not a toolchain. 16
tests over the key algebra and the sweep, no Electron, ~20 ms. Imports there
must be relative - ts-node does not resolve the `src/*` alias without
tsconfig-paths.

The benchmark grows two cumulative columns, payloads and elements. On the
benchmark canvas they read 8 / 25785, rising to 10 while the metadata panel
fetches and falling back afterwards: that is the sweep, visible.

Full e2e suite green, 20 passing. All five benchmark scenarios unchanged.

Assisted-by: Claude/opus-5
Swapping two axes rebuilt every matrix in the grid with tfjs and threw the
previous one away, so swapping back rebuilt it again. A transposed matrix is
now a payload in its own right, named by composing the permutation onto the
key of the array it came from.

Composition is what makes it useful. A transposition permutes axes -
out.dim[k] = in.dim[p[k]] - so applying q to an array already held as the base
under p leaves the base under p[q[k]]. Keys never stack: every axis order of
one payload names one entry, whichever route reached it, and a swap followed by
the same swap back composes to the identity and names the untransposed array
itself. The benchmark's new scenario pins that - swapping derives one payload,
swapping back derives none and fetches nothing.

The sweep now treats a payload and the views derived from it as one family, so
a live view keeps its base and a live base keeps its views. Resident elements
therefore go up, 25785 to 50985 on the benchmark canvas, and that is the trade:
freeing the untransposed matrix as soon as the user looks at the transposed one
would make going back cost a full transpose. A deleted grid still frees the
whole family.

Three deep copies on that path are gone with it, replaced by
cloneGridStructure, which copies every object a transform assigns fields on and
shares the arrays hanging off them: swapAxis deep-cloned every grid of the
active configuration to modify one, getErrorYVectors deep-cloned every error
payload on every slider tick, and transposeDataGrid cloned coordinates only to
read axeIndex off them. Sharing arrays is sound because nothing in the renderer
writes into a payload - transforms replace them wholesale.

The stride scheme the plan called for is deferred to stage 11, where something
finally reads through it; until the derived vectors leave the store every
transform materialises at the boundary anyway.

20/20 e2e, 5/5 benchmark with every pre-existing count unchanged, 30 unit tests.

Assisted-by: Claude/opus-5
A range used to be applied by cutting the array in the store down to size,
which made it one-way: the wider data was gone, and the only way back was to
ask the back end again. `restoreRange` therefore refetched once per trace plus
once per error band, purely to recover what the trim had destroyed. Widening a
range, or adding a trace to a grid that already had one, needed `oldRange`
threaded through five async functions to turn an absolute range into an offset
into an array that had already been cut, and `rangeAlreadyAppliedInPlot`,
flipped by membership of a `newPlotsUri` list, to guess whether a particular
trace had been narrowed yet. None of it had a test.

A range is now a window on the payload: absolute bounds against the array the
back end sent, named `...#value|range:3:119-177`, with the payload it was cut
from still resident. Windows replace rather than stack, so applying [40,120]
then [50,60] names the same entry as applying [50,60] to the untouched payload,
and asking for no window names the payload itself. Re-applying a grid's ranges
is therefore idempotent, which is what let `applyRangeInCoord`,
`applyRangeInPlot`, `trimCoordData`, `trimPlotData` and
`formatTrimmedCoordinate` collapse into one `applyRangesToGrid`, and deleted
`oldRange`, `rangeAlreadyAppliedInPlot`, `shouldApplyRangeOriginInCoord` and
the `newPlotsUri` plumbing outright. `applyRange` is pure now.

Restoring lands on the full window rather than on the payload itself, which
costs one slice the first time and nothing after: the payload is the response
as parsed, in double precision, while every window is a tfjs slice and so
single precision, and landing on the payload would move every value the moment
a range was dropped. The same argument decides where a typed bound is resolved
- against `Math.fround` of the coordinate, because a bound typed as 0.6 has to
pick the point the axis labels 0.6. The old code got that by accident and
inconsistently, resolving the first range of a grid against the raw response
and every later one against a float32 array.

Measured on the benchmark canvas: restoring a range goes from one plot_data
per trace to none, and re-applying a range used before adds zero derivations.
Resident elements rise from 50985 to 89100 and stay, which is the price of the
restore no longer destroying anything; it is one family, so deleting the grid
still frees all of it. plot-ui.spec.ts's range test passes unmodified, e2e
20/20, benchmark 6/6, unit 39.

Assisted-by: Claude/opus-5
A trace's `y` is one row of a payload, taken with the integers the grid's
coordinates carry, and its `x` is the vector of the axis on display. Both are
determined by payload plus cursor, so storing them stored a copy that every
writer touching either half had to rebuild. `handleUpdateCoordinate` was 130
lines doing exactly that on every slider tick, for the grid and for every grid
synchronized with it: a new x, a new y, a new customdata and a new array per
error band, each copied out of the payload.

It is now one store action, `setCursor`, writing integers and labels and no
array at all - so a tick costs the same whatever the payload weighs. What is
drawn comes from a new `derive/vectors.ts`, called on the render path and
memoised on the payload array itself and, within that, on the cursor. Moving one
grid's cursor therefore cannot invalidate another grid's rows, and two
synchronized grids at the same cursor derive once between them. The map is weak
on the payload, so a family the sweep frees takes its derived rows with it.

Heatmap2D held x, y, z and three axis descriptors in useState, filled by a chain
of effects; Plotly compares `data` by reference, so each link was a separate
draw. They are one useMemo now.

`isSameAxisData` is deleted - it walked both coordinates element by element on
every tick to decide whether a synchronized grid showed the same data, which
`setCursor` now answers with a payload-key compare, and answers correctly: two
grids can hold equal numbers from different nodes. `limitSlidersToMaxLength`
survives as `clampCursors`, because a cursor is still an index that narrowing a
range can leave out of bounds.

`plot.x`, `plot.y`, `plot.customdata` and `error_bands[].array` are no longer
written anywhere. `getErrorsAreaToPlot` became `buildTraces`, creating the
objects Plotly is handed rather than copying the store's. `projectTestState`
computes them on the way out and `mergeTestState` drops them on the way back in,
so plot-ui.spec.ts's exact floats keep working unmodified - and now assert on
what is drawn rather than on a stored copy.

Measured: coordinate slider, 2 steps goes from 6 redraws / 28 renders to 2 / 16
- the row that had not moved since stage 2 - and swapping two axes from 3 / 10
to 1 / 4. The rows that open a customization panel show three more redraws and
four fewer renders, because its preview no longer stays blank until its effects
settle. e2e 20/20, benchmark 6/6, unit 55.

Assisted-by: Claude/opus-5
Stage 11 decided which synchronized grids a slider moves by comparing the
payload keys of their coordinates, on the grounds that two grids can hold
equal numbers from different nodes. That is true, and it is exactly the
case synchronization is used in: two panels plotting different nodes of
one IDS each fetched the shared time coordinate inside their own
response, so the arrays are equal under two different keys. Every one of
them silently stopped following.

The element-wise answer comes back, as `isSameAxisData` did it before.
What the keys are used for now is remembering it: a key is stable while a
cursor moves, so the walk runs once per pair of payloads and every later
tick is a Map lookup, which is what stage 11 was after. An entry cannot go
stale, because a key names the request that produced it.

Guarded by a new e2e spec on a canvas of two panels over different nodes
of one IDS - the shape the feature is used in, and one no existing spec
built, which is why twenty green specs said nothing about this.

Assisted-by: Claude/opus-5
`mergeTestState` kept `active` and its entry in `configurations` the same
object only when the spec sent `configurations` too. A spec writing `active`
alone - plot-sync.spec.ts links its panels that way - left the store's entry
stale, and writers that start from `configurations` later put it back over
`active`, silently undoing what the spec wrote. The re-link now falls back to
the store's own list.

Assisted-by: Claude/opus-5
…e awaiting

`HoverButtons`' error-band effect deep-copied the configuration its render had
closed over, awaited the band fetches, and wrote the whole copy back. Anything
that happened during the await was undone when it landed: a slider moved, a
panel linked. That is why plot-sync.spec.ts failed on this branch - the linked
panel followed the slider, then the write reset both cursors and emptied both
synchronisation lists.

It also wrote on every mount and every toggle, whether or not a band arrived,
replacing the configuration and every grid reference in it for nothing.

The fetches now write into a structural copy of the store as it is when the
effect runs - payload arrays shared, not cloned - and `mergeErrorBands` carries
only what they are about onto the store as it is after the await: each trace's
`error_bands` for the grids whose bands changed, and the band nodes checked or
unchecked in the tree. Nothing changed, nothing is written.

Benchmark: applying or restoring a data range goes from 12 redraws / 54
renders to 6 / 22, and a metadata revisit from 10 / 44 to 4 / 12 - those
scenarios mount panels, and every mount used to cost a configuration write.

Assisted-by: Claude/opus-5
… is now

Two writers of the same shape as the error-band one, masked until now because
that write happened to restore what they destroyed.

Tree updates - expanding a node, listing a data entry's IDSs, searching,
toggling error bars - awaited the backend and then wrote
`{ ...configurationBeforeAwait, customDataTree }`, dropping any plot made
meanwhile. They now go through `writeCustomDataTree`, which writes the tree
onto the latest configuration and nothing else.

`getNodesChecked` received the whole checked list as the tree saw it, and each
plot operation wrote back the configuration it had started from. Unchecking one
node while another was still loading therefore computed a list without the node
still loading, removed its grid, and the load then wrote its grid back over the
removal - every later click was one toggle out of step. A click is now reduced
to what it added and removed against the list the tree was showing, and the
operations run one at a time, each onto the configuration the previous one
left.

plot-ui.spec.ts caught the second one deterministically once the error-band
write stopped hiding it.

Assisted-by: Claude/opus-5
The request cache held response text and parsed it for every caller. Its own
header said why: `fetchDataPlot` post-processed the parsed graph in place, and
callers aliased its arrays into the store, where transforms then mutated them.
Stages 9 to 11 made every transform assign a new array under a new registry
key, so that reason is gone and the header with it.

The cache now retains the response as the fetch layer finished it. The
`plot_data` post-processing - renaming, forcing `[:]` targets, tensorising an
irregular shape, the complex transform, nulls to NaN - moves into
`normalizeDataPlotResponse` and runs once per request instead of once per
caller, so it no longer has to be idempotent. A hit costs a copy of the objects
around the arrays (`share`), never a `JSON.parse` of a multi-megabyte body.
Metadata responses are small and are handed out as deep copies. The byte
budgets still count the body length, and in-flight de-duplication and the
expected 404/464 error-band suppression are unchanged - failures are never
retained.

The arrays a hit hands out are the ones the payload registry holds, so a
payload is resident once rather than once parsed and once as text. Registration
happens on every hand-out: the registry sweeps what no grid references while
the cache may still hold it. A complex node is finished differently, so it is
retained and registered under its own `#cpx` / `value:cpx` name. The
"Data are incomplete" warning still fires on every hand-out.

Sharing is sound only while nothing writes into a payload, so under E2E_TEST
registration now deep-freezes each array and such a write fails the suites with
a TypeError. Both ran green with it on.

BASELINE.md records the stage, and prettier reformats the stage 10 and 11
tables, which `prettier --check .` in CI was rejecting.

e2e 23/23 twice on fresh app instances, benchmark 6/6, unit 80.

Assisted-by: Claude/opus-5
…forms

Six deep clones were left in plot.ts, each copying every matrix of a grid to
change fields around them: `updateInterpolatedPlots` and its caller in
`handleExistingPlot`, `updateCoordsAfterInterpolation` (to set two integers per
coordinate), the coordinate match check (to sort), `initPlotColors` (to set a
line colour) and the saved-configuration loader (twice, once only to read axis
indices). Every one of them reassigns fields and never writes into an array, so
they become `cloneGridStructure`, a shallow copy, or no copy at all.

`initPlotColors` was the one that wrote into a nested object, `plot.line`, which
a structural copy shares with its source; it builds a new line instead.

Assisted-by: Claude/opus-5
Opening visual customization or data manipulation on a 2-D node took over 13 s
on the real 720-slice entry, against 1.6 s for a 1-D panel. The panel edits a
detached copy of the grid, and that copy was a structuredClone - as were most
edits made in it: a colour, a colourscale, a line shape, a smoothing or
operation parameter, and the grid each data panel rebuilds after a fetch. Each
duplicated every matrix the grid holds. The panel also deep-copied every trace
on every render into a variable nothing read.

They are structural copies now, and the handlers that wrote into nested objects
- `line`, `customPreferences`, the geometry list - build new ones. Saving the
panel's synchronisation links replaced the store's grids instead of assigning
onto them in place.

Measured on the real entry, opening and closing a menu on the psi heatmap:
13.5 s -> 1.5 s for visual customization, 13.4 s -> 1.4 s for data
manipulation, 15.3 s -> 1.6 s opening the Heatmap section - the same as a 1-D
panel now. Counts are unchanged, so the benchmark gains two rows that guard only
that opening a panel fetches nothing; BASELINE.md records the real numbers.

It also fixes what the copy exposed: nothing pinned it, so a payload fetched
inside the panel was freed by the next sweep, and the next range applied there
cut the already windowed array with absolute bounds. The panel pins every key
its copy references, and a pin now keeps its payload's whole family alive, as a
reachable key already did.

e2e 23/23, benchmark 7/7, unit 81.

Assisted-by: Claude/opus-5
The backend's `shape` is the node's shape before downsampling; the array in
the response has `downsampled_shape`. The registry recorded the former, so
restoring a data range on a downsampled trace (e.g. a DIII-D flux loop,
327680 samples sent as 1000) widened the window to the full node and tfjs
threw "begin[1] + size[1] (327680) would overflow input.shape[1] (1000)".

Assisted-by: Claude/opus-5
Introduces some reactivity tests (this will be useful later for more advanced improvement on the separation of data and configuration in the configuration store from the frontend)
- try to reduce the number of redraw and backend requests by removing the un-necessary ones
- fix few issues with error bands:
   - wrong axis scale when displaying two quantities with error bands on y1 and y2
   - missing error bands in the case of symmetrical error (error_upper only)
…ated grid

Removing a trace from a grid that combines several URIs refetches the others,
each interpolated over the rest. The loop called `applyRangesToGrid` on every
pass, which returns a structural copy, so from the second trace on it wrote
into objects no longer in the grid: those traces kept the values interpolated
over the removed URI and were drawn against the new common time base (e.g. URI-1's
ip looking rescaled after unchecking URI-2's). Window once, after the loop.

Adding a trace dropped the windowed copy the same way, so a new trace was not
cut to the grid's data ranges. Put that copy back in the configuration.

Assisted-by: Claude/opus-5
…e_object

Split configuration/data store object
… URIs

The tree browser undid what the user did. Opening a URI reloaded its IDS
list with empty children, so its open folders showed an open icon over
nothing; expansion lived in each `TreeLibrary`, and was lost with the side
panel or the configuration; a folder expanded before its children loaded,
and stayed open when that failed; and editing a plot expanded its signals'
folders in whichever tree ran first, for the first URI only.

- Expanded nodes and open URIs are session state in the ui slice, per
  configuration. Several URIs can be open at once, and only the user
  collapses anything.
- A folder opens once its children have loaded (a spinner meanwhile), and
  only if it has some; a second click closes it.
- Editing a plot loads and opens the folders of every signal it uses, in
  each of its URIs.
- A "Search all URIs" switch searches every loaded URI in parallel. Results
  show in their own fully expanded view, built without touching the browsed
  tree; clearing the search brings the browsed trees back and reveals the
  nodes checked among the results. Result nodes keep `[:]`, like the browsed
  tree, so a leaf is the same node in both.

Plotting wrote back the configuration it started from, tree included, so
folders loaded while a plot was computed disappeared; it no longer writes
the tree. The tree now fills the height left under the search controls
instead of the panel's minus a fixed guess, which the new switch made
overflow the panel.

Assisted-by: Claude/opus-5
Comparing a signal across data entries meant finding and checking it in each
one. Ctrl+click on a leaf (⌘+click on macOS, where Ctrl+click is a right
click) now checks it in every data entry that has it, loading and opening the
folders down to it on the way; on a checked leaf it unchecks it, error bands
included, everywhere. Entries without the node are left alone. It works in
the browsed trees and in search results.

A modifier, not Alt (window dragging on many Linux desktops) nor Shift
(range selection), and a hint under the search switches tells it once the
configuration has two entries.

The tree spec's in-panel check measured the scroll areas nested in the tree
too, which are taller than the panel by design; only the outermost counts.

Assisted-by: Claude/opus-5
…e as

"How to launch IBEX?" gets a "Command-line options" section: every
option, the rules between them, path resolution, and examples through
launch.sh. The feature pages point to it where an option does the same
as the UI (add configuration with a template, add URIs, load, default
configuration, template folders).

A new "Rename configuration" page describes the pencil button, and
"Save configuration" now explains Save versus "Save as…".

Assisted-by: Claude/opus-5
The header now has the rename pencil and separate "Save" and "Save as…"
buttons. Retaken from the app itself: the first window, empty
configuration, configuration creation (with and without the template
list), URI selection, tree, canvas, metadata, default configuration and
delete confirmation.

The screenshots showing a native file dialog or the native menu
(save_configuration, load_configuration, template_preferences) are kept:
they cannot be captured from the app's page.

Assisted-by: Claude/opus-5
Users start IBEX from the directory holding their configurations and data
entries, but the launch scripts cd elsewhere, so file dialogs never opened
there. The directory is kept in IBEX_CALLER_CWD.

The native open dialog cannot start in that directory on RHEL 9: Chromium
(Electron 43) always shows file dialogs through the XDG desktop portal on
Linux, with no GTK fallback left, and the portal there (xdg-desktop-portal
1.12, FileChooser interface version 3) ignores the start folder of OpenFile;
current_folder is only honoured from version 4 (xdg-desktop-portal 1.18). The
dialog opens on "Recent" whatever defaultPath is, and neither
--xdg-portal-required-version nor --gtk-version changes that. SaveFile does
honour it.

Hence the shortcut is in-app:
- the header Load button becomes a menu: "Browse…" (native dialog) plus the
  .json files of the launch directory, each loaded with one click;
- Select URIs gets a folder button listing the launch directory's data
  entries: netCDF files, and HDF5 entries in it or one level down;
- both read the directory through a new listLaunchDirectory IPC, which only
  ever lists the launch directory (src/main/launchDirectory.ts, unit-tested).

The native dialogs are still given the launch directory: Save as / first Save
start there for a configuration without a file (an existing file's folder
still wins), and the open dialog does too on desktops with a portal >= 4.

Assisted-by: Claude/opus-5
User manual: the Load menu and the Select URIs folder button list the
directory IBEX was started from; Save as starts there for a configuration
not saved yet. Repo guidance: why the open dialog cannot start there on
RHEL 9 (XDG portal < 4) and where the in-app listing lives.

Assisted-by: Claude/opus-5
…g traces

A grid draws its traces against the union of the coordinates of its
sources, but a source was keyed on the data entry alone. Two occurrences
of one IDS in one entry (e.g. summary:0 sampled fast and summary:1 with a
few points) were then never interpolated over each other: one trace's
values were drawn against the other's time vector. Across entries, the
URI interpolated over also kept the trace's own occurrence instead of the
other source's.

A source is now a data entry and an IDS occurrence (an occurrence left
out is 0, as the back-end reads it). Error bands share the same helper
and are fixed with it. The back-end already supported interpolating over
another occurrence of the same entry; tests now cover it.

Assisted-by: Claude/opus-5
A source was a data entry and an occurrence, whatever its IDS. With
summary:1 and core_profiles:0 in one entry, a core_profiles trace added
to a grid holding the summary one was interpolated over core_profiles:1,
and the summary trace over summary:0, neither of which exists.

A source is now a data entry, an IDS and an occurrence. A trace is
interpolated over the other occurrences of its own IDS; another IDS of
its own entry is left out as before, and another IDS of an entry where
the trace's IDS is not drawn keeps the trace's own occurrence.

Assisted-by: Claude/opus-5
…ation menus

The trace tabs of the visual customization and data manipulation menus
tell traces sharing a name apart by the end of their node paths, and fell
back to the full node URI when no path told them apart: one node of an
entry in summary:0 and summary:1 showed the whole URI next to short
labels. Such traces now get their name and their IDS occurrence,
`ip_URI-1#summary:0` and `ip_URI-1#summary:1`.

plotTabLabels moves to plotSources.ts, next to the parsing of a node URI
it now uses, and gets unit tests.

Assisted-by: Claude/opus-5
Unknown long options were all passed through as Chromium/Electron
switches, so --conf, --uris or --tempalte silently opened the app
without doing what was asked. Options that look like a mistyped IBEX
option are now rejected with a "did you mean" hint; other switches
still reach Electron.

Assisted-by: Claude/opus-5
CLAUDE.md now only imports AGENTS.md, and the Copilot copy is removed,
so the guidance lives in a single file. Also notes that only
near-misses of IBEX options are rejected on the command line.

Assisted-by: Claude/opus-5
Fix/occurrence handling regression
A node was only interpolated over nodes of the same IDS and path, but
only the coordinates of the target are used, and their number and names
are already checked. Each IDS has its own time vector, even in one data
entry: core_profiles/global_quantities/ip, summary/global_quantities/ip/
value and equilibrium/time_slice[:]/global_quantities/ip of ITER
130013/10 have 108 points each, 189 together. Any target node is now
accepted, so they can be put on common coordinates.

A node in an AoS of an IDS with heterogeneous time (time_slice[:]/time)
could not be plotted at all: its per-slice times were not converted to
numbers before taking their union.

Assisted-by: Claude/opus-5
core_profiles, summary and equilibrium ip could each be plotted but not
on the same plot. The grid took the IDSs of one entry to share their
coordinates, yet each IDS has its own time vector: no common coordinates
were asked for, and the x axis check refused the time path of another
IDS (#summary/time against #core_profiles/time).

A trace is now interpolated over a node drawn from each source of
another IDS, and the x axis check compares the time path only when
there is nothing to interpolate over. This needs the back-end to accept
a target node of another IDS.

Assisted-by: Claude/opus-5
…ible_axis

Fix/interpolation with compatible axis

@SimonPinches SimonPinches left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add a "Reload data" action since otherwise the caching prevents viewing updates of datasets that are still being written, e.g. from JINTRAC. (They will always appear as when first opened).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file appears as a binary file because of line 59 (const PAIR = '<NUL>';). If this was changed to const PAIR = '\u0000'; then it could be reviewed as a normal file without changing the behaviour.

Comment thread docs/source/users_manual/features/add_uri.rst

@prasad-sawantdesai prasad-sawantdesai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Help -> "Learn more" goes to https://www.electronjs.org/

@prasad-sawantdesai

Copy link
Copy Markdown
Contributor

Where do we use these commands?
image

@prasad-sawantdesai

Copy link
Copy Markdown
Contributor

Should it show internal server error or meaningful message The requested array has an inhomogeneous shape after 2 dimensions?
On user interface you only see 500 Internal Server Error

17:01:01 ERROR    Traceback (most recent call last):                                                                                                                                                               
  File "/home/ITER/sawantp1/github/IBEX/ibex_venv/lib/python3.13/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app                                                                           
    await app(scope, receive, sender)                                                                                                                                                                              
  File "/home/ITER/sawantp1/github/IBEX/ibex_venv/lib/python3.13/site-packages/starlette/routing.py", line 73, in app                                                                                              
    response = await f(request)                                                                                                                                                                                    
               ^^^^^^^^^^^^^^^^                                                                                                                                                                                    
  File "/home/ITER/sawantp1/github/IBEX/ibex_venv/lib/python3.13/site-packages/fastapi/routing.py", line 301, in app                                                                                               
    raw_response = await run_endpoint_function(                                                                                                                                                                    
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                    
    ...<3 lines>...                                                                                                                                                                                                
    )                                                                                                                                                                                                              
    ^                                                                                                                                                                                                              
  File "/home/ITER/sawantp1/github/IBEX/ibex_venv/lib/python3.13/site-packages/fastapi/routing.py", line 212, in run_endpoint_function                                                                             
    return await dependant.call(**values)                                                                                                                                                                          
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                          
  File "/home/ITER/sawantp1/github/IBEX/backend/ibex/core/ibex_service.py", line 20, in wrapper                                                                                                                    
    response = func(*args, **kwargs)                                                                                                                                                                               
  File "/home/ITER/sawantp1/github/IBEX/backend/ibex/endpoints/data.py", line 120, in plot_data                                                                                                                    
    return CustomORJSONResponse(ibex_service.get_plot_data(plot_data_query))                                                                                                                                       
                                ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^                                                                                                                                        
  File "/home/ITER/sawantp1/github/IBEX/backend/ibex/core/ibex_service.py", line 121, in get_plot_data                                                                                                             
    return data_source.get_plot_data(plot_data_query)                                                                                                                                                              
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^                                                                                                                                                              
  File "/home/ITER/sawantp1/github/IBEX/backend/ibex/data_source/imas_python_source.py", line 1207, in get_plot_data                                                                                               
    expected_flattened_shape = np.array(c["value"]).shape[-1]                                                                                                                                                      
                               ~~~~~~~~^^^^^^^^^^^^                                                                                                                                                                
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 2 dimensions. The detected shape was (11, 121) + inhomogeneous part.                                    
 @exception_handlers.py:16                                                                                                                                                                                         
INFO:     127.0.0.1:59552 - "GET /data/plot_data?uri=imas%3A%2F%2Fuda.iter.org%2Fuda%3Fpath%3D%2Fwork%2Fimas%2Fshared%2Fsimdb%2F66b45fc3b83411f1ae09d4f5ef75ec04%2Fimasdb%2Fiter%2F3%2F53298%2F2%26backend%3Dhdf5%23edge_sources%3A0%2Fsource%5B%3A%5D%2Fggd%5B%3A%5D%2Ftotal_ion_energy%5B%3A%5D%2Fgrid_index&interpolate_over=imas%3A%2F%2Fuda.iter.org%2Fuda%3Fpath%3D%2Fwork%2Fimas%2Fshared%2Fsimdb%2F66b45fc3b83411f1ae09d4f5ef75ec04%2Fimasdb%2Fiter%2F3%2F53298%2F2%26backend%3Dhdf5%23summary%3A0%2Fglobal_quantities%2Fip%2Fvalue&interpolation_method=exact_value HTTP/1.1" 500 Internal Server Error

@prasad-sawantdesai prasad-sawantdesai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried plotting edge_profiles/ggd/electrons/temperature/values. It ended up with following error..

Image
  File "/home/ITER/sawantp1/github/IBEX/ibex_venv/lib/python3.13/site-packages/fastapi/routing.py", line 301, in app                                                                                               
    raw_response = await run_endpoint_function(                                                                                                                                                                    
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                    
    ...<3 lines>...                                                                                                                                                                                                
    )                                                                                                                                                                                                              
    ^                                                                                                                                                                                                              
  File "/home/ITER/sawantp1/github/IBEX/ibex_venv/lib/python3.13/site-packages/fastapi/routing.py", line 212, in run_endpoint_function                                                                             
    return await dependant.call(**values)                                                                                                                                                                          
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                          
  File "/home/ITER/sawantp1/github/IBEX/backend/ibex/core/ibex_service.py", line 20, in wrapper                                                                                                                    
    response = func(*args, **kwargs)                                                                                                                                                                               
  File "/home/ITER/sawantp1/github/IBEX/backend/ibex/endpoints/data.py", line 120, in plot_data                                                                                                                    
    return CustomORJSONResponse(ibex_service.get_plot_data(plot_data_query))                                                                                                                                       
                                ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^                                                                                                                                        
  File "/home/ITER/sawantp1/github/IBEX/backend/ibex/core/ibex_service.py", line 121, in get_plot_data                                                                                                             
    return data_source.get_plot_data(plot_data_query)                                                                                                                                                              
           ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^                                                                                                                                                              
  File "/home/ITER/sawantp1/github/IBEX/backend/ibex/data_source/imas_python_source.py", line 1433, in get_plot_data                                                                                               
    coordinates_to_be_returned[0]["value"], data_to_be_returned = downsample_data(                                                                                                                                 
                                                                  ~~~~~~~~~~~~~~~^                                                                                                                                 
        data_to_be_returned,                                                                                                                                                                                       
        ^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                       
    ...<3 lines>...                                                                                                                                                                                                
        single_x_axis=(coordinates_to_be_returned[0]["path"] == f"#{ids}/time"),                                                                                                                                   
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                   
    )                                                                                                                                                                                                              
    ^                                                                                                                                                                                                              
  File "/home/ITER/sawantp1/github/IBEX/backend/ibex/core/utils.py", line 193, in downsample_data                                                                                                                  
    _x1, _data1 = downsample_data(_data, target_size, method, _x)                                                                                                                                                  
                  ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                  
  File "/home/ITER/sawantp1/github/IBEX/backend/ibex/core/utils.py", line 184, in downsample_data                                                                                                                  
    _x1, _data1 = downsample_data(_data, target_size, method, x=x)                                                                                                                                                 
                  ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                 
  File "/home/ITER/sawantp1/github/IBEX/backend/ibex/core/utils.py", line 184, in downsample_data                                                                                                                  
    _x1, _data1 = downsample_data(_data, target_size, method, x=x)                                                                                                                                                 
                  ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                 
  File "/home/ITER/sawantp1/github/IBEX/backend/ibex/core/utils.py", line 213, in downsample_data                                                                                                                  
    return x, data                                                                                                                                                                                                 
           ~^^^^^^                                                                                                                                                                                                 
TypeError: only integer scalar arrays can be converted to a scalar index                                                                                                                                           
 @exception_handlers.py:16 

@prasad-sawantdesai prasad-sawantdesai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the Error label I see Data is incomplete. But I could see plot is correct. What does it mean by Data is incomplete.

Image

@prasad-sawantdesai prasad-sawantdesai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While plotting sometimes application screen goes white.

Image

Plotting equilibrium.profiles_2d.psi from two different URIs

==========> Endpoint 'plot_data' executed in 16.9670 seconds
INFO:     127.0.0.1:40584 - "GET /data/plot_data?uri=imas%3Ahdf5%3Fpath%3D%2Fwork%2Fprojects%2Fbridge%2Finputs%2Fscenario%23equilibrium%3A0%2Ftime_slice%5B%3A%5D%2Fprofiles_2d%5B%3A%5D%2Fpsi&downsampling_method=M4&downsampled_size=1000&interpolate_over=imas%3Ahdf5%3Fpath%3D%2Fwork%2Fimas%2Fshared%2Fsimdb%2Fc5bd554fd82d11ef9d5e9440c9e7706c%2Fimasdb%2Fiter%2F3%2F53298%2F2%23equilibrium%3A0%2Ftime_slice%5B%3A%5D%2Fprofiles_2d%5B%3A%5D%2Fpsi&interpolation_method=exact_value HTTP/1.1" 200 OK
==========> Endpoint 'version' executed in 0.0000 seconds
INFO:     127.0.0.1:40598 - "GET /info/version HTTP/1.1" 200 OK

@prasad-sawantdesai prasad-sawantdesai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just have Metadata instead of Metadatas

Image

@prasad-sawantdesai

Copy link
Copy Markdown
Contributor

Added summary:0/global_quantities/ip/value and updated with multiply operation

image

Then added equilibrium:0/time_slice[0]/global_quantities/ip on the same plot

image

summary:0/global_quantities/ip/value resets to original values..
image

@prasad-sawantdesai prasad-sawantdesai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested with launch.sh. Most of the use cases works. minor comments added..

@olivhoenen

Copy link
Copy Markdown
Contributor Author

Where do we use these commands? menus?

--> #157

This branch was successfully deployed

1 active deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants