Repository navigation
Dev tools - #130
Closed
vladimir-tsymbal wants to merge 25 commits into
Closed
Dev tools#130vladimir-tsymbal wants to merge 25 commits into
vladimir-tsymbal wants to merge 25 commits into
Conversation
This fork continues the legacy PTI-GPU tools and samples only. - Remove sdk/ and tools/unitrace/. - Move .clang-format and .clang-tidy from sdk/ to the repo root. - gtpin_utils: take pti_assert.h from utils/ instead of sdk/src/utils, the same header the GTPin tools already use. - third-party-programs.txt: keep only the CMake notice (still needed for tools/sysmon/cmake/FindLevelZero.cmake). - Drop unitrace-specific .gitignore entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace the sdk/unitrace top-level build with an aggregate build of the
standalone tools and samples. Each one is an ExternalProject in
build/{tools,samples}/<name>, so the per-folder builds used by
tests/run.py keep working unchanged.
- Options: PTI_GPU_BUILD_TOOLS, PTI_GPU_BUILD_SAMPLES, PTI_GPU_TOOLS,
PTI_GPU_SAMPLES, PTI_GPU_SKIP.
- icpx samples are skipped when icpx is not found, and the debug_info
samples when libiga64 is not found.
- Tools install through per-tool staging prefixes into
CMAKE_INSTALL_PREFIX, one CPack component (TGZ/DEB/RPM) per tool.
- BUILD_TESTING adds a "legacy" ctest that runs tests/run.py.
- Linux only. Homepage points at the fork.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- build_utils.clone(): serialize clones of the same repository with a file lock. get_iga_headers.py and get_igc_headers.py both clone intel-graphics-compiler into the same path; under a parallel (Ninja) build one of them copied from a half-finished clone. - ze_metric_info/query/streamer: call GetMDHeaders(). These samples include utils/metric_utils.h, which needs metrics_discovery_api.h, but never fetched it, so they failed unless the MD dev package was installed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Remove the PTI SDK, lint, docs and nightly workflows and the detectChanges and linuxTest actions they used. - ci.yml: build the aggregate project on ubuntu-24.04 without a GPU or oneAPI (GTPin tools skipped). Drop the Windows job. - pti-tools_build_and_test.yml: keep only the self-hosted PVC job that runs tests/run.py for sysmon, cl_gpu_metrics and cl_debug_info. Drop the unitrace steps and the Windows job. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Windows is not tested in this fork, but its build paths stay. - Warn instead of failing when configuring on a non-Linux system. - Run the icpx and libiga64 auto-skip probes on Linux only; on Windows the samples use other compilers and build_utils finds IGA via its DLL. - Use the ZIP CPack generator on Windows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- README: replace the deprecation notice with an "About this fork" section, list all tools, document the aggregate top-level build and its options, and drop the unitrace/PTI SDK sections. Windows build paths are kept but marked untested. - onetrace, oneprof, ze_tracer, cl_tracer: drop the "deprecated, use unitrace" banners. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- run.py -c no longer deletes every *.log under the repository root. It removes only the per-component build dirs, the Python caches and the stderr.log written by run.py itself. - New --log-dir <path> option (or PTI_TEST_LOG_DIR) for stderr.log. The default is unchanged (current directory). - New --timeout <seconds> option (default 600, 0 disables): a per-test deadline covering configure, build and run. On expiry the process group is killed and the test is reported as "FAILED (timeout)". - -s and -d behave as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Raise SetRequiredCMakeVersion from 3.12 to 3.22, as the README already states, and update the per-component READMEs to match. On Linux, FindIGALibrary, CheckForMDLibrary and CheckForMetricsLibrary now also accept the versioned runtime sonames (libiga64.so.2/.so.1, libigdmd.so.1, libigdml.so.1), so a dev symlink package is no longer required. The metrics library is now found on Ubuntu 24.04, which removes its "not found" configure warning. The WIN32 branches are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FetchContent's download could hang forever when downloadmirror.intel.com was unreachable (e.g. a no_proxy entry bypassing the required proxy): INACTIVITY_TIMEOUT does not cover a stalled TCP connect, and ExternalProject retries a timed-out download 5 times. GetGTPinPackage now downloads the archive once with file(DOWNLOAD), using GTPIN_DOWNLOAD_TIMEOUT (default 900 s) and GTPIN_DOWNLOAD_INACTIVITY_TIMEOUT (default 60 s), and gives the local file to FetchContent. On failure, configure stops with an error that points to the GTPin 4.6.4 download page and to -DGTPIN_PATH / -DGTPIN_LINK. GTPIN_LINK may also be a local archive path. The GTPin version (4.6.4) is unchanged. The instcount/memaccess READMEs document this and correct the Linux -DGTPIN_PATH value: the Linux package has no Profilers/ subdirectory. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
info::device::preferred_interop_user_sync throws on non-OpenCL backends, which made "dpc_info -a" abort on Level Zero devices. Query it only for OpenCL devices and print N/A otherwise. The line also printed is_compiler_avilable instead of the queried value. The profile, printf_buffer_size, preferred_interop_user_sync and extensions descriptors are deprecated in SYCL 2020 and have no drop-in replacement. Keep printing them, but suppress -Wdeprecated-declarations around those queries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
icpx ignores -gline-tables-only for the spir64 device target (with a warning), so the device binary had no line information, and "ze_debug_info dpc" failed with "Failed to get source mapping for kernel ID: 1". -g emits device debug info, and the test passes again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The SYCL host device no longer exists. The "host" option had already fallen back to default_selector_v, which picks the GPU, so it tested nothing new. Remove it from the sample, its README and the test lists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add utils::ze::InitGpuDrivers(), following the pattern in tools/sysmon/main.cc. It calls zeInitDrivers() for GPU drivers and falls back to zeInit(ZE_INIT_FLAG_GPU_ONLY) when no driver is found or the call fails, so oneprof still sees ZE_RESULT_ERROR_DEPENDENCY_UNAVAILABLE. zeDriverGet() keeps working after zeInitDrivers() (checked with loader 1.24). zeInitDrivers is declared weak, so binaries built with L0 >= 1.10 headers still load with an older loader that lacks the symbol; this was checked against a stub loader. With older headers, and on Windows (untested there), only zeInit() is used. Use it in the ze_* samples, ze_tracer, onetrace and oneprof. sysmon keeps its own copy, and the legacy ze_sysman path is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ze_sysman now calls zesInit() and enumerates devices through zesDriverGet/zesDeviceGet. It falls back to the legacy ZES_ENABLE_SYSMAN=1 + zeInit() mode on stacks without zesInit. On PVC the new path reports both tiles and 2 x 64 GiB of memory, where the legacy handle reported 0 subdevices and a single 0 MB module. oneprof only used sysman to read the device PCI address for matching Level Zero and OpenCL devices. It now uses the core zeDevicePciGetPropertiesExt(), so it no longer enables sysman inside the profiled application. zesDevicePciGetProperties stays as a fallback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The OpenMP offload runtime submits through synchronous immediate command lists and never synchronizes, resets or destroys the kernel events. The collector only processes kernels on those calls, so it reported nothing for "ze_hot_kernels omp". DisableTracing() now processes the completed kernels that are still pending. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The library is built with -fvisibility=hidden, so the OpenMP runtime
could not find ompt_start_tool ("No OMP tool loaded") and the sample
printed nothing. Export it with C linkage.
The test compared stderr instead of the option with "gpu"/"cpu", so its
Target/Parallel region checks never ran. They now run, and they pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test built an environment with the OpenCL offload plugin selected, then passed environ=None. omp_gemm ran on Level Zero, so no OpenCL calls were recorded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add/Sub/Mul with an immediate asserted that the value lies in [-mask, mask], where mask is all ones for the destination size. For an 8-byte destination, static_cast<int64_t>(mask) is -1, so the check rejected every value, and memaccess aborted in StrideDistrCalc (64-bit address subtraction) on PVC. Any 64-bit immediate fits an 8-byte destination, so skip the range check there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A test, or a sample it builds on, can now raise utils.TestSkipped. Skip conditions: - Metrics tests (cl_gpu_metrics, cl_gpu_query, gpu_perfmon_set, ze_metric_*, gpuinfo, and every oneprof option except -i) need /proc/sys/dev/i915/perf_stream_paranoid (or xe observation_paranoid) set to 0, or root, plus the Metrics Discovery library. - dpc_gemm, omp_gemm, dpc_info and omp_hot_regions need icpx. The skip propagates to every test that builds these samples. - instcount and memaccess are skipped when GTPin cannot be downloaded or GTPIN_PATH is incomplete. At the end, run.py prints the unchanged "PASSED: x / FAILED: y" line, then a SKIPPED count and a list of failed and skipped tests with reasons. It also writes results.json and junit.xml next to stderr.log; -c removes them. Skips do not change the exit code. test() still returns a bool; run_test() returns the full result. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the system has no Level Zero headers, FindL0Headers downloads them with get_ze_headers.py (pinned at v1.17.19). That path was broken: - It copied zer_api.h, which no release before v1.25 has, and nothing in the tree includes it. It is no longer copied or listed as an output. - ze_gen_headers (the tracing callback generator used by ze_tracer and onetrace) did not depend on the download, so parallel builds read level_zero/ze_api.h before it existed. Tested by forcing the fallback (-DLO_CORE_INC_FOUND=OFF): ze_gemm, ze_tracer and onetrace build without warnings and trace correctly. The other 7 pinned commits in build_utils/get_*.py are reachable and need no bump. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Windows: the lock added for parallel builds used fcntl, which does not exist on Windows, so every header fetch failed there. Fall back to msvcrt.locking, retrying while the lock is held (LK_LOCK gives up after about 10 s). Only tested in a simulation. - clone() only checked that the clone directory existed. A clone killed by a test timeout (a stalled compute-runtime clone in a clean run) was reused, and all 15 cl_tracer tests failed. It now fetches the pinned commit shallowly (falling back to a full clone) into <dir>.partial, checks git's exit codes, writes a .pti-clone-commit stamp and renames into place. Clones without the stamp are fetched again, and git aborts transfers below 1 KB/s for 60 s. A clean aggregate build goes from 48 s to 24 s, mostly from the shallow compute-runtime fetch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The installed tools could not find libgtpin.so/libged.so: they were not installed, and only the build tree's RPATH pointed at them. InstallGTPinRuntime() installs the GTPin libraries (gtpin, gtpin_core, ged, iga_wrapper) into a private lib/<tool> directory and sets INSTALL_RPATH to $ORIGIN/../lib/<tool>. The GTPin libraries already carry RPATH $ORIGIN. Each tool has its own copy, so the separate DEB/RPM components do not share files. The Intel Simplified Software License allows redistributing the unmodified package with its terms, so the package's Licensing/ files (license, third-party notices, IGC notices) are installed to share/doc/<tool>/gtpin. Windows installs are unchanged. Verified: cmake --install, then ldd shows no missing libraries, and the installed tools run on ze/cl/dpc gemm from outside the tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
All pins move to the latest release tags. Each now has a comment naming its release: - metrics-discovery: 4ff8df0 (2024-12, API 1.13) -> 1.14.182 - gmmlib: 9290546 (2022-01) -> intel-gmmlib-22.10.2 - intel-graphics-compiler (IGA, IGC): d5bef0c (2022-02) -> v2.43.9 - ittapi: 83d6ca4 (2022-02) -> v3.28.4 - compute-runtime: 26.31.39395.14 -> 26.44.40734.8 - OpenCL-Headers: dcd5bed (2022-02) -> v2026.05.29 - level-zero: v1.17.19 -> v1.29.0 The MD, OpenCL and Level Zero headers are fetched only when the system has none (e.g. CI runners without the dev packages), so those builds now get current headers. The IGA header gains the Xe2/Xe3 platform definitions (GfxCoreToIgaGen() still needs extending for them). Tested on PVC (L0 1.24, compute-runtime 25.18, MDAPI 1.14.180): the clean aggregate build is 30/30 with no warnings, all 90 non-OA tests pass, the forced MD/OpenCL/L0 fallbacks build without warnings, and the ze/cl_debug_info disassembly is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both steps were disabled pending Phase 1 fixes, which have landed: - GTPin tools work on current runtimes (gtpin_utils immediate range fix, installed runtime libraries). - FindIGALibrary accepts libiga64.so.2 without a dev symlink. They now use the same PVC condition as the other steps. Locally (PVC, i915), the four workflow commands pass: sysmon 4/4, instcount 3/3, ze_debug_info 2/2, cl_debug_info 1/1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
run.py --skip-oa (or PTI_TEST_SKIP_OA=1) makes every test that needs GPU performance (OA) metric streams report SKIPPED with a reason: cl_gpu_metrics, cl_gpu_query, gpu_perfmon_set, ze_metric_*, gpuinfo and oneprof except -i. Useful on hosts where opening OA streams is unstable. The default is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add a comprehensive description of proposed changes
Fixes # - issue number(s) if exists, JIRA or Git issue
Area of the change
Type(s) of change
Choose one or multiple, leave empty if none of the other choices apply
Review
Tests
Specific HW and OS where to run the test unless generic:
For example, 2 discrete GPUs, integrated GPU, specific GPU model, PyTorch integration test(s)
Checklist
Details on API(s) or command line option(s) changes
If applies - details on the broken backward compatibility
Indicate what API(s) backward compatibility or option(s) is broken, why it might be OK, or suggest on how to deal with it moving forward
Notify the following users
List users with
@to send notificationsOther information