Skip to content

Commit fdd33c5

Browse files
intuibaseintuibase
authored
Development guide (#18)
1 parent c203bb4 commit fdd33c5

File tree

3 files changed

+308
-2
lines changed

3 files changed

+308
-2
lines changed

‎DEVELOPMENT.md‎

Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
# Local development for OpenTelemetry PHP Distro Local development
2+
## Build and package
3+
4+
The best method for building is to use a set of Bash scripts that we utilize in production workflows for building releases.
5+
6+
7+
8+
All scripts are located in the `tools/build` folder, but they should be called from the root folder of the repository. To ensure everything works correctly on your system, you need to have Docker installed.
9+
Each of the scripts mentioned below has a help page; to display it, simply provide the `--help` argument.
10+
11+
### Building the native library like on CI
12+
13+
```bash
14+
cd opentelemetry-php-distro
15+
./tools/build/build_native.sh --build_architecture linux-x86-64 --interactive --ncpu 2
16+
```
17+
18+
This script will configure the project and build the libraries for the linux-x86-64 architecture. Adding the interactive argument allows you to interrupt the build using the `Ctrl + C` combination, and with the ncpu option, you can build in parallel using the specified number of processor threads.
19+
If you are not adding new files to the project and just want to rebuild your changes,
20+
you can provide the `--skip_configure` argument - this will save time on reconfiguring the project.
21+
You can also save a lot of time by creating a local cache for Conan packages;
22+
the files will then be stored outside the container and reused repeatedly.
23+
To do this, provide a path to the `--conan_cache_path` argument, e.g., `~/.conan_cache`.
24+
The script will automatically execute native unit tests just after the build.
25+
If you would like to skip native unit tests you can use `--skip_unit_tests` command line option.
26+
27+
Currently, we support the following architectures:
28+
29+
```bash
30+
linux-x86-64
31+
linuxmusl-x86-64
32+
linux-arm64
33+
linuxmusl-arm64
34+
```
35+
36+
If you want to enable debug logging in tested classes, you need to export environment variable `OTEL_PHP_DEBUG_LOG_TESTS=1` before run.
37+
38+
### Building the native library for other platforms
39+
40+
You can always try to compile the native part for an unsupported architecture or platform. To facilitate this, we have made it possible to remove hard dependencies on Docker images, the compiler, and build profiles.
41+
42+
To make everything work on your system, you will need the gcc compiler (at the time of writing, version 15.2.0+), cmake (v4.2.1+), and python 3.14+.
43+
44+
Since our system uses Conan as the repository for required dependencies, you need to install them first. The following script will install everything necessary in the `~/.conan2` folder. If you haven't used Conan before, provide the argument `--detect_conan_profile` to create a default profile – if you have used Conan before, you can skip this. If you are not using python-venv and have Conan installed directly on your system, you can pass the argument `--skip_venv_conan`, which will cause the script to skip creating a venv and installing Conan.
45+
46+
```bash
47+
./prod/native/building/install_dependencies.sh --build_output_path ./prod/native/_build/custom-release --build_type Release --detect_conan_profile
48+
```
49+
50+
The script will install dependencies and generate the files necessary to configure the project in the next step (prod/native/_build/custom-release):
51+
52+
```bash
53+
cmake -S ./prod/native/ -B ./prod/native/_build/custom-release/ -DCMAKE_PREFIX_PATH=./prod/native/_build/custom-release/build/Release/generators/ -DSKIP_CONAN_INSTALL=1 -DCMAKE_BUILD_TYPE=Release
54+
```
55+
56+
Building:
57+
```bash
58+
cmake --build ./prod/native/_build/custom-release/
59+
```
60+
61+
If the build is successful, you can find the built libraries using the following command:
62+
```bash
63+
find prod/native/_build/custom-release -name opentelemetry*.so
64+
```
65+
66+
As a result you should see:
67+
```bash
68+
prod/native/_build/custom-release/loader/code/opentelemetry_php_distro_loader.so
69+
prod/native/_build/custom-release/extension/code/opentelemetry_php_distro_84.so
70+
prod/native/_build/custom-release/extension/code/opentelemetry_php_distro_83.so
71+
prod/native/_build/custom-release/extension/code/opentelemetry_php_distro_82.so
72+
prod/native/_build/custom-release/extension/code/opentelemetry_php_distro_81.so
73+
```
74+
75+
76+
77+
### Testing the native library
78+
79+
The following script will run the phpt tests for the native library, which should be built in the previous step - make sure to use the same architecture. You can run tests for multiple PHP versions simultaneously by providing several versions separated by a space to the `--php_versions` parameter.
80+
81+
```bash
82+
cd opentelemetry-php-distro
83+
./tools/build/test_phpt.sh --build_architecture linux-x86-64 --php_versions '81 82 83 84'
84+
```
85+
86+
### Building PHP dependencies
87+
88+
To ensure the instrumentation is fully successful, it is required to download and install dependencies for the PHP implementation. You can do this automatically using a script that will download and install them separately for each specified PHP version. Similar to the previous step, you need to provide the PHP versions separated by spaces as a parameter to the `--php_versions` argument.
89+
90+
```bash
91+
cd opentelemetry-php-distro
92+
./tools/build/build_php_deps.sh --php_versions '81 82 83 84'
93+
```
94+
95+
### Building Packages
96+
97+
We currently support building packages for Debian-based systems (deb), Red Hat-based systems (rpm), and Alpine Package Keeper (apk) for each supported CPU architecture.
98+
99+
To build a package, use the `./tools/build/build_packages.sh` script with the following arguments:
100+
```
101+
--package_version Required. Version of the package.
102+
--build_architecture Required. Architecture of the native build. (eg. linux-x86-64)
103+
--package_goarchitecture Required. Architecture of the package in Golang convention. (eg. amd64)
104+
--package_sha Optional. SHA of the package. Default is fetch from git commit hash or unknown if got doesn't exists.
105+
--package_types Required. List of package types separated by spaces (e.g., 'deb rpm').
106+
```
107+
108+
For the `--package_goarchitecture` parameter, we currently distinguish between two architectures: amd64 and arm64. These should correspond to the value of the `--build_architecture` argument.
109+
110+
Remember, it's best if the package version reflects the version recorded in the `project.properties` file.
111+
112+
```bash
113+
cd opentelemetry-php-distro
114+
./tools/build/build_packages.sh --package_version v1.0.0-dev --build_architecture linux-x86-64 --package_goarchitecture amd64 --package_types 'deb rpm'
115+
```
116+
117+
# Updating docker images used for building and testing
118+
## Building and updating docker images used to build the agent extension
119+
120+
For detailed information about Docker images architecture, versioning, and build instructions, please refer to [prod/native/building/dockerized/README.md](prod/native/building/dockerized/README.md).
121+
122+
All image versions are parameterized in [docker-compose.yml](prod/native/building/dockerized/docker-compose.yml). During CI builds, image versions are automatically read from this file - if an image doesn't exist in DockerHub, it will be automatically built.
123+
124+
Be aware that if you want to build images for ARM64 you must run it on ARM64 hardware or inside emulator. The same applies to x86-64.
125+
126+
To test freshly built images, you need to udate image version in ```./tools/build/build_native.sh``` script and run build task described in [Build/package](#build-and-package)
127+
128+
## Adding or removing support for PHP release
129+
130+
- Add the new version to the `supported_php_versions` list in the [project.properties](project.properties) file.
131+
- Update supported PHP version detection in function `is_php_supported` in [post-install.sh](packaging/scripts/post-install.sh)
132+
- Add or modify the supported versions array in the loader's [phpdetection.cpp](prod/native/loader/code/phpdetection.cpp) file.
133+
- Add or remove metadata for the specified PHP version in [conandata.yml](prod/native/building/dependencies/php-headers/conandata.yml).
134+
- Add or remove the Conan dependency for php-headers-* in [conanfile.txt](prod/native/conanfile.txt).
135+
- Follow the steps in the ["Building the native library like on CI"](#building-the-native-library-like-on-ci) section to configure and build the agent.
136+
- To speed up CI builds, upload Conan artifacts to Artifactory if support for the new PHP release has been added (see [Building and publishing conan artifacts](#building-and-publishing-conan-artifacts))
137+
138+
# Managing PHP 3rd party dependencies
139+
This documentation section describes how to manage PHP 3rd party dependencies
140+
i.e., `vendor` directory, `composer.json` and `composer.lock`
141+
142+
We would like to have reproducible builds, so we need to ensure that the same
143+
versions of dependencies are used for each build of the source code repository snapshot.
144+
To achieve this, we committed `composer.lock` files to version control.
145+
There are multiple `composer.lock` files - one for each supported major.minor PHP version.
146+
147+
## To install dependencies
148+
149+
Run
150+
```
151+
./tools/build/install_PHP_deps_in_dev_env.sh
152+
```
153+
Instead of the usual `composer install`.
154+
This will select one of the generated composer's lock files (the one that corresponds to the current PHP version),
155+
copy it to `<repo root>/composer.lock`and install the packages using that `composer.lock` file.
156+
157+
## To check which dependencies can be updated
158+
Run
159+
```
160+
composer outdated
161+
```
162+
163+
## To update dependencies
164+
1) Update `composer.json` to the desired version of the dependency
165+
2) Run
166+
```
167+
./tools/build/generate_composer_lock_files.sh && ./tools/build/install_PHP_deps_in_dev_env.sh
168+
```
169+
170+
instead of the usual `composer update`
171+
172+
3) Commit the changes to the composer's lock files
173+

‎README.md‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,17 @@
11
# OpenTelemetry Php Distro
22

3+
> **⚠️ Note:** This project is currently in the process of being contributed to the OpenTelemetry community. The first official release is coming soon. Stay tuned for updates!
4+
35
## Maintainers
46

5-
- TODO
7+
- [@intuibase](https://github.com/intuibase)
8+
- [@SergeyKleyman](https://github.com/SergeyKleyman)
69

710
For more information about the maintainer role, see the [community repository](https://github.com/open-telemetry/community/blob/main/guides/contributor/membership.md#maintainer).
811

912
## Approvers
1013

11-
- TODO
14+
- [@intuibase](https://github.com/intuibase)
15+
- [@SergeyKleyman](https://github.com/SergeyKleyman)
1216

1317
For more information about the approver role, see the [community repository](https://github.com/open-telemetry/community/blob/main/guides/contributor/membership.md#approver).

‎prod/native/building/dockerized/README.md‎

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
# Docker Build Images
2+
3+
This directory contains Docker Compose configuration for building native components of the OpenTelemetry PHP Distro across multiple architectures.
4+
5+
## Image Architecture
6+
7+
The build system uses a two-layer image structure:
8+
9+
### 1. Base Build Images (GCC Toolchain)
10+
11+
Base images contain the core build toolchain including GCC, Binutils, CMake, and Python.
12+
13+
**Image naming pattern:**
14+
```
15+
otel/opentelemetry-php-distro-dev:native-build-{architecture}-gcc{gcc_version}-v{base_version}
16+
```
17+
18+
**Supported architectures:**
19+
- `linux-x86-64` - Linux x86-64 (glibc)
20+
- `linuxmusl-x86-64` - Linux x86-64 (musl)
21+
- `linux-arm64` - Linux ARM64 (glibc)
22+
- `linuxmusl-arm64` - Linux ARM64 (musl)
23+
24+
**Current versions:**
25+
- GCC: 15.2.0
26+
- Binutils: 2.45.1
27+
- CMake: 4.2.1
28+
- Python: 3.14.0
29+
- Base version: v0.0.2
30+
31+
**Example:**
32+
```
33+
otel/opentelemetry-php-distro-dev:native-build-linux-x86-64-gcc15.2.0-v0.0.2
34+
```
35+
36+
### 2. Conan Cache Images
37+
38+
Built on top of base images, these images include pre-populated Conan package caches to speed up builds.
39+
40+
**Image naming pattern:**
41+
```
42+
otel/opentelemetry-php-distro-dev:native-build-{architecture}-gcc{gcc_version}-v{base_version}-conancache-v{cache_version}
43+
```
44+
45+
**Current cache version:** v0.0.1
46+
47+
**Example:**
48+
```
49+
otel/opentelemetry-php-distro-dev:native-build-linux-x86-64-gcc15.2.0-v0.0.2-conancache-v0.0.1
50+
```
51+
52+
## Image Dependency Chain
53+
54+
```
55+
Base Image (GCC Toolchain)
56+
└─→ Conan Cache Image
57+
```
58+
59+
Each Conan cache image depends on its corresponding base image:
60+
61+
| Conan Cache Image | Base Image |
62+
|------------------|------------|
63+
| `build_linux-x86-64_conan` | `build_linux-x86-64` |
64+
| `build_linuxmusl-x86-64_conan` | `build_linuxmusl-x86-64` |
65+
| `build_linux-arm64_conan` | `build_linux-arm64` |
66+
| `build_linuxmusl-arm64_conan` | `build_linuxmusl-arm64` |
67+
68+
## Building Images
69+
70+
Build base images only:
71+
```bash
72+
docker compose build build_linux-x86-64
73+
docker compose build build_linuxmusl-x86-64
74+
docker compose build build_linux-arm64
75+
docker compose build build_linuxmusl-arm64
76+
```
77+
78+
Build Conan cache images (automatically builds base images as dependencies):
79+
```bash
80+
docker compose build build_linux-x86-64_conan
81+
docker compose build build_linuxmusl-x86-64_conan
82+
docker compose build build_linux-arm64_conan
83+
docker compose build build_linuxmusl-arm64_conan
84+
```
85+
86+
Build all images:
87+
```bash
88+
docker compose build
89+
```
90+
91+
## Platform Requirements
92+
93+
ARM64 images (`linux-arm64` and `linuxmusl-arm64`) require ARM64 platform support. These are built with `platform: linux/arm64` specification and should be built on ARM64 hardware or using Docker BuildKit with cross-platform support.
94+
95+
## Version Management
96+
97+
**All versions are parameterized and only require editing the `docker-compose.yml` file.**
98+
99+
All toolchain versions (GCC, Binutils, CMake, Python) and image tags are defined in `docker-compose.yml`. This single source of truth is used by both local builds and CI/CD pipelines.
100+
101+
### CI/CD Automatic Image Building
102+
103+
During CI builds, the image versions are automatically read from `docker-compose.yml`:
104+
- If the image already exists in DockerHub, it will be pulled and used
105+
- If the image doesn't exist in DockerHub, it will be automatically built during the CI process
106+
107+
This ensures that new image versions are created on-demand without manual intervention.
108+
109+
### Updating Toolchain Versions
110+
111+
When updating toolchain versions:
112+
113+
1. **Edit only `docker-compose.yml`** to update:
114+
- Version arguments (`GCC_VERSION`, `BINUTILS_VERSION`, `CMAKE_VERSION`, `PYTHON_VERSION`)
115+
- Image tags (increment base version, e.g., `v0.0.2` → `v0.0.3`)
116+
- Cache version if needed (e.g., `conancache-v0.0.1` → `conancache-v0.0.2`)
117+
- `BASEIMAGE` argument in Conan cache services to match new base image tags
118+
119+
2. **Commit and push** to trigger CI build
120+
121+
3. CI will automatically detect the new image versions and build them if they don't exist in DockerHub
122+
123+
## Publishing Images
124+
125+
After building and testing images locally, you must create a branch in the upstream repository. Only in this case will the images be built in CI and pushed to Docker Hub. For pull requests from forks, images will be built but push is blocked for security reasons.
126+
127+
## Updating the Base Image for Distro Builds
128+
129+
To update the base image used for building the distro, modify the image reference in the [`tools/build/build_native.sh`](../../../../tools/build/build_native.sh) script.

0 commit comments

Comments
 (0)