Skip to content

Commit b3593be

Browse files
igorpecovnikigorpecovnik
committed
Add automatic target YAML generation from image-info.json
Implements automated generation of build target YAML files with hardware-based board classification and kernel description generation. **Features:** - Automatic target generation for apps, standard-support, nightly, and community builds - Hardware-based classification (fast/slow/headless/riscv64/loongarch) for desktop selection - Automatic extensions (v4l2loopback-dkms, mesa-vpu) for fast HDMI boards - Kernel description generation with template-based categorization - Manual overrides via blacklist, manual targets, and extensions map files **Desktop selection by hardware:** - Fast HDMI (ARM64/x86): GNOME desktop - Slow HDMI (32-bit, older SoCs): XFCE desktop - RISC-V: XFCE desktop - LoongArch: CLI only (bookworm minimal) - Headless: CLI only **New files:** - `.github/workflows/generate-build-lists.yaml`: GitHub Actions workflow - `scripts/generate_targets.py`: Main target generation script - `scripts/generate_kernel_descriptions.py`: Kernel description generator - `release-targets/targets-extensions.map`: Manual board extensions - `release-targets/targets-release-*.blacklist`: Board exclusions - `release-targets/targets-release-*.manual`: Custom build targets **Generated outputs (published to data branch):** - `targets-release-apps.yaml`: One image per board with app extensions - `targets-release-standard-support.yaml`: Full stable/release builds - `targets-release-nightly.yaml`: Nightly builds with hardware-based desktops - `targets-release-community-maintained.yaml`: Community/CSC/TVB builds - `exposed.map`: Regex patterns for recommended images - `kernel-description.json`: Kernel branch descriptions for download pages Signed-off-by: Igor Pecovnik <igor@armbian.com>
1 parent 04b2e97 commit b3593be

8 files changed

Lines changed: 2028 additions & 0 deletions

‎.github/workflows/generate-build-lists.yaml‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
name: "Infrastructure: Generate image build lists"
2+
3+
on:
4+
workflow_dispatch: # Manually triggered via GitHub Actions UI
5+
repository_dispatch:
6+
types: ["Generate lists"]
7+
push:
8+
paths:
9+
- 'release-targets/**'
10+
11+
env:
12+
IMAGE_INFO_URL: "https://github.armbian.com/image-info.json"
13+
14+
jobs:
15+
16+
Check:
17+
name: "Check permissions"
18+
runs-on: "ubuntu-24.04"
19+
if: ${{ github.repository_owner == 'Armbian' }}
20+
steps:
21+
22+
- name: "Check membership"
23+
uses: armbian/actions/team-check@main
24+
with:
25+
ORG_MEMBERS: ${{ secrets.ORG_MEMBERS }}
26+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
27+
TEAM: "Release manager"
28+
29+
generate:
30+
name: "Generate target YAML files"
31+
runs-on: ubuntu-24.04
32+
if: ${{ github.repository_owner == 'Armbian' }}
33+
needs: Check
34+
steps:
35+
36+
- name: Checkout armbian.github.io repository
37+
uses: actions/checkout@v6
38+
with:
39+
fetch-depth: 0
40+
41+
- name: Download image-info.json
42+
run: |
43+
curl -fL -o image-info.json "${{ env.IMAGE_INFO_URL }}"
44+
45+
- name: Install Python dependencies
46+
run: |
47+
python3 -m pip install --upgrade pip
48+
49+
- name: Run generate_targets.py
50+
run: |
51+
python3 scripts/generate_targets.py image-info.json release-targets 2>&1 | tee -a generation.log
52+
53+
- name: Generate kernel descriptions
54+
run: |
55+
python3 scripts/generate_kernel_descriptions.py image-info.json release-targets/kernel-description.json
56+
57+
- name: Show generation summary
58+
run: |
59+
echo "### Generated files summary" >> $GITHUB_STEP_SUMMARY
60+
echo "" >> $GITHUB_STEP_SUMMARY
61+
62+
for yaml_file in release-targets/targets-release-*.yaml; do
63+
if [[ -f "$yaml_file" ]]; then
64+
base_name=$(basename "$yaml_file" .yaml)
65+
count=$(grep -c '^ *- { BOARD:' "$yaml_file" || echo "0")
66+
echo "- **$base_name**: $count boards" >> $GITHUB_STEP_SUMMARY
67+
fi
68+
done
69+
70+
- name: Checkout data branch
71+
uses: actions/checkout@v6
72+
with:
73+
ref: data
74+
path: data-branch
75+
fetch-depth: 0
76+
77+
- name: Copy YAML files to data branch
78+
run: |
79+
mkdir -p data-branch/data/release-targets
80+
cp release-targets/targets-release-*.yaml data-branch/data/release-targets/
81+
cp release-targets/exposed.map data-branch/data/release-targets/
82+
cp release-targets/kernel-description.json data-branch/data/release-targets/
83+
84+
- name: Commit to data branch
85+
run: |
86+
cd data-branch
87+
git config user.name "GitHub Actions"
88+
git config user.email "actions@github.com"
89+
git add data/release-targets/
90+
if git diff --staged --quiet; then
91+
echo "No changes to commit"
92+
else
93+
git commit -m "Update release targets and kernel descriptions [skip ci]"
94+
git push
95+
fi
96+
97+
- name: Trigger website generation
98+
if: success()
99+
uses: peter-evans/repository-dispatch@v4
100+
with:
101+
token: ${{ secrets.DISPATCH }}
102+
repository: armbian/actions
103+
event-type: "Web: Directory listing"

‎release-targets/README.md‎

Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
# Armbian Target YAML Generator
2+
3+
This script generates Armbian CI/CD pipeline configuration YAML files from `image-info.json`.
4+
5+
## Quick Start
6+
7+
All configuration files should be in this directory (`release-targets/`):
8+
9+
```bash
10+
# From repository root
11+
python3 scripts/generate_targets.py image-info.json release-targets/
12+
```
13+
14+
## Configuration Files
15+
16+
The script reads the following files from the output directory (same location as generated files):
17+
18+
- **`targets-extensions.map`** - Manual extensions for specific boards/branches (optional)
19+
- **`targets-release-<type>.blacklist`** - Boards to exclude from each target type (optional)
20+
- **`targets-release-<type>.manual`** - Additional YAML to append to each target (optional)
21+
22+
## Generated Files
23+
24+
The script generates 4 YAML files with the naming pattern `targets-release-<type>.yaml`:
25+
26+
### 1. targets-release-apps.yaml
27+
Application-specific images for conf/wip boards
28+
- `apps-ha`: Home Assistant images (Ubuntu Noble + Gnome)
29+
- `apps-openhab`: openHAB images (Ubuntu Noble + Gnome)
30+
- `apps-kali`: Kali Linux images (Kali rolling + XFCE)
31+
32+
### 2. targets-release-standard-support.yaml
33+
Standard support release images for conf/wip boards, split by performance:
34+
35+
**Lists:**
36+
- `stable-current-fast-hdmi`: Fast 64-bit boards with video
37+
- `stable-current-slow-hdmi`: Slow 32-bit boards with video
38+
- `stable-current-headless`: Boards without video output
39+
- `stable-vendor-fast-hdmi`: Fast 64-bit vendor branch boards
40+
- `stable-vendor-slow-hdmi`: Slow 32-bit vendor branch boards
41+
- `stable-vendor-headless`: Headless vendor branch boards
42+
43+
**Targets:**
44+
- `minimal-stable-debian`: Debian Trixie minimal
45+
- `minimal-stable-ubuntu`: Ubuntu Noble minimal
46+
- `desktop-stable-ubuntu`: Ubuntu Noble XFCE desktop (fast-hdmi only)
47+
48+
### 3. targets-release-nightly.yaml
49+
Nightly builds for conf/wip boards, split by performance:
50+
51+
**Lists:**
52+
- `nightly-fast-hdmi`: Fast 64-bit boards with video
53+
- `nightly-slow-hdmi`: Slow 32-bit boards with video
54+
- `nightly-headless`: Boards without video output
55+
56+
**Targets:**
57+
- `nightly-forky-all`: Debian Forky minimal CLI for all boards
58+
- `nightly-noble-gnome`: Ubuntu Noble GNOME desktop (fast HDMI)
59+
- `nightly-noble-xfce`: Ubuntu Noble XFCE desktop (slow HDMI)
60+
- `nightly-noble-minimal`: Ubuntu Noble minimal CLI (headless/exotic)
61+
62+
### 4. targets-release-community-maintained.yaml
63+
Community-maintained builds for csc/tvb boards:
64+
65+
**Lists:**
66+
- `community-current`: Current branch community boards
67+
- `community-vendor`: Vendor branch community boards
68+
69+
**Targets:**
70+
- `community-forky-all`: Debian Forky minimal CLI for all boards
71+
- `community-noble-gnome`: Ubuntu Noble GNOME desktop (fast HDMI)
72+
- `community-noble-xfce`: Ubuntu Noble XFCE desktop (slow HDMI)
73+
- `community-noble-minimal`: Ubuntu Noble minimal CLI (headless/exotic)
74+
75+
## Board Classification
76+
77+
### By Performance (KERNEL_SRC_ARCH)
78+
- **Slow HDMI**: `arm` (32-bit boards)
79+
- **Fast HDMI**: `arm64`, `x86` (modern 64-bit boards)
80+
- **RISC-V**: `riscv64` (separate category)
81+
- **LoongArch**: `loongarch64` (separate category)
82+
- **Headless**: Boards with `BOARD_HAS_VIDEO: false`
83+
84+
### By Support Level
85+
- **conf/wip**: Higher quality support (stable and nightly builds)
86+
- **csc/tvb**: Community/experimental support (community builds)
87+
88+
## Configuration File Formats
89+
90+
### targets-extensions.map
91+
92+
Add manual extensions for specific boards (merged with automatic fast‑HDMI extensions):
93+
94+
```ini
95+
# Format: BOARD_NAME:branch1:branch2:...:ENABLE_EXTENSIONS="extension1,extension2"
96+
97+
# Add to specific branches
98+
khadas-vim1s:legacy:current:edge::ENABLE_EXTENSIONS="image-output-oowow"
99+
100+
# Add to single branch
101+
rock-5b:current::ENABLE_EXTENSIONS="custom-extension"
102+
103+
# Add to all branches
104+
nanopi-r4s:::ENABLE_EXTENSIONS="test-extension"
105+
106+
# Multiple extensions
107+
board-name:current::ENABLE_EXTENSIONS="ext1,ext2,ext3"
108+
```
109+
110+
### targets-release-<type>.blacklist
111+
112+
Exclude specific boards from a target type (one board name per line):
113+
114+
```
115+
# Lines starting with # are comments
116+
ayn-odin2
117+
mekotronics-r58hd
118+
mekotronics-r58-4x4
119+
bananapim5
120+
```
121+
122+
### targets-release-<type>.manual
123+
124+
Additional YAML that gets appended to the auto-generated targets section:
125+
126+
```yaml
127+
# Ubuntu stable minimal cloud
128+
minimal-cli-stable-ubuntu-cloud:
129+
enabled: yes
130+
configs: [ armbian-cloud ]
131+
pipeline:
132+
gha: *armbian-gha
133+
build-image: "yes"
134+
vars:
135+
RELEASE: noble
136+
BUILD_MINIMAL: "yes"
137+
BUILD_DESKTOP: "no"
138+
items:
139+
- { BOARD: uefi-arm64, BRANCH: cloud, ENABLE_EXTENSIONS: "image-output-qcow2" }
140+
```
141+
142+
## Automatic Extensions
143+
144+
All fast HDMI boards (64-bit boards with video output) automatically get:
145+
- `v4l2loopback-dkms`
146+
- `mesa-vpu`
147+
148+
**Note**: Manual extensions from `targets-extensions.map` are MERGED with automatic extensions.
149+
150+
## Usage
151+
152+
```bash
153+
python3 scripts/generate_targets.py <image-info.json> <output_directory>
154+
```
155+
156+
Both arguments are required. The output directory should contain `targets-extensions.map` and any `.blacklist`/`.manual` files.
157+
158+
## Example
159+
160+
```bash
161+
# Download latest image-info.json
162+
curl -L -o image-info.json https://github.armbian.com/image-info.json
163+
164+
# Generate target YAML files to https://github.armbian.com/release-targets/ directory
165+
python3 scripts/generate_targets.py image-info.json release-targets/
166+
```
167+
168+
## Requirements
169+
170+
- Python 3.6+
171+
- image-info.json (from Armbian build framework or github.armbian.com)
172+
- targets-extensions.map (optional, but recommended in output directory)
173+
- .blacklist and .manual files (optional, per target type)

‎release-targets/targets-extensions.map‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Manual extensions mapping for specific boards
2+
# Format: BOARD_NAME:branch1:branch2:...:ENABLE_EXTENSIONS="extension1,extension2"
3+
4+
# Example 1: Board with automatic RK35* extensions getting additional manual extension
5+
# 9tripod-x3568-v4 has RK3568, so it gets v4l2loopback-dkms,mesa-vpu automatically
6+
# We add image-output-oowow manually
7+
# 9tripod-x3568-v4:current::ENABLE_EXTENSIONS="image-output-oowow"
8+
# Example 2: Board without automatic extensions, adding manual extensions
9+
# khadas-vim1:current::ENABLE_EXTENSIONS="image-output-oowow"
10+
# Example 3: Multiple manual extensions
11+
# nanopi-r4s:::ENABLE_EXTENSIONS="ext1,ext2,ext3"
12+
13+
khadas-edge2:::ENABLE_EXTENSIONS="image-output-oowow"
14+
khadas-vim1s:::ENABLE_EXTENSIONS="image-output-oowow"
15+
khadas-vim3:::ENABLE_EXTENSIONS="image-output-oowow"
16+
khadas-vim3l:::ENABLE_EXTENSIONS="image-output-oowow"
17+
khadas-vim4:::ENABLE_EXTENSIONS="image-output-oowow"

‎release-targets/targets-release-community-maintained.blacklist‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
qemu-uboot-arm64
2+
qemu-uboot-x86
3+
qemu-uefi-x86
4+
qemu-uefi-arm64
5+
thinkpad-x13s
6+
xiaomi-elish
7+
oneplus-kebab
8+
ayn-odin2
9+
mekotronics-r58hd
10+
mekotronics-r58-4x4
11+
retroidpocket-rp5
12+
retroidpocket-rpmini
13+
orangepi5-ultra
14+
orangepi5-max

‎release-targets/targets-release-nightly.blacklist‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
radxa-dragon-q6a
2+
bananapim2pro
3+
bananapi-m2s
4+
jethub-d1
5+
jethub-h1
6+
jethubj80
7+
jethubj100
8+
jethubj200
9+
khadas-vim1
10+
khadas-vim2
11+
lafrite
12+
lepotato
13+
odroidc2
14+
olimex-teres-a64
15+
onecloud
16+
pine64
17+
radxa-zero
18+
renegade
19+
tritium-h3
20+
tritium-h5
21+
xiaomi-elish
22+
oneplus-kebab

‎release-targets/targets-release-nightly.manual‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# Manual target additions for nightly builds
2+
# This content will be appended to the auto-generated targets-release-nightly.yaml
3+
4+
# Ubuntu stable minimal cloud
5+
minimal-cli-stable-ubuntu-cloud:
6+
enabled: yes
7+
configs: [ armbian-cloud ]
8+
pipeline:
9+
gha: *armbian-gha
10+
build-image: "yes"
11+
vars:
12+
RELEASE: plucky
13+
BUILD_MINIMAL: "yes"
14+
BUILD_DESKTOP: "no"
15+
items:
16+
- { BOARD: uefi-arm64, BRANCH: cloud, ENABLE_EXTENSIONS: "image-output-qcow2" }
17+
- { BOARD: uefi-x86, BRANCH: cloud, ENABLE_EXTENSIONS: "image-output-qcow2" }
18+
- { BOARD: uefi-arm64, BRANCH: cloud, ENABLE_EXTENSIONS: "image-output-vhdx" }
19+
- { BOARD: uefi-x86, BRANCH: cloud, ENABLE_EXTENSIONS: "image-output-vhdx" }
20+
21+
# Debian unstable minimal cloud
22+
minimal-cli-unstable-debian-cloud:
23+
enabled: yes
24+
configs: [ armbian-cloud ]
25+
pipeline:
26+
gha: *armbian-gha
27+
build-image: "yes"
28+
vars:
29+
RELEASE: forky
30+
BUILD_MINIMAL: "yes"
31+
BUILD_DESKTOP: "no"
32+
items:
33+
- { BOARD: uefi-arm64, BRANCH: cloud, ENABLE_EXTENSIONS: "image-output-qcow2" }
34+
- { BOARD: uefi-x86, BRANCH: cloud, ENABLE_EXTENSIONS: "image-output-qcow2" }
35+
- { BOARD: uefi-arm64, BRANCH: cloud, ENABLE_EXTENSIONS: "image-output-vhdx" }
36+
- { BOARD: uefi-x86, BRANCH: cloud, ENABLE_EXTENSIONS: "image-output-vhdx" }

0 commit comments

Comments
 (0)