Skip to content

Commit c3c06b6

Browse files
edersonbrilhanteedersonbrilhante
committed
docs: add documentation and mask as experimental
1 parent 79efa8d commit c3c06b6

File tree

1 file changed

+206
-0
lines changed

1 file changed

+206
-0
lines changed

‎docs/configuration.md‎

Lines changed: 206 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -328,6 +328,212 @@ Below is an example of the log messages created.
328328
}
329329
```
330330

331+
### Dynamic Labels
332+
333+
This feature is in early stage and therefore disabled by default. To enable dynamic labels, set `enable_dynamic_labels = true`.
334+
335+
Dynamic labels allow workflow authors to pass arbitrary metadata and EC2 instance overrides directly from the `runs-on` labels in their GitHub Actions workflows. All labels prefixed with `ghr-` are treated as dynamic labels. A deterministic hash of all `ghr-` prefixed labels is computed and used for runner matching, ensuring that each unique combination of dynamic labels routes to the correct runner configuration.
336+
337+
Dynamic labels serve two purposes:
338+
339+
1. **Custom identity / restriction labels (`ghr-<key>:<value>`)** — Any `ghr-` prefixed label that is *not* `ghr-ec2-` acts as a custom identity label. These can represent a unique job ID, a team name, a cost center, an environment tag, or any arbitrary restriction. They do not affect EC2 configuration but are included in the label hash, guaranteeing unique runner matching per combination.
340+
2. **EC2 override labels (`ghr-ec2-<key>:<value>`)** — Labels prefixed with `ghr-ec2-` are parsed by the scale-up lambda to dynamically configure the EC2 fleet request — including instance type, vCPU/memory requirements, GPU/accelerator specs, EBS volumes, placement, and networking. This eliminates the need to create separate runner configurations for each hardware combination.
341+
342+
#### How it works
343+
344+
When `enable_dynamic_labels` is enabled, the webhook and scale-up lambdas inspect the `runs-on` labels of incoming `workflow_job` events. Labels starting with `ghr-ec2-` are parsed into an EC2 override configuration that is applied to the [CreateFleet](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateFleet.html) API call. All other `ghr-` prefixed labels are carried through as custom identity labels. A deterministic hash of **all** `ghr-` prefixed labels (both custom and EC2) is used to ensure consistent and unique runner matching.
345+
346+
#### Configuration
347+
348+
```hcl
349+
module "runners" {
350+
source = "github-aws-runners/github-runners/aws"
351+
352+
...
353+
enable_dynamic_labels = true
354+
...
355+
}
356+
```
357+
358+
#### Custom identity labels
359+
360+
Any label matching `ghr-<key>:<value>` (where `<key>` does **not** start with `ec2-`) is a custom identity label. These labels have no effect on EC2 instance configuration but are included in the runner matching hash. Use them to:
361+
362+
- Assign a **unique job identity** so each workflow run targets a dedicated runner (e.g., `ghr-job-id:abc123`).
363+
- Apply **team or cost-center restrictions** (e.g., `ghr-team:platform`, `ghr-cost-center:eng-42`).
364+
- Tag runners with **environment or deployment context** (e.g., `ghr-env:staging`, `ghr-region:us-west-2`).
365+
- Enforce **any custom constraint** that differentiates one runner request from another.
366+
367+
```yaml
368+
jobs:
369+
deploy:
370+
runs-on:
371+
- self-hosted
372+
- linux
373+
- ghr-team:platform
374+
- ghr-env:staging
375+
- ghr-job-id:${{ github.run_id }}
376+
```
377+
378+
In the example above, the three `ghr-` labels produce a unique hash, ensuring this job is matched to a runner created specifically for this combination. No EC2 overrides are applied — the runner uses the default fleet configuration.
379+
380+
#### EC2 override labels
381+
382+
Labels using the format `ghr-ec2-<key>:<value>` override EC2 fleet configuration. Values with multiple items use comma-separated lists.
383+
384+
##### Basic Fleet Overrides
385+
386+
| Label | Description | Example value |
387+
| -------------------------------------------- | ------------------------------------ | ------------------- |
388+
| `ghr-ec2-instance-type:<type>` | Set specific instance type | `c5.xlarge` |
389+
| `ghr-ec2-max-price:<price>` | Set maximum spot price | `0.10` |
390+
| `ghr-ec2-subnet-id:<id>` | Set subnet ID | `subnet-abc123` |
391+
| `ghr-ec2-availability-zone:<zone>` | Set availability zone | `us-east-1a` |
392+
| `ghr-ec2-availability-zone-id:<id>` | Set availability zone ID | `use1-az1` |
393+
| `ghr-ec2-weighted-capacity:<number>` | Set weighted capacity | `2` |
394+
| `ghr-ec2-priority:<number>` | Set launch priority | `1` |
395+
| `ghr-ec2-image-id:<ami-id>` | Override AMI ID | `ami-0abcdef123` |
396+
397+
##### Instance Requirements — vCPU & Memory
398+
399+
| Label | Description | Example value |
400+
| -------------------------------------------- | ------------------------------------ | ------------------- |
401+
| `ghr-ec2-vcpu-count-min:<number>` | Minimum vCPU count | `4` |
402+
| `ghr-ec2-vcpu-count-max:<number>` | Maximum vCPU count | `16` |
403+
| `ghr-ec2-memory-mib-min:<number>` | Minimum memory in MiB | `16384` |
404+
| `ghr-ec2-memory-mib-max:<number>` | Maximum memory in MiB | `65536` |
405+
| `ghr-ec2-memory-gib-per-vcpu-min:<number>` | Min memory per vCPU ratio (GiB) | `2` |
406+
| `ghr-ec2-memory-gib-per-vcpu-max:<number>` | Max memory per vCPU ratio (GiB) | `8` |
407+
408+
##### Instance Requirements — CPU & Performance
409+
410+
| Label | Description | Example value |
411+
| -------------------------------------------- | ----------------------------------------------------------------- | -------------------------- |
412+
| `ghr-ec2-cpu-manufacturers:<list>` | CPU manufacturers (comma-separated) | `intel,amd` |
413+
| `ghr-ec2-instance-generations:<list>` | Instance generations (comma-separated) | `current` |
414+
| `ghr-ec2-excluded-instance-types:<list>` | Exclude instance types (comma-separated) | `t2.micro,t3.nano` |
415+
| `ghr-ec2-allowed-instance-types:<list>` | Allow only specific instance types (comma-separated) | `c5.xlarge,c5.2xlarge` |
416+
| `ghr-ec2-burstable-performance:<value>` | Burstable performance (`included`, `excluded`, `required`) | `excluded` |
417+
| `ghr-ec2-bare-metal:<value>` | Bare metal (`included`, `excluded`, `required`) | `excluded` |
418+
419+
##### Instance Requirements — Accelerators / GPU
420+
421+
| Label | Description | Example value |
422+
| ------------------------------------------------- | ------------------------------------------------------------------------ | -------------------------------- |
423+
| `ghr-ec2-accelerator-types:<list>` | Accelerator types (comma-separated: `gpu`, `fpga`, `inference`) | `gpu` |
424+
| `ghr-ec2-accelerator-count-min:<number>` | Minimum accelerator count | `1` |
425+
| `ghr-ec2-accelerator-count-max:<number>` | Maximum accelerator count | `4` |
426+
| `ghr-ec2-accelerator-manufacturers:<list>` | Accelerator manufacturers (comma-separated) | `nvidia` |
427+
| `ghr-ec2-accelerator-names:<list>` | Specific accelerator names (comma-separated) | `t4,v100` |
428+
| `ghr-ec2-accelerator-memory-mib-min:<number>` | Min accelerator total memory in MiB | `8192` |
429+
| `ghr-ec2-accelerator-memory-mib-max:<number>` | Max accelerator total memory in MiB | `32768` |
430+
431+
##### Instance Requirements — Network & Storage
432+
433+
| Label | Description | Example value |
434+
| -------------------------------------------------- | ----------------------------------------------------------------- | ------------------- |
435+
| `ghr-ec2-network-interface-count-min:<number>` | Min network interfaces | `1` |
436+
| `ghr-ec2-network-interface-count-max:<number>` | Max network interfaces | `4` |
437+
| `ghr-ec2-network-bandwidth-gbps-min:<number>` | Min network bandwidth in Gbps | `10` |
438+
| `ghr-ec2-network-bandwidth-gbps-max:<number>` | Max network bandwidth in Gbps | `25` |
439+
| `ghr-ec2-local-storage:<value>` | Local storage (`included`, `excluded`, `required`) | `required` |
440+
| `ghr-ec2-local-storage-types:<list>` | Local storage types (comma-separated: `hdd`, `ssd`) | `ssd` |
441+
| `ghr-ec2-total-local-storage-gb-min:<number>` | Min total local storage in GB | `100` |
442+
| `ghr-ec2-total-local-storage-gb-max:<number>` | Max total local storage in GB | `500` |
443+
| `ghr-ec2-baseline-ebs-bandwidth-mbps-min:<number>` | Min baseline EBS bandwidth in Mbps | `1000` |
444+
| `ghr-ec2-baseline-ebs-bandwidth-mbps-max:<number>` | Max baseline EBS bandwidth in Mbps | `5000` |
445+
446+
##### Placement
447+
448+
| Label | Description | Example value |
449+
| ------------------------------------------------------ | ---------------------------------------------------------- | --------------------- |
450+
| `ghr-ec2-placement-group:<name>` | Placement group name | `my-cluster-group` |
451+
| `ghr-ec2-placement-tenancy:<value>` | Tenancy (`default`, `dedicated`, `host`) | `dedicated` |
452+
| `ghr-ec2-placement-host-id:<id>` | Dedicated host ID | `h-abc123` |
453+
| `ghr-ec2-placement-affinity:<value>` | Affinity (`default`, `host`) | `host` |
454+
| `ghr-ec2-placement-partition-number:<number>` | Partition number | `1` |
455+
| `ghr-ec2-placement-availability-zone:<zone>` | Placement availability zone | `us-east-1a` |
456+
| `ghr-ec2-placement-spread-domain:<domain>` | Spread domain | `my-domain` |
457+
| `ghr-ec2-placement-host-resource-group-arn:<arn>` | Host resource group ARN | `arn:aws:...` |
458+
459+
##### Block Device Mappings (EBS)
460+
461+
| Label | Description | Example value |
462+
| ------------------------------------------------ | -------------------------------------------------------------- | -------------- |
463+
| `ghr-ec2-ebs-volume-size:<size>` | EBS volume size in GB | `100` |
464+
| `ghr-ec2-ebs-volume-type:<type>` | EBS volume type (`gp2`, `gp3`, `io1`, `io2`, `st1`, `sc1`) | `gp3` |
465+
| `ghr-ec2-ebs-iops:<number>` | EBS IOPS | `3000` |
466+
| `ghr-ec2-ebs-throughput:<number>` | EBS throughput in MB/s (gp3 only) | `125` |
467+
| `ghr-ec2-ebs-encrypted:<boolean>` | EBS encryption (`true`, `false`) | `true` |
468+
| `ghr-ec2-ebs-kms-key-id:<id>` | KMS key ID for encryption | `key-abc123` |
469+
| `ghr-ec2-ebs-delete-on-termination:<boolean>` | Delete on termination (`true`, `false`) | `true` |
470+
| `ghr-ec2-ebs-snapshot-id:<id>` | Snapshot ID for EBS volume | `snap-abc123` |
471+
| `ghr-ec2-block-device-virtual-name:<name>` | Virtual device name (ephemeral storage) | `ephemeral0` |
472+
| `ghr-ec2-block-device-no-device:<string>` | Suppresses device mapping | `true` |
473+
474+
##### Pricing & Advanced
475+
476+
| Label | Description | Example value |
477+
| ----------------------------------------------------------------------------- | ------------------------------------------------------------------ | -------------- |
478+
| `ghr-ec2-spot-max-price-percentage-over-lowest-price:<number>` | Spot max price as % over lowest price | `20` |
479+
| `ghr-ec2-on-demand-max-price-percentage-over-lowest-price:<number>` | On-demand max price as % over lowest price | `10` |
480+
| `ghr-ec2-max-spot-price-as-percentage-of-optimal-on-demand-price:<number>` | Max spot price as % of optimal on-demand | `50` |
481+
| `ghr-ec2-require-hibernate-support:<boolean>` | Require hibernate support (`true`, `false`) | `true` |
482+
| `ghr-ec2-require-encryption-in-transit:<boolean>` | Require encryption in-transit (`true`, `false`) | `true` |
483+
| `ghr-ec2-baseline-performance-factors-cpu-reference-families:<list>` | CPU baseline performance reference families (comma-separated) | `c5,m5` |
484+
485+
#### Examples
486+
487+
Custom identity labels only — unique runner per job run:
488+
489+
```yaml
490+
jobs:
491+
deploy:
492+
runs-on:
493+
- self-hosted
494+
- linux
495+
- ghr-job-id:${{ github.run_id }}
496+
```
497+
498+
Specific instance type with a larger EBS volume:
499+
500+
```yaml
501+
jobs:
502+
build:
503+
runs-on:
504+
- self-hosted
505+
- linux
506+
- ghr-ec2-instance-type:c5.2xlarge
507+
- ghr-ec2-ebs-volume-size:200
508+
- ghr-ec2-ebs-volume-type:gp3
509+
```
510+
511+
Attribute-based instance selection with Intel CPUs only:
512+
513+
```yaml
514+
jobs:
515+
test:
516+
runs-on:
517+
- self-hosted
518+
- linux
519+
- ghr-ec2-vcpu-count-min:2
520+
- ghr-ec2-vcpu-count-max:8
521+
- ghr-ec2-memory-mib-min:8192
522+
- ghr-ec2-cpu-manufacturers:intel
523+
- ghr-ec2-burstable-performance:excluded
524+
```
525+
526+
#### Considerations
527+
528+
- This feature requires `enable_dynamic_labels = true` in your Terraform configuration.
529+
- When using `ghr-ec2-instance-type`, the fleet request uses a direct instance type override. When using `ghr-ec2-vcpu-count-*`, `ghr-ec2-memory-mib-*`, or other instance requirement labels, the fleet request uses [attribute-based instance type selection](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-fleet-attribute-based-instance-type-selection.html).
530+
- Labels are parsed at the scale-up lambda level — they do not change after the instance is launched.
531+
- A deterministic hash of all `ghr-` prefixed labels (both custom identity and EC2 override) is used for runner matching. Different label combinations produce different hashes, ensuring each unique set of requirements gets its own runner.
532+
- Custom `ghr-` labels (non-`ec2`) are free-form — you can use any key/value pair. They are not validated by the module.
533+
- Multiple EBS labels apply to the same (first) block device mapping. If you need more complex block device configurations, use a custom AMI or launch template instead.
534+
- This feature is compatible with both org-level and repo-level runners, spot and on-demand instances, and ephemeral and non-ephemeral runners.
535+
- Be mindful of the security implications: enabling this feature allows workflow authors to influence EC2 instance configuration via `ghr-ec2-` labels. Ensure your IAM policies and subnet configurations provide appropriate guardrails.
536+
331537
### EventBridge
332538

333539
This module can be deployed in `EventBridge` mode. The `EventBridge` mode will publish an event to an eventbus. Within the eventbus, there is a target rule set, sending events to the dispatch lambda. The `EventBridge` mode is enabled by default.

0 commit comments

Comments
 (0)