feat: add bidirectionalLabelMatch option and deprecate exactMatch

Introduce a new bidirectionalLabelMatch option that performs strict
two-way label matching (runner labels must equal workflow labels as a
set). This preserves the existing exactMatch behavior (unidirectional
subset check) to avoid breaking changes.

The bidirectionalLabelMatch option for runner label matching
requires labels to be identical in both directions. Previously, a runner
with labels [A, B, C, D] would match a job requesting [A, B, C] when
exactMatch was true. Now, bidirectionalLabelMatch=true requires the labels to be
exactly identical - the runner will only match if the job requests
exactly [A, B, C, D].

This change affects users who have runners with extra labels (e.g.,
on-demand) that were previously matching jobs not explicitly
requesting those labels. After this change, such runners will only
be used when jobs explicitly request all of the runner labels.

Before: Job [A,B,C] + Runner [A,B,C,D] + exactMatch=true -> Match
After:  Job [A,B,C] + Runner [A,B,C,D] + exactMatch=true -> No Match

ExactMatch was suppose to have this behaviour, but the avoid breaking
changes, the variable will be deprecated to give users time to migrate.

To migrate, use bidirectionalLabelMatch instead of exactMatch in your runner configs.
Then either:
1. Remove extra labels from runner configurations, or
2. Add the extra labels to your workflow job runs-on

Signed-off-by: Brend Smits <brend.smits@philips.com>

Co-authored-by: Stuart Pearson <stuart.pearson@philips.com>

Author: Brend-Smits

README.md

@@ -126,10 +126,11 @@ Join our discord community via [this invite link](https://discord.gg/bxgXW8jJGh)
 | <a name="input_enable_job_queued_check"></a> [enable\_job\_queued\_check](#input\_enable\_job\_queued\_check) | Only scale if the job event received by the scale up lambda is in the queued state. By default enabled for non ephemeral runners and disabled for ephemeral. Set this variable to overwrite the default behavior. | `bool` | `null` | no |
 | <a name="input_enable_managed_runner_security_group"></a> [enable\_managed\_runner\_security\_group](#input\_enable\_managed\_runner\_security\_group) | Enables creation of the default managed security group. Unmanaged security groups can be specified via `runner_additional_security_group_ids`. | `bool` | `true` | no |
 | <a name="input_enable_organization_runners"></a> [enable\_organization\_runners](#input\_enable\_organization\_runners) | Register runners to organization, instead of repo level | `bool` | `false` | no |
+| <a name="input_enable_runner_bidirectional_label_match"></a> [enable\_runner\_bidirectional\_label\_match](#input\_enable\_runner\_bidirectional\_label\_match) | If set to true, the runner labels and workflow job labels must be an exact two-way match (same set, any order, no extras or missing labels). This is stricter than `enable_runner_workflow_job_labels_check_all` which only checks that workflow labels are a subset of runner labels. When false, if __any__ label matches it will trigger the webhook. | `bool` | `false` | no |
 | <a name="input_enable_runner_binaries_syncer"></a> [enable\_runner\_binaries\_syncer](#input\_enable\_runner\_binaries\_syncer) | Option to disable the lambda to sync GitHub runner distribution, useful when using a pre-build AMI. | `bool` | `true` | no |
 | <a name="input_enable_runner_detailed_monitoring"></a> [enable\_runner\_detailed\_monitoring](#input\_enable\_runner\_detailed\_monitoring) | Should detailed monitoring be enabled for the runner. Set this to true if you want to use detailed monitoring. See https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-cloudwatch-new.html for details. | `bool` | `false` | no |
 | <a name="input_enable_runner_on_demand_failover_for_errors"></a> [enable\_runner\_on\_demand\_failover\_for\_errors](#input\_enable\_runner\_on\_demand\_failover\_for\_errors) | Enable on-demand failover. For example to fall back to on demand when no spot capacity is available the variable can be set to `InsufficientInstanceCapacity`. When not defined the default behavior is to retry later. | `list(string)` | `[]` | no |
-| <a name="input_enable_runner_workflow_job_labels_check_all"></a> [enable\_runner\_workflow\_job\_labels\_check\_all](#input\_enable\_runner\_workflow\_job\_labels\_check\_all) | If set to true all labels in the workflow job must match the GitHub labels (os, architecture and `self-hosted`). When false if __any__ label matches it will trigger the webhook. | `bool` | `true` | no |
+| <a name="input_enable_runner_workflow_job_labels_check_all"></a> [enable\_runner\_workflow\_job\_labels\_check\_all](#input\_enable\_runner\_workflow\_job\_labels\_check\_all) | DEPRECATED: Use `enable_runner_bidirectional_label_match` instead. If set to true all labels in the workflow job must match the GitHub labels (os, architecture and `self-hosted`). When false if __any__ label matches it will trigger the webhook. Note: this only checks that workflow labels are a subset of runner labels, not the reverse. | `bool` | `true` | no |
 | <a name="input_enable_ssm_on_runners"></a> [enable\_ssm\_on\_runners](#input\_enable\_ssm\_on\_runners) | Enable to allow access to the runner instances for debugging purposes via SSM. Note that this adds additional permissions to the runner instances. | `bool` | `false` | no |
 | <a name="input_enable_user_data_debug_logging_runner"></a> [enable\_user\_data\_debug\_logging\_runner](#input\_enable\_user\_data\_debug\_logging\_runner) | Option to enable debug logging for user-data, this logs all secrets as well. | `bool` | `false` | no |
 | <a name="input_enable_userdata"></a> [enable\_userdata](#input\_enable\_userdata) | Should the userdata script be enabled for the runner. Set this to false if you are using your own prebuilt AMI. | `bool` | `true` | no |

lambdas/functions/webhook/src/ConfigLoader.test.ts

@@ -165,7 +165,7 @@ describe('ConfigLoader Tests', () => {
       });
 
       await expect(ConfigWebhook.load()).rejects.toThrow(
-        'Failed to load config: Failed to load parameter for matcherConfig from path /path/to/matcher/config: Failed to load matcher config', // eslint-disable-line max-len
+        'Failed to load config: Failed to load parameter for matcherConfig from path /path/to/matcher/config: Failed to load matcher config',
       );
     });
 
@@ -213,7 +213,7 @@ describe('ConfigLoader Tests', () => {
       });
 
       await expect(ConfigWebhook.load()).rejects.toThrow(
-        "Failed to load config: Failed to parse combined matcher config: Expected ',' or ']' after array element in JSON at position 196", // eslint-disable-line max-len
+        "Failed to load config: Failed to parse combined matcher config: Expected ',' or ']' after array element in JSON at position 196",
       );
     });
   });
@@ -244,7 +244,7 @@ describe('ConfigLoader Tests', () => {
       });
 
       await expect(ConfigWebhookEventBridge.load()).rejects.toThrow(
-        'Failed to load config: Environment variable for eventBusName is not set and no default value provided., Failed to load parameter for webhookSecret from path undefined: Parameter undefined not found', // eslint-disable-line max-len
+        'Failed to load config: Environment variable for eventBusName is not set and no default value provided., Failed to load parameter for webhookSecret from path undefined: Parameter undefined not found',
       );
     });
   });
@@ -309,7 +309,7 @@ describe('ConfigLoader Tests', () => {
       });
 
       await expect(ConfigDispatcher.load()).rejects.toThrow(
-        'Failed to load config: Failed to load parameter for matcherConfig from path undefined: Parameter undefined not found', // eslint-disable-line max-len
+        'Failed to load config: Failed to load parameter for matcherConfig from path undefined: Parameter undefined not found',
       );
     });
 

lambdas/functions/webhook/src/ConfigLoader.ts

@@ -61,7 +61,7 @@ abstract class BaseConfig {
         this.loadProperty(propertyName, value);
       })
       .catch((error) => {
-        const errorMessage = `Failed to load parameter for ${String(propertyName)} from path ${paramPath}: ${(error as Error).message}`; // eslint-disable-line max-len
+        const errorMessage = `Failed to load parameter for ${String(propertyName)} from path ${paramPath}: ${(error as Error).message}`;
         this.configLoadingErrors.push(errorMessage);
       });
   }

lambdas/functions/webhook/src/runners/dispatch.test.ts

@@ -225,6 +225,86 @@ describe('Dispatcher', () => {
       const runnerLabels = [['self-hosted', 'linux', 'x64']];
       expect(canRunJob(workflowLabels, runnerLabels, false)).toBe(true);
     });
+
+    it('should match when runner has more labels than workflow requests with exactMatch=true (unidirectional).', () => {
+      const workflowLabels = ['self-hosted', 'linux', 'x64', 'staging', 'ubuntu-2404'];
+      const runnerLabels = [['self-hosted', 'linux', 'x64', 'staging', 'ubuntu-2404', 'on-demand']];
+      expect(canRunJob(workflowLabels, runnerLabels, true)).toBe(true);
+    });
+
+    it('should match when labels are exactly identical with exactMatch=true.', () => {
+      const workflowLabels = ['self-hosted', 'linux', 'on-demand'];
+      const runnerLabels = [['self-hosted', 'linux', 'on-demand']];
+      expect(canRunJob(workflowLabels, runnerLabels, true)).toBe(true);
+    });
+
+    it('should match with exactMatch=true when labels are in different order.', () => {
+      const workflowLabels = ['linux', 'self-hosted', 'x64'];
+      const runnerLabels = [['self-hosted', 'linux', 'x64']];
+      expect(canRunJob(workflowLabels, runnerLabels, true)).toBe(true);
+    });
+
+    it('should match with exactMatch=true when labels are completely shuffled.', () => {
+      const workflowLabels = ['x64', 'ubuntu-latest', 'self-hosted', 'linux'];
+      const runnerLabels = [['self-hosted', 'linux', 'x64', 'ubuntu-latest']];
+      expect(canRunJob(workflowLabels, runnerLabels, true)).toBe(true);
+    });
+
+    it('should match with exactMatch=false when labels are in different order.', () => {
+      const workflowLabels = ['gpu', 'self-hosted'];
+      const runnerLabels = [['self-hosted', 'gpu']];
+      expect(canRunJob(workflowLabels, runnerLabels, false)).toBe(true);
+    });
+
+    // bidirectionalLabelMatch tests
+    it('should NOT match when runner has more labels than workflow requests (bidirectionalLabelMatch=true).', () => {
+      const workflowLabels = ['self-hosted', 'linux', 'x64', 'staging', 'ubuntu-2404'];
+      const runnerLabels = [['self-hosted', 'linux', 'x64', 'staging', 'ubuntu-2404', 'on-demand']];
+      expect(canRunJob(workflowLabels, runnerLabels, false, true)).toBe(false);
+    });
+
+    it('should NOT match when workflow has more labels than runner (bidirectionalLabelMatch=true).', () => {
+      const workflowLabels = ['self-hosted', 'linux', 'x64', 'ubuntu-latest', 'gpu'];
+      const runnerLabels = [['self-hosted', 'linux', 'x64']];
+      expect(canRunJob(workflowLabels, runnerLabels, false, true)).toBe(false);
+    });
+
+    it('should match when labels are exactly identical with bidirectionalLabelMatch=true.', () => {
+      const workflowLabels = ['self-hosted', 'linux', 'on-demand'];
+      const runnerLabels = [['self-hosted', 'linux', 'on-demand']];
+      expect(canRunJob(workflowLabels, runnerLabels, false, true)).toBe(true);
+    });
+
+    it('should match with bidirectionalLabelMatch=true when labels are in different order.', () => {
+      const workflowLabels = ['linux', 'self-hosted', 'x64'];
+      const runnerLabels = [['self-hosted', 'linux', 'x64']];
+      expect(canRunJob(workflowLabels, runnerLabels, false, true)).toBe(true);
+    });
+
+    it('should match with bidirectionalLabelMatch=true when labels are completely shuffled.', () => {
+      const workflowLabels = ['x64', 'ubuntu-latest', 'self-hosted', 'linux'];
+      const runnerLabels = [['self-hosted', 'linux', 'x64', 'ubuntu-latest']];
+      expect(canRunJob(workflowLabels, runnerLabels, false, true)).toBe(true);
+    });
+
+    it('should match with bidirectionalLabelMatch=true ignoring case.', () => {
+      const workflowLabels = ['Self-Hosted', 'Linux', 'X64'];
+      const runnerLabels = [['self-hosted', 'linux', 'x64']];
+      expect(canRunJob(workflowLabels, runnerLabels, false, true)).toBe(true);
+    });
+
+    it('should NOT match empty workflow labels with bidirectionalLabelMatch=true.', () => {
+      const workflowLabels: string[] = [];
+      const runnerLabels = [['self-hosted', 'linux', 'x64']];
+      expect(canRunJob(workflowLabels, runnerLabels, false, true)).toBe(false);
+    });
+
+    it('bidirectionalLabelMatch takes precedence over exactMatch when both are true.', () => {
+      const workflowLabels = ['self-hosted', 'linux', 'x64'];
+      const runnerLabels = [['self-hosted', 'linux', 'x64', 'ubuntu-latest']];
+      // exactMatch alone would accept this (runner has extra labels), but bidirectional should reject
+      expect(canRunJob(workflowLabels, runnerLabels, true, true)).toBe(false);
+    });
   });
 });
 

lambdas/functions/webhook/src/runners/dispatch.ts

@@ -42,12 +42,21 @@ async function handleWorkflowJob(
       `Job ID: ${body.workflow_job.id}, Job Name: ${body.workflow_job.name}, ` +
       `Run ID: ${body.workflow_job.run_id}, Labels: ${JSON.stringify(body.workflow_job.labels)}`,
   );
-  // sort the queuesConfig by order of matcher config exact match, with all true matches lined up ahead.
+  // sort the queuesConfig by order of matcher config exact/bidirectional match, with all true matches lined up ahead.
   matcherConfig.sort((a, b) => {
-    return a.matcherConfig.exactMatch === b.matcherConfig.exactMatch ? 0 : a.matcherConfig.exactMatch ? -1 : 1;
+    const aStrict = a.matcherConfig.bidirectionalLabelMatch || a.matcherConfig.exactMatch;
+    const bStrict = b.matcherConfig.bidirectionalLabelMatch || b.matcherConfig.exactMatch;
+    return aStrict === bStrict ? 0 : aStrict ? -1 : 1;
   });
   for (const queue of matcherConfig) {
-    if (canRunJob(body.workflow_job.labels, queue.matcherConfig.labelMatchers, queue.matcherConfig.exactMatch)) {
+    if (
+      canRunJob(
+        body.workflow_job.labels,
+        queue.matcherConfig.labelMatchers,
+        queue.matcherConfig.exactMatch,
+        queue.matcherConfig.bidirectionalLabelMatch,
+      )
+    ) {
       await sendActionRequest({
         id: body.workflow_job.id,
         repositoryName: body.repository.name,
@@ -80,14 +89,24 @@ export function canRunJob(
   workflowJobLabels: string[],
   runnerLabelsMatchers: string[][],
   workflowLabelCheckAll: boolean,
+  bidirectionalLabelMatch = false,
 ): boolean {
   runnerLabelsMatchers = runnerLabelsMatchers.map((runnerLabel) => {
     return runnerLabel.map((label) => label.toLowerCase());
   });
-  const matchLabels = workflowLabelCheckAll
-    ? runnerLabelsMatchers.some((rl) => workflowJobLabels.every((wl) => rl.includes(wl.toLowerCase())))
-    : runnerLabelsMatchers.some((rl) => workflowJobLabels.some((wl) => rl.includes(wl.toLowerCase())));
-  const match = workflowJobLabels.length === 0 ? !matchLabels : matchLabels;
+
+  let match: boolean;
+  if (bidirectionalLabelMatch) {
+    const workflowLabelsLower = workflowJobLabels.map((wl) => wl.toLowerCase());
+    match = runnerLabelsMatchers.some(
+      (rl) => workflowLabelsLower.every((wl) => rl.includes(wl)) && rl.every((r) => workflowLabelsLower.includes(r)),
+    );
+  } else {
+    const matchLabels = workflowLabelCheckAll
+      ? runnerLabelsMatchers.some((rl) => workflowJobLabels.every((wl) => rl.includes(wl.toLowerCase())))
+      : runnerLabelsMatchers.some((rl) => workflowJobLabels.some((wl) => rl.includes(wl.toLowerCase())));
+    match = workflowJobLabels.length === 0 ? !matchLabels : matchLabels;
+  }
 
   logger.debug(
     `Received workflow job event with labels: '${JSON.stringify(workflowJobLabels)}'. The event does ${

lambdas/functions/webhook/src/sqs/index.ts

@@ -17,6 +17,7 @@ export interface ActionRequestMessage {
 export interface MatcherConfig {
   labelMatchers: string[][];
   exactMatch: boolean;
+  bidirectionalLabelMatch?: boolean;
 }
 
 export type RunnerConfig = RunnerMatcherConfig[];

main.tf

@@ -114,6 +114,7 @@ module "webhook" {
       matcherConfig : {
         labelMatchers : [local.runner_labels]
         exactMatch : var.enable_runner_workflow_job_labels_check_all
+        bidirectionalLabelMatch : var.enable_runner_bidirectional_label_match
       }
     }
   }

modules/multi-runner/README.md

@@ -181,9 +181,10 @@ variable "multi_runner_config" {
       }), {})
     })
     matcherConfig = object({
-      labelMatchers = list(list(string))
-      exactMatch    = optional(bool, false)
-      priority      = optional(number, 999)
+      labelMatchers            = list(list(string))
+      exactMatch               = optional(bool, false)
+      bidirectionalLabelMatch  = optional(bool, false)
+      priority                 = optional(number, 999)
     })
     redrive_build_queue = optional(object({
       enabled         = bool
@@ -252,7 +253,8 @@ variable "multi_runner_config" {
       }
       matcherConfig: {
         labelMatchers: "The list of list of labels supported by the runner configuration. `[[self-hosted, linux, x64, example]]`"
-        exactMatch: "If set to true all labels in the workflow job must match the GitHub labels (os, architecture and `self-hosted`). When false if __any__ workflow label matches it will trigger the webhook."
+        exactMatch: "DEPRECATED: Use `bidirectionalLabelMatch` instead. If set to true all labels in the workflow job must match the GitHub labels (os, architecture and `self-hosted`). When false if __any__ workflow label matches it will trigger the webhook. Note: this only checks that workflow labels are a subset of runner labels, not the reverse."
+        bidirectionalLabelMatch: "If set to true, the runner labels and workflow job labels must be an exact two-way match (same set, any order, no extras or missing labels). This is stricter than `exactMatch` which only checks that workflow labels are a subset of runner labels. When false, if __any__ workflow label matches it will trigger the webhook."
         priority: "If set it defines the priority of the matcher, the matcher with the lowest priority will be evaluated first. Default is 999, allowed values 0-999."
       }
       redrive_build_queue: "Set options to attach (optional) a dead letter queue to the build queue, the queue between the webhook and the scale up lambda. You have the following options. 1. Disable by setting `enabled` to false. 2. Enable by setting `enabled` to `true`, `maxReceiveCount` to a number of max retries."

modules/multi-runner/variables.tf

@@ -87,7 +87,7 @@ yarn run dist
 | <a name="input_repository_white_list"></a> [repository\_white\_list](#input\_repository\_white\_list) | List of github repository full names (owner/repo\_name) that will be allowed to use the github app. Leave empty for no filtering. | `list(string)` | `[]` | no |
 | <a name="input_role_path"></a> [role\_path](#input\_role\_path) | The path that will be added to the role; if not set, the environment name will be used. | `string` | `null` | no |
 | <a name="input_role_permissions_boundary"></a> [role\_permissions\_boundary](#input\_role\_permissions\_boundary) | Permissions boundary that will be added to the created role for the lambda. | `string` | `null` | no |
-| <a name="input_runner_matcher_config"></a> [runner\_matcher\_config](#input\_runner\_matcher\_config) | SQS queue to publish accepted build events based on the runner type. When exact match is disabled the webhook accepts the event if one of the workflow job labels is part of the matcher. The priority defines the order the matchers are applied. | <pre>map(object({<br/>    arn = string<br/>    id  = string<br/>    matcherConfig = object({<br/>      labelMatchers = list(list(string))<br/>      exactMatch    = bool<br/>      priority      = optional(number, 999)<br/>    })<br/>  }))</pre> | n/a | yes |
+| <a name="input_runner_matcher_config"></a> [runner\_matcher\_config](#input\_runner\_matcher\_config) | SQS queue to publish accepted build events based on the runner type. When exact match is disabled the webhook accepts the event if one of the workflow job labels is part of the matcher. The priority defines the order the matchers are applied. | <pre>map(object({<br/>    arn = string<br/>    id  = string<br/>    matcherConfig = object({<br/>      labelMatchers            = list(list(string))<br/>      exactMatch               = bool<br/>      bidirectionalLabelMatch  = optional(bool, false)<br/>      priority                 = optional(number, 999)<br/>    })<br/>  }))</pre> | n/a | yes |
 | <a name="input_ssm_paths"></a> [ssm\_paths](#input\_ssm\_paths) | The root path used in SSM to store configuration and secrets. | <pre>object({<br/>    root    = string<br/>    webhook = string<br/>  })</pre> | n/a | yes |
 | <a name="input_tags"></a> [tags](#input\_tags) | Map of tags that will be added to created resources. By default resources will be tagged with name and environment. | `map(string)` | `{}` | no |
 | <a name="input_tracing_config"></a> [tracing\_config](#input\_tracing\_config) | Configuration for lambda tracing. | <pre>object({<br/>    mode                  = optional(string, null)<br/>    capture_http_requests = optional(bool, false)<br/>    capture_error         = optional(bool, false)<br/>  })</pre> | `{}` | no |