Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 40 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<p align="center">
<img src="https://app.qamax.co/static/img/quality-max-logo.png" alt="QualityMax" width="200" />
<img src="https://app.qualitymax.io/static/img/qualitymax-logo-color.png" alt="QualityMax" width="200" />
</p>

<h1 align="center">QualityMax Test Runner</h1>
Expand All @@ -9,7 +9,7 @@
</p>

<p align="center">
<a href="https://github.com/marketplace/actions/qualitymax-test-runner"><img src="https://img.shields.io/badge/GitHub%20Marketplace-QualityMax-blue?logo=github" alt="GitHub Marketplace" /></a>
<a href="https://github.com/marketplace/actions/qualitymax-e2e-tests"><img src="https://img.shields.io/badge/GitHub%20Marketplace-QualityMax-blue?logo=github" alt="GitHub Marketplace" /></a>
<a href="https://github.com/Quality-Max/qualitymax-github-action/releases"><img src="https://img.shields.io/github/v/release/Quality-Max/qualitymax-github-action?label=version" alt="Version" /></a>
<a href="https://github.com/Quality-Max/qualitymax-github-action/blob/main/LICENSE"><img src="https://img.shields.io/github/license/Quality-Max/qualitymax-github-action" alt="License" /></a>
</p>
Expand Down Expand Up @@ -85,7 +85,8 @@ If your GitHub repository is linked to a QualityMax project, omit both — the a
- **PR Comments** — Automatic test result summaries on pull requests
- **Fast Feedback** — Results in minutes, not hours
- **Auto-Retry** — Flaky test detection and automatic retries
- **Artifacts** — Screenshots and videos for failed tests
- **Seed Mode** — Bootstrap tests via AI discovery directly from CI
- **Local Execution** — Run tests in the GitHub runner when configured

## Inputs

Expand All @@ -102,6 +103,10 @@ If your GitHub repository is linked to a QualityMax project, omit both — the a
| `timeout-minutes` | Maximum execution time | No | `30` |
| `fail-on-test-failure` | Fail workflow if tests fail | No | `true` |
| `post-pr-comment` | Post results as PR comment | No | `true` |
| `mode` | Action mode: `run` (execute tests) or `seed` (bootstrap tests via AI) | No | `run` |
| `auto-discover` | Auto-discover test scenarios in seed mode | No | `true` |
| `max-seed-tests` | Maximum tests to generate in seed mode (1-10) | No | `3` |
| `seed-descriptions` | Newline-separated test descriptions for seed mode | No | — |

> **Note:** Either `project-id`, `project-name`, or a linked repository is required. If none are provided, the action attempts auto-detection from the repository URL.

Expand All @@ -117,6 +122,9 @@ If your GitHub repository is linked to a QualityMax project, omit both — the a
| `duration-seconds` | Total execution time |
| `report-url` | URL to full test report |
| `summary-markdown` | Pre-formatted markdown summary |
| `tests-created` | Tests created (seed mode only) |
| `tests-skipped` | Tests skipped (seed mode only) |
| `seed-message` | Summary message (seed mode only) |

## Permissions

Expand Down Expand Up @@ -201,6 +209,27 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
```

### Bootstrap Tests with Seed Mode

```yaml
name: Seed Tests
on: workflow_dispatch

jobs:
seed:
runs-on: ubuntu-latest
steps:
- uses: Quality-Max/qualitymax-github-action@v1
with:
api-key: ${{ secrets.QUALITYMAX_API_KEY }}
project-name: 'My Web App'
mode: 'seed'
max-seed-tests: '5'
base-url: 'https://staging.example.com'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
```

### Run Specific Tests

```yaml
Expand Down Expand Up @@ -233,30 +262,30 @@ Generate a complete workflow file from the QualityMax UI:

## Getting Your API Key

1. Go to [app.qamax.co](https://app.qamax.co)
2. Navigate to **Settings** **API Keys**
1. Go to [app.qualitymax.io](https://app.qualitymax.io)
2. Navigate to **Settings** > **API Keys**
3. Click **Generate API Key**
4. Copy the key (starts with `qm_`)
5. Add it as a secret in your repository: **Settings** **Secrets** **QUALITYMAX_API_KEY**
5. Add it as a secret in your repository: **Settings** > **Secrets** > **QUALITYMAX_API_KEY**

## PR Comment Example

When tests complete, a comment is automatically posted to your PR:

---

## 🧪 QualityMax Test Results
## QualityMax Test Results

| Status | Tests | Duration |
|--------|-------|----------|
| Passed | 12/12 | 2m 34s |
| Passed | 12/12 | 2m 34s |

### Summary
- **Browser:** Chromium
- **Base URL:** https://staging.example.com
- **Commit:** `abc1234`

[View Full Report](https://app.qamax.co/results/gha_xyz789)
[View Full Report](https://app.qualitymax.io/results/gha_xyz789)

---

Expand Down Expand Up @@ -300,8 +329,8 @@ env:

## Support

- [Documentation](https://qamax.co)
- [Email Support](mailto:contact@qamax.co)
- [Documentation](https://qualitymax.io)
- [Email Support](mailto:contact@qualitymax.io)
- [Report Issues](https://github.com/Quality-Max/qualitymax-github-action/issues)

## License
Expand Down
2 changes: 1 addition & 1 deletion action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ branding:

inputs:
api-key:
description: 'QualityMax API key (get it from app.qamax.co/settings/api)'
description: 'QualityMax API key (get it from app.qualitymax.io/settings/api)'
required: true
project-id:
description: 'QualityMax project ID (optional if project-name provided or repo auto-detected)'
Expand Down
4 changes: 4 additions & 0 deletions dist/api.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,10 @@ export declare class QualityMaxClient {
* Get execution results
*/
getResults(executionId: string): Promise<TestExecutionResults>;
/**
* Report bulk execution results back to the API
*/
reportResults(executionId: string, result: 'passed' | 'failed', passedTests: number, failedTests: number, totalTests: number): Promise<void>;
/**
* Cancel execution
*/
Expand Down
2 changes: 1 addition & 1 deletion dist/api.d.ts.map

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion dist/index.d.ts.map

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 4 additions & 4 deletions dist/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -29967,7 +29967,7 @@ Object.defineProperty(exports, "__esModule", ({ value: true }));
exports.QualityMaxClient = void 0;
const core = __importStar(__nccwpck_require__(7484));
const http_client_1 = __nccwpck_require__(4844);
const API_BASE_URL = process.env.QUALITYMAX_API_URL || 'https://app.qamax.co/api';
const API_BASE_URL = process.env.QUALITYMAX_API_URL || 'https://app.qualitymax.io/api';
const POLL_INTERVAL_MS = 5000; // 5 seconds
class QualityMaxClient {
client;
Expand Down Expand Up @@ -30507,7 +30507,7 @@ module.exports = defineConfig({
started_at: new Date(startTime).toISOString(),
completed_at: new Date().toISOString(),
browser: inputs.browser,
report_url: `https://app.qamax.co/results/${executionId}`,
report_url: `https://app.qualitymax.io/results/${executionId}`,
tests: testResults,
};
return results;
Expand Down Expand Up @@ -30554,7 +30554,7 @@ async function run() {
core.info('Validating API key...');
const isValid = await client.validateApiKey();
if (!isValid) {
throw new Error('Invalid API key. Get your API key from app.qamax.co/settings/api');
throw new Error('Invalid API key. Get your API key from app.qualitymax.io/settings/api');
}
core.info('API key validated ✓');
// Resolve project ID
Expand Down Expand Up @@ -30675,7 +30675,7 @@ async function run() {
started_at: new Date(startTime).toISOString(),
completed_at: new Date().toISOString(),
browser: inputs.browser,
report_url: `https://app.qamax.co/results/${executionId}`,
report_url: `https://app.qualitymax.io/results/${executionId}`,
tests: [],
};
}
Expand Down
2 changes: 1 addition & 1 deletion dist/index.js.map

Large diffs are not rendered by default.

10 changes: 10 additions & 0 deletions dist/types.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,16 @@ export interface TriggerTestsResponse {
estimated_duration_seconds?: number;
status_url: string;
cancel_url: string;
run_locally?: boolean;
test_files?: string[];
test_command?: string;
scripts?: EmbeddedScript[];
}
export interface EmbeddedScript {
id: number;
name: string;
code: string;
test_case_id?: number;
}
export type TestRunStatus = 'queued' | 'running' | 'completed' | 'failed' | 'cancelled' | 'timeout';
export type TestResult = 'passed' | 'failed' | 'skipped' | 'error';
Expand Down
2 changes: 1 addition & 1 deletion dist/types.d.ts.map

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "qualitymax-github-action",
"version": "1.1.0",
"version": "1.2.0",
"description": "Run AI-powered E2E tests in your CI/CD pipeline with QualityMax",
"main": "dist/index.js",
"scripts": {
Expand All @@ -19,7 +19,7 @@
"automation",
"qualitymax"
],
"author": "QualityMax <contact@qamax.co>",
"author": "QualityMax <contact@qualitymax.io>",
"license": "MIT",
"dependencies": {
"@actions/core": "^1.10.1",
Expand All @@ -45,5 +45,5 @@
"bugs": {
"url": "https://github.com/Quality-Max/qualitymax-github-action/issues"
},
"homepage": "https://qamax.co"
"homepage": "https://qualitymax.io"
}
2 changes: 1 addition & 1 deletion src/api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import {
SeedTestsResponse,
} from './types';

const API_BASE_URL = process.env.QUALITYMAX_API_URL || 'https://app.qamax.co/api';
const API_BASE_URL = process.env.QUALITYMAX_API_URL || 'https://app.qualitymax.io/api';
const POLL_INTERVAL_MS = 5000; // 5 seconds

export class QualityMaxClient {
Expand Down
6 changes: 3 additions & 3 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
TriggerTestsResponse,
TestExecutionResults,
EmbeddedScript,
SeedTestsResponse,

Check failure on line 22 in src/index.ts

View workflow job for this annotation

GitHub Actions / lint

'SeedTestsResponse' is defined but never used
} from './types';

/**
Expand Down Expand Up @@ -259,7 +259,7 @@
// Parse results from JSON reporter
let passedTests = 0;
let failedTests = 0;
let totalTests = scripts.length;

Check failure on line 262 in src/index.ts

View workflow job for this annotation

GitHub Actions / lint

'totalTests' is never reassigned. Use 'const' instead
let skippedTests = 0;
const testResults: TestExecutionResults['tests'] = [];

Expand Down Expand Up @@ -340,7 +340,7 @@
started_at: new Date(startTime).toISOString(),
completed_at: new Date().toISOString(),
browser: inputs.browser,
report_url: `https://app.qamax.co/results/${executionId}`,
report_url: `https://app.qualitymax.io/results/${executionId}`,
tests: testResults,
};

Expand Down Expand Up @@ -396,7 +396,7 @@
const isValid = await client.validateApiKey();
if (!isValid) {
throw new Error(
'Invalid API key. Get your API key from app.qamax.co/settings/api'
'Invalid API key. Get your API key from app.qualitymax.io/settings/api'
);
}
core.info('API key validated ✓');
Expand Down Expand Up @@ -537,7 +537,7 @@
started_at: new Date(startTime).toISOString(),
completed_at: new Date().toISOString(),
browser: inputs.browser,
report_url: `https://app.qamax.co/results/${executionId}`,
report_url: `https://app.qualitymax.io/results/${executionId}`,
tests: [],
};
} else {
Expand Down
Loading