Skip to content

Commit fc7d48b

Browse files
Merge branch 'main' into robertbrignull/enable-await-thenable
2 parents f69a6c5 + 95f46b3 commit fc7d48b

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

52 files changed

+413
-1151
lines changed

.github/workflows/main.yml

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -132,7 +132,12 @@ jobs:
132132
- name: Run unit tests
133133
working-directory: extensions/ql-vscode
134134
run: |
135-
npm run test
135+
npm run test:unit
136+
137+
- name: Run view tests
138+
working-directory: extensions/ql-vscode
139+
run: |
140+
npm run test:view
136141
137142
test:
138143
name: Test
@@ -173,15 +178,15 @@ jobs:
173178
VSCODE_CODEQL_GITHUB_TOKEN: '${{ secrets.GITHUB_TOKEN }}'
174179
run: |
175180
unset DBUS_SESSION_BUS_ADDRESS
176-
/usr/bin/xvfb-run npm run integration
181+
/usr/bin/xvfb-run npm run test:vscode-integration
177182
178183
- name: Run integration tests (Windows)
179184
if: matrix.os == 'windows-latest'
180185
working-directory: extensions/ql-vscode
181186
env:
182187
VSCODE_CODEQL_GITHUB_TOKEN: '${{ secrets.GITHUB_TOKEN }}'
183188
run: |
184-
npm run integration
189+
npm run test:vscode-integration
185190
186191
set-matrix:
187192
name: Set Matrix for cli-test
@@ -254,10 +259,10 @@ jobs:
254259
if: matrix.os == 'ubuntu-latest'
255260
run: |
256261
unset DBUS_SESSION_BUS_ADDRESS
257-
/usr/bin/xvfb-run npm run cli-integration
262+
/usr/bin/xvfb-run npm run test:cli-integration
258263
259264
- name: Run CLI tests (Windows)
260265
working-directory: extensions/ql-vscode
261266
if: matrix.os == 'windows-latest'
262267
run: |
263-
npm run cli-integration
268+
npm run test:cli-integration

CONTRIBUTING.md

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -98,10 +98,11 @@ We have several types of tests:
9898
* Unit tests: these live in the `tests/unit-tests/` directory
9999
* View tests: these live in `src/view/variant-analysis/__tests__/`
100100
* VSCode integration tests:
101-
* `test/vscode-tests/no-workspace` tests: These are intended to cover functionality that is meant to work before you even have a workspace open.
101+
* `test/vscode-tests/activated-extension` tests: These are intended to cover functionality that require the full extension to be activated but don't require the CLI. This suite is not run against multiple versions of the CLI in CI.
102+
* `test/vscode-tests/no-workspace` tests: These are intended to cover functionality around not having a workspace. The extension is not activated in these tests.
102103
* `test/vscode-tests/minimal-workspace` tests: These are intended to cover functionality that need a workspace but don't require the full extension to be activated.
103104
* CLI integration tests: these live in `test/vscode-tests/cli-integration`
104-
* These tests are intendended to be cover functionality that is related to the integration between the CodeQL CLI and the extension.
105+
* These tests are intended to cover functionality that is related to the integration between the CodeQL CLI and the extension. These tests are run against each supported versions of the CLI in CI.
105106

106107
The CLI integration tests require an instance of the CodeQL CLI to run so they will require some extra setup steps. When adding new tests to our test suite, please be mindful of whether they need to be in the cli-integration folder. If the tests don't depend on the CLI, they are better suited to being a VSCode integration test.
107108

@@ -119,7 +120,7 @@ Then, from the `extensions/ql-vscode` directory, use the appropriate command to
119120

120121
* Unit tests: `npm run test:unit`
121122
* View Tests: `npm test:view`
122-
* VSCode integration tests: `npm run integration`
123+
* VSCode integration tests: `npm run test:vscode-integration`
123124

124125
###### CLI integration tests
125126

@@ -130,7 +131,7 @@ The CLI integration tests require the CodeQL standard libraries in order to run
130131
2. Run your test command:
131132

132133
```shell
133-
cd extensions/ql-vscode && npm run cli-integration
134+
cd extensions/ql-vscode && npm run test:cli-integration
134135
```
135136

136137
##### 2. From VSCode
@@ -161,13 +162,13 @@ The easiest way to run a single test is to change the `it` of the test to `it.on
161162
to only run tests for this specific file. For example, to run the test `test/vscode-tests/cli-integration/run-queries.test.ts`:
162163

163164
```shell
164-
npm run cli-integration -- --runTestsByPath test/vscode-tests/cli-integration/run-queries.test.ts
165+
npm run test:cli-integration -- --runTestsByPath test/vscode-tests/cli-integration/run-queries.test.ts
165166
```
166167

167168
You can also use the `--testNamePattern` option to run a specific test within a file. For example, to run the test `test/vscode-tests/cli-integration/run-queries.test.ts`:
168169

169170
```shell
170-
npm run cli-integration -- --runTestsByPath test/vscode-tests/cli-integration/run-queries.test.ts --testNamePattern "should create a QueryEvaluationInfo"
171+
npm run test:cli-integration -- --runTestsByPath test/vscode-tests/cli-integration/run-queries.test.ts --testNamePattern "should create a QueryEvaluationInfo"
171172
```
172173

173174
##### 2. From VSCode

extensions/ql-vscode/jest.config.js

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ module.exports = {
88
projects: [
99
"<rootDir>/src/view",
1010
"<rootDir>/test/unit-tests",
11+
"<rootDir>/test/vscode-tests/activated-extension",
1112
"<rootDir>/test/vscode-tests/cli-integration",
1213
"<rootDir>/test/vscode-tests/no-workspace",
1314
"<rootDir>/test/vscode-tests/minimal-workspace",

extensions/ql-vscode/package.json

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1326,13 +1326,14 @@
13261326
"scripts": {
13271327
"build": "gulp",
13281328
"watch": "gulp watch",
1329-
"test": "npm-run-all -p test:*",
1329+
"test": "npm-run-all test:*",
13301330
"test:unit": "cross-env TZ=UTC LANG=en-US jest --projects test/unit-tests",
13311331
"test:view": "jest --projects src/view",
1332-
"integration": "npm-run-all integration:*",
1333-
"integration:no-workspace": "jest --projects test/vscode-tests/no-workspace",
1334-
"integration:minimal-workspace": "jest --projects test/vscode-tests/minimal-workspace",
1335-
"cli-integration": "jest --projects test/vscode-tests/cli-integration",
1332+
"test:vscode-integration": "npm-run-all test:vscode-integration:*",
1333+
"test:vscode-integration:activated-extension": "jest --projects test/vscode-tests/activated-extension",
1334+
"test:vscode-integration:no-workspace": "jest --projects test/vscode-tests/no-workspace",
1335+
"test:vscode-integration:minimal-workspace": "jest --projects test/vscode-tests/minimal-workspace",
1336+
"test:cli-integration": "jest --projects test/vscode-tests/cli-integration",
13361337
"update-vscode": "node ./node_modules/vscode/bin/install",
13371338
"format": "prettier --write **/*.{ts,tsx} && eslint . --ext .ts,.tsx --fix",
13381339
"lint": "eslint . --ext .js,.ts,.tsx --max-warnings=0",

extensions/ql-vscode/src/cli.ts

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ import { CompilationMessage } from "./pure/legacy-messages";
2828
import { sarifParser } from "./sarif-parser";
2929
import { dbSchemeToLanguage, walkDirectory } from "./helpers";
3030
import { App } from "./common/app";
31+
import { QueryLanguage } from "./qlpack-generator";
3132

3233
/**
3334
* The version of the SARIF format that we are using.
@@ -1216,6 +1217,23 @@ export class CodeQLCliServer implements Disposable {
12161217
);
12171218
}
12181219

1220+
/**
1221+
* Adds a core language QL library pack for the given query language as a dependency
1222+
* of the current package, and then installs them. This command modifies the qlpack.yml
1223+
* file of the current package. Formatting and comments will be removed.
1224+
* @param dir The directory where QL pack exists.
1225+
* @param language The language of the QL pack.
1226+
*/
1227+
async packAdd(dir: string, queryLanguage: QueryLanguage) {
1228+
const args = ["--dir", dir];
1229+
args.push(`codeql/${queryLanguage}-all`);
1230+
return this.runJsonCodeQlCliCommandWithAuthentication(
1231+
["pack", "add"],
1232+
args,
1233+
`Adding and installing ${queryLanguage} pack dependency.`,
1234+
);
1235+
}
1236+
12191237
/**
12201238
* Downloads a specified pack.
12211239
* @param packs The `<package-scope/name[@version]>` of the packs to download.

extensions/ql-vscode/src/databases.ts

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ import { QueryRunner } from "./queryRunner";
2626
import { pathsEqual } from "./pure/files";
2727
import { redactableError } from "./pure/errors";
2828
import { isCodespacesTemplate } from "./config";
29+
import { QlPackGenerator, QueryLanguage } from "./qlpack-generator";
2930

3031
/**
3132
* databases.ts
@@ -655,9 +656,27 @@ export class DatabaseManager extends DisposableObject {
655656
return;
656657
}
657658

658-
await showBinaryChoiceDialog(
659-
`We've noticed you don't have a QL pack downloaded to analyze this database. Can we set up a ${databaseItem.language} query pack for you`,
659+
const answer = await showBinaryChoiceDialog(
660+
`We've noticed you don't have a CodeQL pack available to analyze this database. Can we set up a query pack for you?`,
660661
);
662+
663+
if (!answer) {
664+
return;
665+
}
666+
667+
try {
668+
const qlPackGenerator = new QlPackGenerator(
669+
folderName,
670+
databaseItem.language as QueryLanguage,
671+
this.cli,
672+
this.ctx.storageUri?.fsPath,
673+
);
674+
await qlPackGenerator.generate();
675+
} catch (e: unknown) {
676+
void this.logger.log(
677+
`Could not create skeleton QL pack: ${getErrorMessage(e)}`,
678+
);
679+
}
661680
}
662681

663682
private async reregisterDatabases(
Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
import { writeFile } from "fs-extra";
2+
import { dump } from "js-yaml";
3+
import { join } from "path";
4+
import { Uri, workspace } from "vscode";
5+
import { CodeQLCliServer } from "./cli";
6+
7+
export type QueryLanguage =
8+
| "csharp"
9+
| "cpp"
10+
| "go"
11+
| "java"
12+
| "javascript"
13+
| "python"
14+
| "ruby"
15+
| "swift";
16+
17+
export class QlPackGenerator {
18+
private readonly qlpackName: string;
19+
private readonly qlpackVersion: string;
20+
private readonly header: string;
21+
private readonly qlpackFileName: string;
22+
private readonly folderUri: Uri;
23+
24+
constructor(
25+
private readonly folderName: string,
26+
private readonly queryLanguage: QueryLanguage,
27+
private readonly cliServer: CodeQLCliServer,
28+
private readonly storagePath: string | undefined,
29+
) {
30+
if (this.storagePath === undefined) {
31+
throw new Error("Workspace storage path is undefined");
32+
}
33+
this.qlpackName = `getting-started/codeql-extra-queries-${this.queryLanguage}`;
34+
this.qlpackVersion = "1.0.0";
35+
this.header = "# This is an automatically generated file.\n\n";
36+
37+
this.qlpackFileName = "qlpack.yml";
38+
this.folderUri = Uri.file(join(this.storagePath, this.folderName));
39+
}
40+
41+
public async generate() {
42+
// create QL pack folder and add to workspace
43+
await this.createWorkspaceFolder();
44+
45+
// create qlpack.yml
46+
await this.createQlPackYaml();
47+
48+
// create example.ql
49+
await this.createExampleQlFile();
50+
51+
// create codeql-pack.lock.yml
52+
await this.createCodeqlPackLockYaml();
53+
}
54+
55+
private async createWorkspaceFolder() {
56+
await workspace.fs.createDirectory(this.folderUri);
57+
58+
const end = (workspace.workspaceFolders || []).length;
59+
60+
await workspace.updateWorkspaceFolders(end, 0, {
61+
name: this.folderName,
62+
uri: this.folderUri,
63+
});
64+
}
65+
66+
private async createQlPackYaml() {
67+
const qlPackFilePath = join(this.folderUri.fsPath, this.qlpackFileName);
68+
69+
const qlPackYml = {
70+
name: this.qlpackName,
71+
version: this.qlpackVersion,
72+
dependencies: {},
73+
};
74+
75+
await writeFile(qlPackFilePath, this.header + dump(qlPackYml), "utf8");
76+
}
77+
78+
private async createExampleQlFile() {
79+
const exampleQlFilePath = join(this.folderUri.fsPath, "example.ql");
80+
81+
const exampleQl = `
82+
/**
83+
* This is an automatically generated file
84+
* @name Empty block
85+
* @kind problem
86+
* @problem.severity warning
87+
* @id ${this.queryLanguage}/example/empty-block
88+
*/
89+
90+
import ${this.queryLanguage}
91+
92+
select "Hello, world!"
93+
`.trim();
94+
95+
await writeFile(exampleQlFilePath, exampleQl, "utf8");
96+
}
97+
98+
private async createCodeqlPackLockYaml() {
99+
await this.cliServer.packAdd(this.folderUri.fsPath, this.queryLanguage);
100+
}
101+
}

extensions/ql-vscode/src/stories/remote-queries/TextButton.stories.tsx renamed to extensions/ql-vscode/src/stories/common/TextButton.stories.tsx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ import * as React from "react";
22

33
import { ComponentStory, ComponentMeta } from "@storybook/react";
44

5-
import TextButtonComponent from "../../view/remote-queries/TextButton";
5+
import TextButtonComponent from "../../view/common/TextButton";
66

77
export default {
88
title: "Text Button",

extensions/ql-vscode/src/stories/remote-queries/data/analysesResultsMessage.json renamed to extensions/ql-vscode/src/stories/data/analysesResultsMessage.json

File renamed without changes.

extensions/ql-vscode/src/stories/remote-queries/data/rawResults.json renamed to extensions/ql-vscode/src/stories/data/rawResults.json

File renamed without changes.

0 commit comments

Comments
 (0)