Skip to content

Commit 117e46b

Browse files
sanasonsanason
committed
Improve the testing README
1 parent 3c76a87 commit 117e46b

1 file changed

Lines changed: 100 additions & 17 deletions

File tree

‎features/README.md‎

Lines changed: 100 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,74 @@
33
The automated tests for the DAP code are implemented using [cucumber-js](https://github.com/cucumber/cucumber-js)
44
and [puppeteer](https://pptr.dev/).
55

6-
By default the tests use the local version of the DAP javascript files. Loading
7-
the code into a test HTML page, performing various user actions, and testing
8-
the behavior of the code in response to the user actions.
6+
```mermaid
7+
---
8+
title: Testing architecture
9+
---
10+
11+
flowchart LR
12+
Runner("`Test Runner<br>*Cucumber.js*`"):::local
13+
Puppeteer("`Browser Automation<br>*Puppeteer*`"):::local
14+
Browser(Chromium):::local
15+
Site("`Test Website<br>*nginx in Docker*`"):::local
16+
config_startup@{ shape: diamond, label: "Configurable at<br>website startup" }
17+
subgraph code [DAP code]
18+
DAP_Local(DAP code - local):::local
19+
DAP_Staging(DAP code - staging):::remote
20+
DAP_Prod(DAP code - production):::remote
21+
end
22+
GA4(DAP GA4 test property):::remote
23+
subgraph Legend
24+
legend_local(Running locally):::local
25+
legend_remote(Running remotely):::remote
26+
end
27+
28+
classDef local fill:#f96
29+
classDef remote fill:#f9f
30+
31+
Runner -->|drives| Puppeteer -->|controls| Browser -->|loads| Site
32+
config_startup --> DAP_Local
33+
config_startup --> DAP_Staging
34+
config_startup --> DAP_Prod
35+
Site-.->|sends events to| GA4
36+
Site -->|loads one of|config_startup
37+
```
38+
39+
## Running the test site
40+
The test site is a simple nginx server running in a Docker container. It serves static HTML pages that include the DAP code, allowing us to test the DAP code's behavior in a realistic environment.
41+
42+
To start up the test site, run one of the `test-site-*` commands:
43+
44+
```bash
45+
npm run test-site-dev
46+
```
47+
This particular command will start up the test site at http://localhost:8080/. All pages will be configured to use your local version of the DAP code by including the script tag:
48+
49+
```angular2html
50+
<script async id="_fed_an_ua_tag" src="Universal-Federated-Analytics-Min.js"></script>
51+
```
52+
53+
You can also choose to run the test site configured to load a deployed version of the DAP code, either the staging or production version, by running `npm run test-site-stg` or `npm run test-site-prd`, respectively.
54+
55+
## Configuring DAP in the test site
56+
The DAP code is configurable at load time via query parameters. For instance, to enable DAP's YouTube tracking, a website
57+
can load the DAP code with the `yt` query parameter set to `true`.
58+
59+
```angular2html
60+
<script async type="text/javascript" id="_fed_an_ua_tag" src="https://dap.digitalgov.gov/Universal-Federated-Analytics-Min.js?agency=HHS&yt=true"></script>
61+
```
962

10-
Use the `DAP_ENV` environment variable to insert live versions of the code into
11-
the test HTML page instead of the local version as described below.
63+
The full list of configurable options is available in the [DAP Code Capabilities Summary](https://github.com/digital-analytics-program/gov-wide-code/wiki/DAP-Code-Capabilities-Summary).
64+
65+
The test site is designed to pass through query parameters to the DAP code, allowing you to test different DAP configurations by changing the URL.
66+
For instance, opening http://localhost:8080/youtube.html?yt=true&search=nasa will load the page for testing DAP's YouTube tracking with YouTube tracking enbabled.
67+
Any query parameters that don't match one of DAP's configuration options will be treated as a normal query parameter (e.g. `search=nasa`).
68+
69+
This capability is used extensively in the automated tests to check the DAP code's behavior in various configurations.
1270

1371
## Running the tests
1472

15-
Start up the test site at http://localhost:8080/:
73+
Start up the test site using any of the `test-site-*` commands:
1674

1775
```bash
1876
npm run test-site-dev
@@ -38,24 +96,49 @@ npm run cucumber:report
3896

3997
Test report should be available in `output/test-results.html`.
4098

41-
## Configuring with environment variables
99+
### Running the tests in verbose mode
42100

43-
### Verbose mode
44-
45-
Print debugging information to stdout while running the tests:
101+
Verbose mode logs events that occur within the browser during test execution. If you find yourself wishing that you could
102+
watch what's happening in browser devtools during test execution, try verbose mode:
46103

47104
```bash
48105
VERBOSE=true npm run cucumber
49106
```
50107

51-
### Run tests against the live staging environment
108+
## Testing approach
109+
Here's a high-level overview of how browser activity is turned into analytics events by DAP and GA4. Google owns everything in this diagram except for the "DAP code" box:
52110

53-
```bash
54-
DAP_ENV=staging npm run cucumber
55-
```
111+
```mermaid
112+
---
113+
title: DAP/GA4 processing
114+
---
56115
57-
### Run tests against the live production environment
116+
flowchart LR
117+
subgraph browser [Inside browser]
118+
dap("`DAP code<br>*listens for browser events*`")
119+
gtag("`gtag code<br>*listens for browser events*`")
120+
end
121+
collect("`Google endpoint<br>*https://www.google-analytics.com/g/collect*`")
122+
data@{ shape: cyl, label: "3. GA4 event store" }
58123
59-
```bash
60-
DAP_ENV=production npm run cucumber
124+
dap-->|1. API call|gtag -->|2. POST|collect-->|insert events|data
61125
```
126+
The basic testing strategy is to insert a test probe at one or more of the numbered points in the diagram and then confirm what the probe
127+
sees once we generate some browser events. Which probe(s) should we use?
128+
1. API call: gtag offers an official, [documented](https://developers.google.com/tag-platform/gtagjs/reference) Javascript API. As such,
129+
we can run tests against a mocked version of the API and, assuming we correctly understand how the API works, have some confidence that passing tests ensure correct behavior.
130+
These tests will be fast and fairly reliable (Puppeteer will always introduce some flakiness).
131+
2. POST: Google Analytics does not treat the `collect` endpoint as a public API. It is not officially documented, both in terms of the structure of the requests to it and
132+
in terms of what triggers those requests. It is true that many developer plugins/extensions have been built to interpret these requests but it would
133+
be risky to build our test suite on top of reverse-engineered knowledge of an unofficial API. Tests would need a plugin of this type that was officially released by Google.
134+
3. GA4 event store: For e2e tests, we could use the Google Analytics Data API to query for GA4 events that we expected to be received during the test run.
135+
136+
The current test suite only uses probe 1. Since the gtag API documentation is actually not very thorough, we should probably add some
137+
integration-type tests that use probes 2 and/or 3 to confirm that our understanding of the API is correct and that the events we expect to be generated are actually being received by Google Analytics.
138+
139+
140+
141+
142+
143+
144+

0 commit comments

Comments
 (0)