Build: Dockerfile bug fixes, clean ups, and an integration test

TLDR:

* Fix all the headers for correctness (mime + charset), performance, and security.
* Use much simpler redirect logic and apply the same approach as we use today.
* Verify it all with an integration test.

== Differences in file headers

In summary:

* Missing GZIP support and no "Vary: Accept-Encoding" to avoid cache poisoning.
* Missing CORS support (Access-Control-Allow-Origin).
* Missing caching allowance (Cache-Control public 10 years, max Expires).
* Missing disablement of server_tokens.
* Missing charset utf-8.
* Unstable E-Tag and Last-Modified which would cause serious cache churn
  after every rebuild.

Status quo:

```
krinkle@wp-03:~$ curl -H 'Host: codeorigin.jquery.com' -i 'http://localhost/jquery-3.0.0.js' | head -n20

Server: nginx
Date: Mon, 16 Aug 2021 23:12:52 GMT
Connection: keep-alive
Access-Control-Allow-Origin: *
Cache-Control: max-age=315360000
Cache-Control: public
Content-Length: 263268
Content-Type: application/javascript; charset=utf-8
ETag: "28feccc0-40464"
Expires: Thu, 31 Dec 2037 23:55:55 GMT
Last-Modified: Fri, 18 Oct 1991 12:00:00 GMT
Vary: Accept-Encoding
Accept-Ranges: bytes
```

Previously, without this patch:

```
dockerize$ curl -i 'http://localhost:4000/jquery-3.0.0.js' | head -n20

Server: nginx/1.21.1
Date: Mon, 16 Aug 2021 23:13:30 GMT
Connection: keep-alive
Content-Length: 263268
Content-Type: application/javascript
ETag: "5fdbec22-40464"
Last-Modified: Thu, 17 Dec 2020 23:39:14 GMT
Accept-Ranges: bytes
```

== Match existing Nginx config

Most of the above was addressed by simply using the same Nginx config as we have
in the private infrastructure.git repo for provisioning legacy codeorigin
(copied from wordpress-header.conf.erb and wp/jquery.pp, respectively).

However, the last two points (charset, and unstable etag), require additional
changes.

== Nginx: charset

The default charset from Nginx differs between Debian and Alpine, possibly
due to changes in some of the shared libraries. Fix this by explicitly setting
`charset utf-8`. Note that this does (and should) only apply to files with a
MIME type in `charset_types` list, which by default contains HTML, JS, and CSS.

More importantly, it does not (and should not) cause PNG files and other binary
assets to be served with a charset, which would be wrong.

== Nginx: E-Tag and Last-Modified

This was a tricky one. Git does not store file modification timestamps, which
means the on-disk file mtimes are somewhat arbitrarily set to when the file
was created by the local Git client. For most files, this means the time you
cloned the repository, and for files added later, the time you ran `git pull`.

This was de-facto "stable" on the legacy server because it is a persistent server
with a persistent git directory (we always use the same clone and just update it
to fetch new files after a commit happens). The timestamps themselves don't matter
as long as they remain constant for any given file, and they were.

This is important because E-Tag is computed in Nginx based on file mtime and
size and (unlike Apache) this behaviour is not configurable in Nginx (e.g. to
make it compute a content hash instead). And Last-Modified naturally uses the
mtime as well.

As a workaround, I've added a `touch` command to assign all CDN files a fixed
timestamp far in the past. This is safe for us because all files are expected
to remain static. Even if we would change a file, it wouldn't propagate to the
CDN without a manual CDN API purge (given the all-important high values for
Cache-Control max-age and Expires), and after that would not propagate to
any browser that has previously fetched it unless the user clears their own
cache (given the all-important Cache-Control telling the browser to re-use the
file blindly, which is critical for performance). As such, whether the timestamp
remains constant or changes after the file is changed, would make no difference
for how it propagates.

== New state

After, with this patch (and verified by a simple test suite).

```
dockerize$ curl -i 'http://localhost:4000/jquery-3.0.0.js' | head -n20

Server: nginx
Date: Mon, 16 Aug 2021 23:49:43 GMT
Connection: keep-alive
Access-Control-Allow-Origin: *
Cache-Control: max-age=315360000
Cache-Control: public
Content-Length: 263268
Content-Type: application/javascript; charset=utf-8
ETag: "28feccc0-40464"
Expires: Thu, 31 Dec 2037 23:55:55 GMT
Last-Modified: Fri, 18 Oct 1991 12:00:00 GMT
Vary: Accept-Encoding
Accept-Ranges: bytes
```

== Misc changes

* Remove left-over Nginx config for purge.php.

* Remove left-over php-fpm script.

* Use our own document root at /var/www/cdn instead of mixing our
  files into the default /usr/share/nginx/html/ which contains files
  from the "docker-nginx" project that we may not want to serve blindly.
  This mainly affects the odd index.html file that could otherwise be
  served.

* Pin the Nginx base image version at 1.x. This is mostly a no-op
  since they've been on 1.x for 10 years, but if it does rollover
  we wouldn't want an unmanaged upgrade to a new major version.
  We could pun this more narrowly at some point, but 1.x should be
  fine for now.

* Invert the config file (let Dockerfile remove code instead of
  comment out) so that the most critical lines can be easily
  linted.

* Ensure all files listed are redirected correctly,
  as per https://github.com/jquery/infrastructure/issues/474#issuecomment-844582865
  and verify this with an integration test. The expected values
  are populated based on running `curl` against localhost from a shell
  on the legacy codeorigin. I specifically did not compare against
  live code.jquery.com, since we care about matching what we feed to
  the CDN (although the above problems were surfacing through there
  as well). Some of the CDN behaviours can't and don't need to be
  mimicked from the origin. Prior to launch we will (also) compare the
  live code.jquery.com against the new staging CDN endpoint, but that's
  separate from the test for the container.

Ref https://github.com/jquery/infrastructure/issues/474.

Author: Krinkle

Dockerfile

@@ -1,8 +1,39 @@
-FROM nginx:alpine
+# Allow patch updates
+# https://hub.docker.com/_/nginx/
+FROM nginx:1-alpine
 
-# Install pre-reqs, since we're doing everything in one container for minimum complexity
 RUN apk add vim openrc
 
+# Add CDN assets to the container
+#
+# Avoid wildcard as that would flatten the directory structure.
+#
+# This is an early step in the Dockerfile, so that local docker builds
+# can reuse this layer and perform a fast rebuild when you only tweaked
+# later stuff (nginx config, args, etc.)
+COPY cdn/ /var/www/cdn/
+
+# Set a constant file modification timestamp for all CDN assets
+#
+# Git does not store modified file timestamps, which means the on-disk mtime
+# for most files is set to the time of the Git clone. This is a problem
+# since the container-based server will be re-created after each commit,
+# and we do not want the public "Last-Modified" and "E-Tag" header information
+# to roll over after every rebuild.
+#
+# While Apache has an option to configure how E-Tag is computed (e.g. based on
+# content only), Nginx is always based on file mtime and file size.
+#
+# As a workaround, set a fixed timestamp for all CDN files. This is okay as
+# we don't actually utilized Last-Modified or E-Tag for propagating changes,
+# we only use them as a way to re-assure browsers that files haven't changed
+# and thus reduce bandwidth from needless re-transfers. Given our maximum
+# Cache-Control "max-age", it is already the case that a changed file will not
+# be seen by the CDN unless we purge it via the CDN API, and not seen by previous
+# browser clients until they clear their own caches.
+#
+RUN find /var/www/cdn/ -type f -print0 | TZ=UTC xargs -0 -P 4 -n 50 touch --date='1991-10-18 12:00:00' {} +
+
 # Define the environment variable that will be used in the origin pull magic header
 ARG CDN_ACCESS_KEY=''
 
@@ -11,21 +42,16 @@ COPY cfg/vimrc /etc/vim/vimrc
 COPY cfg/default.conf /etc/nginx/conf.d/default.conf
 
 # If the CDN_ACCESS_KEY environment variable is *not* set, operate in "break glass" mode where the
-# container responds to all requests. Otherwise, look for the secret header the CDN adds to origin
-# pulls and only allow responses to those requests, and 301 the rest back to the CDN.
+# container serves files without restriction. Otherwise, it responds with redirects to requests
+# that don't carry an x-cdn-access request header with the correct key.
 #
-# Note: We're writing directly to the config files because nginx does not currently have a way to
-# access environment variables without significant workarounds. Furthermore, the variables are
-# required because nginx does not currently support nested if statements.
+# Note: We're substituting the config file because nginx does not currently have a way to
+# access environment variables without significant workarounds.
 RUN if [ -n "$CDN_ACCESS_KEY" ]; then \
-    sed -i s/CDN_ACCESS_KEY_PLACEHOLDER/$CDN_ACCESS_KEY/g /etc/nginx/conf.d/default.conf && \
-    sed -i s/##ACTIVATE-XCDNACCESS##//g /etc/nginx/conf.d/default.conf; \
+    sed -i s/CDN_ACCESS_KEY_PLACEHOLDER/$CDN_ACCESS_KEY/g /etc/nginx/conf.d/default.conf; \
+  else \
+    sed -i s/^.*REQUIRE_XCDNACCESS$//g /etc/nginx/conf.d/default.conf; \
   fi
 
-# Load the releases into the container
-#
-# Avoid wildcard as that would flatten the directory structure.
-COPY cdn/ /usr/share/nginx/html/
-
 EXPOSE 80
 

README.md

@@ -1,55 +1,76 @@
-# Official project releases
+codeorigin.jquery.com
 =====================
 
-This repo is used to build a Docker container that serves the codeorigin site for jQuery and related projects. It is designed to deploy easily, and includes a "break glass in case of emergency" minimal config mode should codeorigin need to be redeployed urgently.
+## Add new assets
 
-It also contains the files necessary to build and deploy releases.jquery.com on a separate host, which provides an index of the files on codeorigin.
+To publish a new release, project maintainers should commit new assets to the `cdn/` directory and push to the `main` branch. The jQuery CDN (code.jquery.com) and releases site (releases.jquery.com) will both automatically rebuild.
 
-## Build a local copy of codeorigin
+-------
 
-### Default, no restrictions (development or emergency mode)
+## jQuery CDN
 
-To build a local container (defaults to "break glass" mode):
+The jQuery CDN assets are served from a static file server (Nginx), provisioned through Docker.
 
-1. Install Docker
-1. Clone this repo, and `cd` into it
-1. Build the image: `docker build -t releases ./`
-1. Run the container, exposing port 80: `docker run -p 127.0.0.1:80:80/tcp releases`
-1. To exit the container, press `ctrl+c`
+Prerequisites:
+* Docker (see [Docker CE for Linux](https://docs.docker.com/install/#server), [Docker for Mac](https://hub.docker.com/editions/community/docker-ce-desktop-mac), or [Docker for Windows](https://docs.docker.com/docker-for-windows/install/)).
+* Git.
 
-### Redirect non-origin pulls to CDN (production mode)
+### Local development
 
-To build a local container in deployment mode (redirecting any requests without the magic header that indicates an origin pull), build the container with the header value in an environment variable:
+**Test the container**:
 
-1. Install Docker
-1. Clone this repo, and `cd` into it
-1. Generate a random string for the environment variable: ``CDN_ACCESS_KEY=`openssl rand -hex 32` ``
-1. Build the image: `docker build -t prod-releases --build-arg CDN_ACCESS_KEY=$CDN_ACCESS_KEY ./`
-1. Run the container, exposing port 80: `docker run -p 127.0.0.1:80:80/tcp prod-releases`
-1. To exit the container, press `ctrl+c`
+1. Build the image, by running in a clone of this repo:
+   `docker build -t codeorigin-dev ./`
+1. Start a container, exposing port 80:
+   `docker run --rm -p 4000:80/tcp codeorigin-dev`
+1. The site is now running at <http://localhost:4000/jquery-3.0.0.js>.
+   To stop the container, press `ctrl+c`
 
-Note that you will need to keep track of `$CDN_ACCESS_KEY` and add it to the headers sent for origin pulls. To test whether this is working correctly, you can use `curl`:
+**Inspect the container**:
 
-* This should always redirect to `code.jquery.com`: `curl -i localhost/jquery-3.1.1.js`
-* This should always deliver a copy of the file (don't forget to set the environment variable in your current shell): `curl -i -H "x-cdn-access: ${CDN_ACCESS_KEY}" localhost/jquery-3.1.1.js`
+Run it with a shell entrypoint and set interactive/tty mode by adding the parameters `-i -t --entrypoint /bin/sh`.
+Note that when inspecting the container, nginx will not be started.
 
-## Build the production site
+```
+docker run --rm -p 4000:80/tcp -i -t --entrypoint /bin/sh codeorigin-dev
+```
 
-To deploy, first generate the CDN access key. Next, you'll need to configure the container host to build from the Dockerfile in this repository, and use the CDN access key as build arguments. Finally, you'll configure the CDN to send both the Host header and the access key during origin pulls.
+**Debug nginx**:
 
-1. Generate the access key: ``CDN_ACCESS_KEY=`openssl rand -hex 16` ``
-1. Configure the container host to build from this repo, and set this build variable:
-  * `CDN_ACCESS_KEY=(Insert the value of $CDN_ACCESS_KEY here)`
-1. Add the magic header and the host header at the CDN for origin pulls: `x-cdn-access: (Insert the value of $CDN_ACCESS_KEY here)|Host: (insert URL to app container)`
+In `cfg/defualt.cong`, change `error_log … crit` to `error_log … debug` and then build+run as usual.
 
-## In case of emergency
+**Test the container in restricted mode**:
 
-If you need to deploy a codeorigin container immediately, or if there are origin pull failures and you're not sure why, deploy the container without configuring the `CDN_ACCESS_KEY` environment variable. The codeorigin server will respond to all requests without redirecting non-origin pulls to the CDN, so this should be only used in case of emergencies.
+There is a restricted mode, which responds to unauthorized static file requests with a redirect instead of serving files.
 
-## Build the releases sites
+1. Run `export CDN_ACCESS_KEY="$(openssl rand -hex 32)"`
+1. Build the image, by running in a clone of this repo:
+   `docker build -t codeorigin-strict --build-arg "CDN_ACCESS_KEY=$CDN_ACCESS_KEY" ./`
+1. Start a container, exposing port 80:
+   `docker run --rm -p 4000:80/tcp codeorigin-strict`
+1. The site is now running at <http://localhost:4000/jquery-3.0.0.js>.
+   To stop the container, press `ctrl+c`
 
-To build and deploy your changes for previewing in a [`jquery-wp-content`](https://github.com/jquery/jquery-wp-content) instance, follow the [workflow instructions](http://contribute.jquery.org/web-sites/#workflow) from our documentation on [contributing to jQuery web sites](http://contribute.jquery.org/web-sites/).
+In the restricted mode:
 
-## Add or update project release files
+* Simple requests for static files redirect to `code.jquery.com/*`, e.g. `curl -i 'http://localhost/jquery-3.0.0.js'`.
+* Requests with the access key will respond by serving the file: `curl -i -H "x-cdn-access: $CDN_ACCESS_KEY" 'http://localhost/jquery-3.0.0.js'`
+* Requests with an invalid key will redirect the same as without: `curl -i -H "x-cdn-access: wrong" 'http://localhost/jquery-3.0.0.js'`
 
-To add a new release or update an existing one, simply commit the new file to the `cdn` directory and merge to the `main` branch. The container will rebuild automatically.
+### Production deployment
+
+1. First, generate a CDN access key: `openssl rand -hex 32`.
+1. At a hosting platform of your choosing, build a container from the Dockerfile in this repository, and pass the CDN access key as build arguments.
+1. Finally, configure the CDN to use the container address as its origin, with special handling to augment origin pulls with a `Host: code.jquery.com` header, and a `x-cdn-access` header with the access key.
+
+### In case of emergency
+
+If you need to deploy a standalone codeorigin site immediately, or if there are origin pull failures and you're not sure why, then deploy the container without any `CDN_ACCESS_KEY` environment variable. The codeorigin container then default to its unrestricted mode (no offload redirects).
+
+-------
+
+## WordPress build
+
+This repository is also used to update the asset catalog at <https://releases.jquery.com/>, which is an auto-generated WordPress site.
+
+To preview changes for a [`jquery-wp-content`](https://github.com/jquery/jquery-wp-content) instance, follow the [workflow instructions](http://contribute.jquery.org/web-sites/#workflow) from our documentation on [contributing to jQuery Foundation web sites](http://contribute.jquery.org/web-sites/).