[mic][iso] generate PXE-bootable ISO images. (#10595)

Author: gmileka

toolkit/tools/imagecustomizer/docs/cli.md

@@ -106,6 +106,13 @@ existing RPM repo (such as packages.microsoft.com). Using a cloned repo with
 Disable the base image's installed RPM repos as a source of RPMs during package
 installation.
 
+## --output-pxe-artifacts-dir
+
+Create a folder containing the artifacts to be used for PXE booting.
+
+For an overview of Azure Linux Image Customizer support for PXE, see the 
+[PXE support page](./pxe.md).
+
 ## --log-level=LEVEL
 
 Default: `info`

toolkit/tools/imagecustomizer/docs/configuration.md

@@ -72,6 +72,9 @@ The Azure Linux Image Customizer is configured using a YAML (or JSON) file.
 22. If the output format is set to `iso`, copy additional iso media files.
     ([iso](#iso-type))
 
+23. If [--output-pxe-artifacts-dir](./cli.md#output-pxe-artifacts-dir) is specified,
+    then export the ISO image contents to the specified folder.
+
 ### /etc/resolv.conf
 
 The `/etc/resolv.conf` file is overridden during customization so that the package
@@ -151,6 +154,9 @@ os:
         - [permissions](#permissions-string)
     - [kernelCommandLine](#iso-kernelcommandline)
       - [extraCommandLine](#extracommandline-string)
+  - [pxe](#pxe-type)
+    - [isoImageBaseUrl](#isoimagebaseurl-string)
+    - [isoImageFileUrl](#isoimagefileurl-string)
   - [os type](#os-type)
     - [resetBootLoaderType](#resetbootloadertype-string)
     - [hostname](#hostname-string)
@@ -274,7 +280,11 @@ os:
 
 ### iso [[iso](#iso-type)]
 
-Specifies the configuration for the generated ISO media.
+Optionally specifies the configuration for the generated ISO media.
+
+### pxe [[pxe](#pxe-type)]
+
+Optionally specifies the PXE-specific configuration for the generated OS artifacts.
 
 ### os [[os](#os-type)]
 
@@ -316,6 +326,76 @@ Must be a multiple of 1 MiB.
 
 The partitions to provision on the disk.
 
+## pxe type
+
+Specifies the PXE-specific configuration for the generated OS artifacts.
+
+### isoImageBaseUrl [string]
+
+Specifies the base URL for the ISO image to download at boot time. The Azure
+Linux Image Customizer will append the output image name to the specified base
+URL to form the full URL for downloading the image. The output image name is
+specified on the command-line using the `--output-image file` argument (see the
+  [command-line interface](./cli.md) document for more details).
+
+This can be useful if the ISO image name changes with each build and the
+script deploying the artifacts to the PXE server does not update grub.cfg with
+the ISO image name.
+
+For example,
+
+- If the user has the following content in the configuration file:
+
+  ```yaml
+  pxe:
+    isoImageBaseUrl: http://hostname-or-ip/iso-publish-path
+  ```
+
+- and specifies the following on the command line:
+
+  ```bash
+  sudo imagecustomizer \
+    --image-file "./input/azure-linux.vhdx" \
+    --config-file "./input/customization-config.yaml" \
+    --rpm-source "./input/rpms" \
+    --build-dir "./build" \
+    --output-image-format "iso" \
+    --output-image-file "./build/output/output.iso" \
+    --output-pxe-artifacts-dir "./build/output/pxe-artifacts"
+  ```
+
+- then, during PXE booting, the ISO image will be downloaded from:
+
+  ```bash
+  http://hostname-or-ip/iso-publish-path/output.iso
+  ```
+
+This field is mutually exclusive with `isoImageFileUrl`.
+
+For an overview of Azure Linux Image Customizer support for PXE, see the 
+[PXE support page](./pxe.md).
+
+### isoImageFileUrl [string]
+
+Specifies the URL of the ISO image to download at boot time.
+The ISO image must be a LiveOS ISO image generated by the Azure Linux Image
+Customizer. The booting process will pivot to the root file system embedded
+in the ISO image after downloading it.
+
+PXE Configuration Example:
+
+- ```yaml
+  pxe:
+    isoImageFileUrl: http://hostname-or-ip/iso-publish-path/my-liveos.iso
+  ```
+
+The supported download protocols are: nfs, http, https, ftp, torent, tftp.
+
+This field is mutually exclusive with `isoImageBaseUrl`.
+
+For an overview of Azure Linux Image Customizer support for PXE, see the 
+[PXE support page](./pxe.md).
+
 ## iso type
 
 Specifies the configuration for the generated ISO media.

toolkit/tools/imagecustomizer/docs/pxe.md

@@ -0,0 +1,133 @@
+# Azure Linux Image Customizer PXE Support
+
+## PXE Overview
+
+Booting a host with an OS served over the network is one of the most popular
+methods for booting baremetal hosts. It requires no physical access to individual
+hosts and also centralizes the deployment configuration to a single server.
+
+One way of enabling such setup is using the PXE (Preboot eXecution Environment)
+Boot protocol. The user can setup a server with all the OS artifacts, a DHCP
+endpoint, and a tftp connection endpoint. When a client machine is powered on,
+and its firmware will look for a DHCP server on the same network and find the
+one configured by the user.
+
+The DHCP server will serve information about the tftp endpoint to the client,
+and the client firmware can then proceed with retrieving the OS artifacts over
+tftp, then loading them into memory, and finally handing control over to the
+loaded OS.
+
+The tftp protocol expects certain artifacts to be present on the server:
+
+- the boot loader (the shim and something like grub).
+- the boot loader configuration (like grub.cfg).
+- the kernel image.
+- the initrd image.
+
+Once retrieved, the boot loader is run. Then the boot loader reads the
+boot loader configuration and then transfers control over to the kernel image
+with the retrieved initrd image as its file system.
+
+The initrd image is customized to perform the next set of tasks now that an
+OS is running. The tasks can range from just running some local scripts all
+the way to installing another OS.
+
+## LiveOS ISOs and PXE Support
+
+A LiveOS ISO image is a bootable ISO image that runs all the necessary
+components from memory (i.e. does not need to install anything to the host
+persistent storage).
+
+The necessary components can be either embedded into the initrd image itself
+or embedded into a separate 'rootfs' image (to allow much smaller
+initrd images). If separate, then, the initrd image must be configured with an
+agent that will look for the rootfs image, and transition control over to the
+rootfs at boot time.
+
+Dracut provides the `dmsquash-live` module which managed this transition from
+the initrd image over to the rootfs image.
+
+The **Azure Linux Image Customizer** produces such LiveOS ISO images. A typical
+image holds the following artifacts:
+
+- the boot loader (the shim and something like grub).
+- the boot loader configuration.
+- the kernel image.
+- the initrd image.
+- the rootfs image.
+- other user defined artifacts (optional).
+
+Note that the first 4 artifacts are what is necessary to get an OS kernel up
+and running in a network boot scenario. What remains for a successful booting
+of a LiveOS over the network is to make the rootfs image available for the final
+transition (during the initrd phase).
+
+Dracut enables that entire flow through the use of the `livenet` module - where
+it inspects the `root=live:liveos-iso-url` kernel parameter from the boot loader
+config file, and if it recognizes the `liveos-iso-url` protocol, it downloads
+the ISO, and then proceeds to pivot to the embedded rootfs image.
+
+The user can customize the rootfs using the Azure Linux Image Customizer as
+usual. In case of additional artifacts that need downloading, the user can
+install a daemon on the rootfs which will run when control is transferred to
+the rootfs image and download any additional items.
+
+## Creating and Deploying PXE Boot Artifacts
+
+The Azure Linux Image Customizer produces LiveOS ISO images that are also PXE
+bootable. So, the user can simply create an ISO image as usual, and the output
+can be taken and deployed to a PXE server.
+
+To make the deployment of the generated artifacts easier for the user, the
+Azure Linux Image Customizer offers the following configurations:
+
+- In the input configuration, there is a `pxe` node under which the user can
+  configure PXE related properties - like the URL of the LiveOS ISO image to
+  download (note that this image is the same image being built).
+  See the [Azure Linux Image Customizer configuration](./configuration.md#pxe-type)
+  page for more information.
+- When invoking the Azure Linux Image Customizer, the user can also elect to
+  export the artifacts to a local folder.
+  See the [Azure Linux Image Customizer command line](./cli.md#output-pxe-artifacts-dir)
+  page for more information.
+
+Below is a list of required artifacts and where on the PXE server they should
+be deployed:
+
+```
+ISO media layout           artifacts local folder      target on PXE server
+-----------------------    ------------------------    ------------------------------
+|- efi                      |                           <tftp-server-root>
+   |- boot                  |                             |
+      |- bootx64.efi        |- bootx64.efi                |- bootx64.efi
+      |- grubx64.efi        |- grubx64.efi                |- grubx64.efi
+|- boot                     |- boot                       |- boot
+   |- grub2                    |- grub2                      |- grub2
+      |- grub-pxe.cfg             |- grub.cfg                   |- grub.cfg
+      |- grubenv                  |- grubenv                    |- grubenv
+      |- grub.cfg
+   |- vmlinuz                  |- vmlinuz                    |- vmlinuz
+   |- initrd.img               |- initrd.img                 |- initrd.img
+
+                                                        <yyyy-server-root>
+|- other-user-artifacts     |- other-user-artifacts       |- other-user-artifacts
+                            |- <liveos>.iso               |- <liveos>.iso
+```
+
+Notes:
+
+- Note that the `/boot/grub2/grub.cfg` file in the ISO media is not used for
+  PXE booting. Instead, the `/boot/grub2/grub-pxe.cfg` gets renamed to `grub.cfg`
+  and is used instead.
+- `yyyy` can be any protocol supported by Dracut's `livenet` module (i.e
+  tftp, http, etc).
+- The ISO image file location under the server root is customizable -
+  but it must be such that its URL matches what is specified in the grub.cfg
+  `root=live:<URL>`.
+- While the core OS artifacts (the bootloader, its configuration, the kernel,
+  initrd image, and rootfs image) will be downloaded and used automatically,
+  the user will need to independently implement a way to download any
+  additional artifacts. For example, the user can implement a daemon (and place
+  it on the root file system) that will reach out and download the additional
+  artifacts when it is up and running. The daemon can be configured with where
+  to download the artifacts from, and what to do with them.

toolkit/tools/imagecustomizer/main.go

@@ -27,6 +27,7 @@ var (
 	rpmSources                  = app.Flag("rpm-source", "Path to a RPM repo config file or a directory containing RPMs.").Strings()
 	disableBaseImageRpmRepos    = app.Flag("disable-base-image-rpm-repos", "Disable the base image's RPM repos as an RPM source").Bool()
 	enableShrinkFilesystems     = app.Flag("shrink-filesystems", "Enable shrinking of filesystems to minimum size. Supports ext2, ext3, ext4 filesystem types.").Bool()
+	outputPXEArtifactsDir       = app.Flag("output-pxe-artifacts-dir", "Create a directory with customized image PXE booting artifacts. '--output-image-format' must be set to 'iso'.").String()
 	logFlags                    = exe.SetupLogFlags(app)
 	profFlags                   = exe.SetupProfileFlags(app)
 	timestampFile               = app.Flag("timestamp-file", "File that stores timestamps for this program.").String()
@@ -70,7 +71,8 @@ func customizeImage() error {
 	var err error
 
 	err = imagecustomizerlib.CustomizeImageWithConfigFile(*buildDir, *configFile, *imageFile,
-		*rpmSources, *outputImageFile, *outputImageFormat, *outputSplitPartitionsFormat, !*disableBaseImageRpmRepos, *enableShrinkFilesystems)
+		*rpmSources, *outputImageFile, *outputImageFormat, *outputSplitPartitionsFormat, *outputPXEArtifactsDir,
+		!*disableBaseImageRpmRepos, *enableShrinkFilesystems)
 	if err != nil {
 		return err
 	}

toolkit/tools/imagecustomizerapi/config.go

@@ -8,6 +8,7 @@ import "fmt"
 type Config struct {
 	Storage Storage `yaml:"storage"`
 	Iso     *Iso    `yaml:"iso"`
+	Pxe     *Pxe    `yaml:"pxe"`
 	OS      *OS     `yaml:"os"`
 	Scripts Scripts `yaml:"scripts"`
 }
@@ -27,6 +28,13 @@ func (c *Config) IsValid() (err error) {
 		}
 	}
 
+	if c.Pxe != nil {
+		err = c.Pxe.IsValid()
+		if err != nil {
+			return fmt.Errorf("invalid 'pxe' field:\n%w", err)
+		}
+	}
+
 	hasResetBootLoader := false
 	if c.OS != nil {
 		err = c.OS.IsValid()

toolkit/tools/imagecustomizerapi/pxe.go

@@ -0,0 +1,57 @@
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT License.
+
+package imagecustomizerapi
+
+import (
+	"fmt"
+	"net/url"
+	"strings"
+)
+
+var PxeIsoDownloadProtocols = []string{"ftp://", "http://", "https://", "nfs://", "tftp://"}
+
+// Iso defines how the generated iso media should be configured.
+type Pxe struct {
+	IsoImageBaseUrl string `yaml:"isoImageBaseUrl"`
+	IsoImageFileUrl string `yaml:"isoImageFileUrl"`
+}
+
+func IsValidPxeUrl(urlString string) error {
+	if urlString == "" {
+		return nil
+	}
+
+	_, err := url.Parse(urlString)
+	if err != nil {
+		return fmt.Errorf("invalid URL value (%s):\n%w", urlString, err)
+	}
+
+	protocolFound := false
+	for _, protocol := range PxeIsoDownloadProtocols {
+		if strings.HasPrefix(urlString, protocol) {
+			protocolFound = true
+			break
+		}
+	}
+	if !protocolFound {
+		return fmt.Errorf("unsupported iso image URL protocol in (%s). One of (%v) is expected.", urlString, PxeIsoDownloadProtocols)
+	}
+
+	return nil
+}
+
+func (p *Pxe) IsValid() error {
+	if p.IsoImageBaseUrl != "" && p.IsoImageFileUrl != "" {
+		return fmt.Errorf("cannot specify both 'isoImageBaseUrl' and 'isoImageFileUrl' at the same time.")
+	}
+	err := IsValidPxeUrl(p.IsoImageBaseUrl)
+	if err != nil {
+		return fmt.Errorf("invalid 'isoImageBaseUrl' field value (%s):\n%w", p.IsoImageBaseUrl, err)
+	}
+	err = IsValidPxeUrl(p.IsoImageFileUrl)
+	if err != nil {
+		return fmt.Errorf("invalid 'isoImageFileUrl' field value (%s):\n%w", p.IsoImageFileUrl, err)
+	}
+	return nil
+}

toolkit/tools/pkg/imagecustomizerlib/customizefiles_test.go

@@ -80,7 +80,7 @@ func TestCustomizeImageAdditionalFiles(t *testing.T) {
 
 	// Customize image.
 	err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", "",
-		false /*useBaseImageRpmRepos*/, false /*enableShrinkFilesystems*/)
+		"" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, false /*enableShrinkFilesystems*/)
 	if !assert.NoError(t, err) {
 		return
 	}
@@ -126,7 +126,7 @@ func TestCustomizeImageAdditionalFilesInfiniteFile(t *testing.T) {
 
 	// Customize image.
 	err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", "",
-		false /*useBaseImageRpmRepos*/, false /*enableShrinkFilesystems*/)
+		"" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, false /*enableShrinkFilesystems*/)
 	assert.ErrorContains(t, err, "failed to copy (/dev/zero)")
 	assert.ErrorContains(t, err, "No space left on device")
 }
@@ -202,7 +202,7 @@ func TestCustomizeImageAdditionalDirs(t *testing.T) {
 
 	// Customize image.
 	err := CustomizeImageWithConfigFile(buildDir, configFile, baseImage, nil, outImageFilePath, "raw", "",
-		false /*useBaseImageRpmRepos*/, false /*enableShrinkFilesystems*/)
+		"" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, false /*enableShrinkFilesystems*/)
 	if !assert.NoError(t, err) {
 		return
 	}
@@ -258,7 +258,7 @@ func TestCustomizeImageAdditionalDirsInfiniteFile(t *testing.T) {
 
 	// Customize image.
 	err = CustomizeImage(buildDir, testTmpDir, &config, baseImage, nil, outImageFilePath, "raw", "",
-		false /*useBaseImageRpmRepos*/, false /*enableShrinkFilesystems*/)
+		"" /*outputPXEArtifactsDir*/, false /*useBaseImageRpmRepos*/, false /*enableShrinkFilesystems*/)
 	assert.ErrorContains(t, err, "failed to copy directory")
 	assert.ErrorContains(t, err, "failed to copy file")
 	assert.ErrorContains(t, err, "No space left on device")