Booting Flatcar Linux via PXE

These instructions will walk you through booting Flatcar Linux via PXE on real or virtual hardware. By default, this will run Flatcar Linux completely out of RAM. Flatcar Linux can also be installed to disk.

A mininum of 3 GB of RAM is required to boot Flatcar Linux via PXE.

Configuring pxelinux

This guide assumes you already have a working PXE server using pxelinux. If you need suggestions on how to set a server up, check out guides for Debian, Fedora or Ubuntu.

Setting up pxelinux.cfg

When configuring the Flatcar Linux pxelinux.cfg there are a few kernel options that may be useful but all are optional.

This is an example pxelinux.cfg file that assumes Flatcar Linux is the only option. You should be able to copy this verbatim into /var/lib/tftpboot/pxelinux.cfg/default after providing an Ignition config URL:

default flatcar
prompt 1
timeout 15

display boot.msg

label flatcar
  menu default
  kernel flatcar_production_pxe.vmlinuz
  initrd flatcar_production_pxe_image.cpio.gz
  append flatcar.first_boot=1 flatcar.config.url=https://example.com/pxe-config.ign

Here's a common config example which should be located at the URL from above:

systemd:
  units:
    - name: etcd2.service
      enable: true

passwd:
  users:
    - name: core
      ssh_authorized_keys:
        - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDGdByTgSVHq...

Choose a channel

Flatcar Linux is designed to be updated automatically with different schedules per channel. You can disable this feature, although we don't recommend it. Read the release notes for specific features and bug fixes.

PXE booted machines cannot currently update themselves when new versions are released to a channel. To update to the latest version of Flatcar Linux download/verify these files again and reboot.

The Alpha channel closely tracks master and is released frequently. The newest versions of system libraries and utilities will be available for testing. The current version is Flatcar Linux 1981.0.0.

In the config above you can see that a Kernel image and a initramfs file is needed. Download these two files into your tftp root.

The flatcar_production_pxe.vmlinuz.sig and flatcar_production_pxe_image.cpio.gz.sig files can be used to verify the downloaded files.

cd /var/lib/tftpboot
wget https://alpha.release.flatcar-linux.net/amd64-usr/current/flatcar_production_pxe.vmlinuz
wget https://alpha.release.flatcar-linux.net/amd64-usr/current/flatcar_production_pxe.vmlinuz.sig
wget https://alpha.release.flatcar-linux.net/amd64-usr/current/flatcar_production_pxe_image.cpio.gz
wget https://alpha.release.flatcar-linux.net/amd64-usr/current/flatcar_production_pxe_image.cpio.gz.sig
gpg --verify flatcar_production_pxe.vmlinuz.sig
gpg --verify flatcar_production_pxe_image.cpio.gz.sig
      

The Beta channel consists of promoted Alpha releases. The current version is Flatcar Linux 1967.1.0.

In the config above you can see that a Kernel image and a initramfs file is needed. Download these two files into your tftp root.

The flatcar_production_pxe.vmlinuz.sig and flatcar_production_pxe_image.cpio.gz.sig files can be used to verify the downloaded files.

cd /var/lib/tftpboot
wget https://beta.release.flatcar-linux.net/amd64-usr/current/flatcar_production_pxe.vmlinuz
wget https://beta.release.flatcar-linux.net/amd64-usr/current/flatcar_production_pxe.vmlinuz.sig
wget https://beta.release.flatcar-linux.net/amd64-usr/current/flatcar_production_pxe_image.cpio.gz
wget https://beta.release.flatcar-linux.net/amd64-usr/current/flatcar_production_pxe_image.cpio.gz.sig
gpg --verify flatcar_production_pxe.vmlinuz.sig
gpg --verify flatcar_production_pxe_image.cpio.gz.sig
      

The Stable channel should be used by production clusters. Versions of Flatcar Linux are battle-tested within the Beta and Alpha channels before being promoted. The current version is Flatcar Linux 1911.4.0.

In the config above you can see that a Kernel image and a initramfs file is needed. Download these two files into your tftp root.

The flatcar_production_pxe.vmlinuz.sig and flatcar_production_pxe_image.cpio.gz.sig files can be used to verify the downloaded files.

cd /var/lib/tftpboot
wget https://stable.release.flatcar-linux.net/amd64-usr/current/flatcar_production_pxe.vmlinuz
wget https://stable.release.flatcar-linux.net/amd64-usr/current/flatcar_production_pxe.vmlinuz.sig
wget https://stable.release.flatcar-linux.net/amd64-usr/current/flatcar_production_pxe_image.cpio.gz
wget https://stable.release.flatcar-linux.net/amd64-usr/current/flatcar_production_pxe_image.cpio.gz.sig
gpg --verify flatcar_production_pxe.vmlinuz.sig
gpg --verify flatcar_production_pxe_image.cpio.gz.sig
      

Booting the box

After setting up the PXE server as outlined above you can start the target machine in PXE boot mode. The machine should grab the image from the server and boot into Flatcar Linux. If something goes wrong you can direct questions to the IRC channel or mailing list.

This is localhost.unknown_domain (Linux x86_64 3.10.10+) 19:53:36
SSH host key: 24:2e:f1:3f:5f:9c:63:e5:8c:17:47:32:f4:09:5d:78 (RSA)
SSH host key: ed:84:4d:05:e3:7d:e3:d0:b9:58:90:58:3b:99:3a:4c (DSA)
ens0: 10.0.2.15 fe80::5054:ff:fe12:3456
localhost login:

Logging in

The IP address for the machine should be printed out to the terminal for convenience. If it doesn't show up immediately, press enter a few times and it should show up. Now you can simply SSH in using public key authentication:

ssh core@10.0.2.15

Update Process

Since our upgrade process requires a disk, this image does not have the option to update itself. Instead, the box simply needs to be rebooted and will be running the latest version, assuming that the image served by the PXE server is regularly updated.

Installation

Once booted it is possible to install Flatcar Linux on a local disk or to just use local storage for the writable root filesystem while continuing to boot Flatcar Linux itself via PXE.

If you plan on using Docker we recommend using a local ext4 filesystem with overlayfs, however, btrfs is also available to use if needed.

For example, to setup an ext4 root filesystem on /dev/sda:

storage:
  disks:
  - device: /dev/sda
    wipe_table: true
    partitions:
    - label: ROOT
  filesystems:
  - mount:
      device: /dev/disk/by-partlabel/ROOT
      format: ext4
      wipe_filesystem: true
      label: ROOT

And add root=/dev/sda1 or root=LABEL=ROOT to the kernel options as documented above.

Similarly, to setup a btrfs root filesystem on /dev/sda:

storage:
  disks:
  - device: /dev/sda
    wipe_table: true
    partitions:
    - label: ROOT
  filesystems:
  - mount:
      device: /dev/disk/by-partlabel/ROOT
      format: btrfs
      wipe_filesystem: true
      label: ROOT

Adding a Custom OEM

Similar to the OEM partition in Flatcar Linux disk images, PXE images can be customized with an Ignition config bundled in the initramfs. Simply create a ./usr/share/oem/ directory, add a config.ign file containing the Ignition config, and add the directory tree as an additional initramfs:

mkdir -p usr/share/oem
cp example.ign ./usr/share/oem/config.ign
find usr | cpio -o -H newc -O oem.cpio
gzip oem.cpio

Confirm the archive looks correct and has your config inside of it:

gzip --stdout --decompress oem.cpio.gz | cpio -it
./
usr
usr/share
usr/share/oem
usr/share/oem/config.ign

Add the oem.cpio.gz file to your PXE boot directory, then append it to the initrd line in your pxelinux.cfg:

...
initrd flatcar_production_pxe_image.cpio.gz,oem.cpio.gz
kernel flatcar_production_pxe.vmlinuz flatcar.first_boot=1
...

Using Flatcar Linux

Now that you have a machine booted it is time to play around. Check out the Flatcar Linux Quickstart guide or dig into more specific topics.