PetaLinux

This page explains how to build and configure the host Linux system for HERO using PetaLinux/Yocto. This HOWTO is for HERO based on the Xilinx Zynq UltraScale+ MPSoC platform.

Overview

The specific board used for this HOWTO is the UltraSOM+ TE0808 module (using a XCZU9EG-1FFVC900E chip, with 2 GiB DDR4 RAM) on the UltraITX+ Baseboard TEBF0808 from Trenz Electronic. However, it should also work for similar boards as the ZCU102 from Xilinx. In fact, it should be even simpler to get it running on the Xilinx board as all the required board files come with the PetaLinux installation.

Build system

The following build system was used for this HOWTO:

  • CentOS 7 workstation
  • Xilinx Vivado 2017.2 with SDK and PetaLinux
  • Bash shell

Required input files

The following files are required for this HOWTO:

  • board support files
  • bigpulp-zux.hdf hardware definition file exported from Vivado
  • bigpulp_zux_top.bit bitstream file generated by Vivado
  • the helper scripts provided in the hero-support GIT repository

Generated output files

The following files will be generated throughout this HOWTO:

  1. image.ub FIT image consisting of

    • Linux kernel image
    • device-tree blob (DTB)
    • Root filesystem image
  2. BOOT.BIN consisting of

    • First-stage boot loader (FSBL) image
    • U-Boot boot loader
    • FPGA bitstream
    • Power management unit (PMU) firmware
    • ARM Trusted Firmware image

To boot the platform, these files need to be copied to the boot partition of the SD card.

NOTE: PetaLinux automatically regenerates the individual components from the sources when generating the two output files. Consequently, any changes manually applied to any of these components, for example to tweak a specific node in the device tree, are not propagated to the final images. The purpose of this tutorial is to show how and where to add properly add such customizations in the PetaLinux flow.

NOTE: With the default PetaLinux configuration used also by this tutorial, U-Boot loads the root filesystem image from the SD card into memory at startup. The filesystem is then read- and writeable only in memory (RAMDISK). No changes are written back to the image on the SD card. From a point of usability, this is not ideal. However, there are several important advantages:

  • No filesystem inconsistency and corruption can occur, for example due to system crashes, SD card wearout.
  • Very fast startup
  • Reduced SD card wearout

It is however recommended to setup either a second data partition on the SD card or and network file system (NFS) share and mount this for permanent storage. In the case of the latter, filesystem consistency is handled by the NFS server, which helps to avoid data corruption. This tutorial shows how to include the required scripts to (automatically) mount an NFS share at startup.

PetaLinux Project Setup

Create a new workspace directory on your local scratch partition. Throughout this HOWTO, we assume this directory to be

/scratch/${USER}/${BOARD}/petalinux

Where ${BOARD} has been set to te0808 for this HOWTO. The host name of the target system we generate is wimmis02 and is stored in the variable ${TARGET_NAME}.

These variables are defined in the sourceme.sh script found in linux/petalinux/scripts/ inside the hero-support repository. Copy this file to the new directory and edit the variables accordingly.

Navigate to the workspace directory, source your Vivado environment script, and type the following command to create a new PetaLinux project:

petalinux-create -t project -n ${TARGET_NAME} --template zynqMP

Enter the newly created workspace directory cd os/${TARGET_NAME} and configure the project, i.e., import the hardware definition file exported by Vivado:

petalinux-config --get-hw-description=${VIVADO_EXPORT_PATH}/

This command will open PetaLinux top-level menuconfig. Navigate to the Subsystem AUTO Hardware Settings ---> Ethernet Settings ---> menu and configure the MAC address of your board. Save and exit menuconfig.

Finally, copy the remaining files from linux/petalinux/scripts inside the hero-support repository to images/linux/ inside the workspace directory. These files will be used during the later steps.

To re-open menuconfig at a later point, simply type

petalinux-config

Xilinx Linux Kernel

By default, PetaLinux compiles the Linux kernel from sources delivered as part of the PetaLinux installation, which cannot be modified.

To switch to a different kernel version or modify the sources, PetaLinux must be configured to use external kernel sources. How to achieve this is shown in the following. If you do not need to modify the kernel sources, you can skip this step and directly jump to Device Tree Generation.

To configure PetaLinux for using external kernel sources, navigate to the components folder inside your PetaLinux project folder

cd /scratch/${USER}/${BOARD}/petalinux/${NAME}/components

and create a new folder

mkdir ext_sources
cd ext_sources

and enter it.

Clone the kernel sources for Xilinx Linux

git clone git://github.com/Xilinx/linux-xlnx.git

Rename this newly created folder and enter it

mv linux-xlnx linux-xlnx-plnx
cd linux-xlnx-plnx

Checkout the proper version of the repository

git checkout tags/xilinx-v2017.2

Now, you can navigate back to the PetaLinux project folder. Type

petalinux-config

to open menuconfig. Go to the Linux Components Selection ---> linux-kernel ---> and then select ext-local-src. Select External linux-kernel local source settings ---> and type

${TOPDIR}/../components/ext_sources/linux-xlnx-plnx

for External linux-kernel local source path. Save and exit menuconfig.

Then, open menuconfig for the kernel configuration

petalinux-config -c kernel

and navigate to General setup ---> and enter

-xilinx-v2017.2

for Local version - append to kernel release. Save and exit menuconfig.

To enable shared virtual memory (SVM) between bigPULP and the ARM host using the ARM SMMU, the driver of this device needs to be patched. To this end, type

cd components/ext_sources/linux-xlnx-plnx/drivers/iommu

and apply the patches found in linux/petalinux/kernel inside the hero-support repository:

patch < arm-smmu.patch
patch < io-pgtable-arm.patch

The kernel will automatically be compiled when building the image.ub later. To manually re-compile the kernel, you can execute

petalinux-build -c kernel

Device Tree Generation

By default, PetaLinux generates the device tree based on sources provided with the installation (deployed to components/plnx_workspace/device-tree-generation) as well as information provided in the hardware defintion file (bigpulp-zux.hdf in our case).

For HERO, we need to provide a custom device node for PULP as well as modifications to other nodes, e.g., to reserve some DRAM as contiguous memory, to allocate system DMA channels and an interrupt line, and to enable the SMMU for SVM. The proper way to do this is through the file system-user.dtsi located in project-spec/meta-user/recipes-bsp/device-tree/files/. If you do not need to modify the device tree, you can skip this step and directly jump to Yocto Root Filesystem.

Copy the provided system-user.dtsi from linux/petalinux/device-tree inside the hero-support repository to project-spec/meta-user/recipes-bsp/device-tree/files/.

The device tree will automatically be compiled when building the image.ub later. To manually re-compile the device tree, you can execute

 petalinux-build -c device-tree

The compiled device tree blob system.dtb can be found in images/linux in the workspace directory. To convert back and forth between the blob and the source, you can use the provided scripts dtb_to_dts.sh and dts_to_dtb.sh, respectively.

Yocto Root Filesystem

For HERO, we make the following adjustments to the Yocto root filesystem.

Enable the NTPD

To enable the synchronization of the system time (useful for generating logs), we install the network time protocol deamon (NTPD). By default, the NTPD is not enabled by PetaLinux but it is part of BusyBox, which itself is part of Yocto.

To enable the NTPD, type

cd project-spec/meta-plnx-generated/recipes-core/busybox

to enter the BusyBox recipe folder. From the folder linux/petalinux/ntpd/ of the hero-support GIT repository, copy

  1. the Yocto recipe busybox_1.%.bbappend to the current folder, and
  2. the BusyBox config snippet ntpd.cfg to the files folder inside the current folder.

Include Custom Files

To include a set of custom files, such as SSH keys, startup scripts etc., in the root filesystem image (RAMDISK), a new PetaLinux app is created. We call this app “custom-files”.

Execute

petalinux-create -t apps -n custom-files --enable

to create this app and enable its build as part of the root filesystem.

Then, type

cd project-spec/meta-user/recipes-apps/custom-files

to enter the app folder. From the folder linux/petalinux/custom-files/ of the hero-support GIT repository, copy

  1. the Yocto recipe custom-files.bb to the app folder, and
  2. the actual scripts and custom files from inside files to the files folder inside the app folder.

The custom-files.bb recipe basically tells PetaLinux to copy the stuff inside files to the root filesystem image. One of these scripts (/etc/rc5.d/S25time) is used to synchronize the system time using NTPD. Modify this script synchronize with a time server of your choice.

NOTE: The actual SSH keys are not part of the GIT repository for security reasons. They are randomly generated on the target system upon startup. You need to extract and add them to the image after having booted the system for the first time.

NOTE: Sometimes, PetaLinux accidentally removes the actual custom files located inside files when generating the root filesystem image. It then complains about not finding them anymore. Therefore, it is worth to create a backup of the files including the SSH keys to copy them back if needed.

The root filesystem will automatically be generated when building the image.ub later. To manually trigger the build of the root filesystem execute

 petalinux-build -c rootfs

The generated root filesystem rootfs.cpio.gz can be found in images/linux in the workspace directory. Use the script unpack_rootfs.sh to unpack and inspect the image. This way, you can check whether the custom files are indeed included in the image.

Generating the Boot Files

Execute

petalinux-build

to let PetaLinux build the boot loaders, firmware, kernel, device tree and root filesystem. Some of the generated files will be copied automatically to images/linux. In case the PetaLinux claims that system.hdf cannot be found, execute

cp project-spec/hw-description/system.hdf build/tmp/work/aarch64-xilinx-linux/external-hdf/1.0-r0/git/plnx_aarch64/.

To generate the actual images to be deployed on the SD card, three more steps are required.

FIT Image

To let PetaLinux build the FIT image image.ub (out of the kernel image, the device tree blob system.dtb and the root filesystem image rootfs.cpio), execute

petalinux-package --image

ARM Trusted Firmware

For some reason, the Zynq MPSoC does by default not allow accelerators placed in the PL to access memory coherent with the caches of the ARM host, despite being connected through the coherent ACE-Lite ports. In particular the cache coherent interconnect (CCI-400) rejects any transcations originating from the PL which are marked as shareable. To work around this issue, the ARM Trusted Firmware needs to be patched. More precisely, we need to enable access to the CCI-400 configuration registers by non-secure software, such that we can later reconfigure the CCI-400 using a driver module.

To this end, navigate to the following directory in your workspace folder

cd /scratch/${USER}/${BOARD}/petalinux/${NAME}/components/ext_sources

and execute

git clone https://github.com/Xilinx/arm-trusted-firmware.git

Then, enter the new folder and checkout the proper version of the repository

cd arm-trusted-firmware
git checkout tags/xilinx-v2017.2

navigate to the folder containing the CCI driver

cd drivers/arm/cci

and apply the patch found in linux/petalinux/arm-trusted-firmware inside the hero-support repository:

patch < cci.patch

Navigate back to the directory images/linux and run

./compile_arm-trusted_firmware.sh

To build the patched firmware.

BOOT.BIN

To generate the BOOT.BIN containing FSBL, U-Boot image, PMU firmware, FPGA bitstream, and ARM Trusted Firmware, go to images/linux and execute the script

./boot_gen.sh

Finally, the images image.ub and BOOT.BIN can be deployed to the SD card by executing

./copy_to_sd_card.sh

Dynamic FPGA Reconfiguration

To deploy a different FPGA bitstream without having to fiddle around with the SD card, PetaLinux allows to reconfigure the FPGA while the system is running. By default, the corresponding drivers are already installed on the system.

To generate the .bin file required by the dynamic reconfiguration infrastructure out of the bitstream and copy this file to the running target system, execute the script

./bit_to_bin.sh

located in images/linux.

Then, connect to the running system and execute the script

./update_bitstream.sh

NOTE: It is not safe to load a different bitstream at runtime if this changes the interfaces between the ARM host and the FPGA.

References