123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205 |
- ===================
- Testing BPF on s390
- ===================
- 1. Introduction
- ***************
- IBM Z are mainframe computers, which are descendants of IBM System/360 from
- year 1964. They are supported by the Linux kernel under the name "s390". This
- document describes how to test BPF in an s390 QEMU guest.
- 2. One-time setup
- *****************
- The following is required to build and run the test suite:
- * s390 GCC
- * s390 development headers and libraries
- * Clang with BPF support
- * QEMU with s390 support
- * Disk image with s390 rootfs
- Debian supports installing compiler and libraries for s390 out of the box.
- Users of other distros may use debootstrap in order to set up a Debian chroot::
- sudo debootstrap \
- --variant=minbase \
- --include=sudo \
- testing \
- ./s390-toolchain
- sudo mount --rbind /dev ./s390-toolchain/dev
- sudo mount --rbind /proc ./s390-toolchain/proc
- sudo mount --rbind /sys ./s390-toolchain/sys
- sudo chroot ./s390-toolchain
- Once on Debian, the build prerequisites can be installed as follows::
- sudo dpkg --add-architecture s390x
- sudo apt-get update
- sudo apt-get install \
- bc \
- bison \
- cmake \
- debootstrap \
- dwarves \
- flex \
- g++ \
- gcc \
- g++-s390x-linux-gnu \
- gcc-s390x-linux-gnu \
- gdb-multiarch \
- git \
- make \
- python3 \
- qemu-system-misc \
- qemu-utils \
- rsync \
- libcap-dev:s390x \
- libelf-dev:s390x \
- libncurses-dev
- Latest Clang targeting BPF can be installed as follows::
- git clone https://github.com/llvm/llvm-project.git
- ln -s ../../clang llvm-project/llvm/tools/
- mkdir llvm-project-build
- cd llvm-project-build
- cmake \
- -DLLVM_TARGETS_TO_BUILD=BPF \
- -DCMAKE_BUILD_TYPE=Release \
- -DCMAKE_INSTALL_PREFIX=/opt/clang-bpf \
- ../llvm-project/llvm
- make
- sudo make install
- export PATH=/opt/clang-bpf/bin:$PATH
- The disk image can be prepared using a loopback mount and debootstrap::
- qemu-img create -f raw ./s390.img 1G
- sudo losetup -f ./s390.img
- sudo mkfs.ext4 /dev/loopX
- mkdir ./s390.rootfs
- sudo mount /dev/loopX ./s390.rootfs
- sudo debootstrap \
- --foreign \
- --arch=s390x \
- --variant=minbase \
- --include=" \
- iproute2, \
- iputils-ping, \
- isc-dhcp-client, \
- kmod, \
- libcap2, \
- libelf1, \
- netcat, \
- procps" \
- testing \
- ./s390.rootfs
- sudo umount ./s390.rootfs
- sudo losetup -d /dev/loopX
- 3. Compilation
- **************
- In addition to the usual Kconfig options required to run the BPF test suite, it
- is also helpful to select::
- CONFIG_NET_9P=y
- CONFIG_9P_FS=y
- CONFIG_NET_9P_VIRTIO=y
- CONFIG_VIRTIO_PCI=y
- as that would enable a very easy way to share files with the s390 virtual
- machine.
- Compiling kernel, modules and testsuite, as well as preparing gdb scripts to
- simplify debugging, can be done using the following commands::
- make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- menuconfig
- make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- bzImage modules scripts_gdb
- make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- \
- -C tools/testing/selftests \
- TARGETS=bpf \
- INSTALL_PATH=$PWD/tools/testing/selftests/kselftest_install \
- install
- 4. Running the test suite
- *************************
- The virtual machine can be started as follows::
- qemu-system-s390x \
- -cpu max,zpci=on \
- -smp 2 \
- -m 4G \
- -kernel linux/arch/s390/boot/compressed/vmlinux \
- -drive file=./s390.img,if=virtio,format=raw \
- -nographic \
- -append 'root=/dev/vda rw console=ttyS1' \
- -virtfs local,path=./linux,security_model=none,mount_tag=linux \
- -object rng-random,filename=/dev/urandom,id=rng0 \
- -device virtio-rng-ccw,rng=rng0 \
- -netdev user,id=net0 \
- -device virtio-net-ccw,netdev=net0
- When using this on a real IBM Z, ``-enable-kvm`` may be added for better
- performance. When starting the virtual machine for the first time, disk image
- setup must be finalized using the following command::
- /debootstrap/debootstrap --second-stage
- Directory with the code built on the host as well as ``/proc`` and ``/sys``
- need to be mounted as follows::
- mkdir -p /linux
- mount -t 9p linux /linux
- mount -t proc proc /proc
- mount -t sysfs sys /sys
- After that, the test suite can be run using the following commands::
- cd /linux/tools/testing/selftests/kselftest_install
- ./run_kselftest.sh
- As usual, tests can be also run individually::
- cd /linux/tools/testing/selftests/bpf
- ./test_verifier
- 5. Debugging
- ************
- It is possible to debug the s390 kernel using QEMU GDB stub, which is activated
- by passing ``-s`` to QEMU.
- It is preferable to turn KASLR off, so that gdb would know where to find the
- kernel image in memory, by building the kernel with::
- RANDOMIZE_BASE=n
- GDB can then be attached using the following command::
- gdb-multiarch -ex 'target remote localhost:1234' ./vmlinux
- 6. Network
- **********
- In case one needs to use the network in the virtual machine in order to e.g.
- install additional packages, it can be configured using::
- dhclient eth0
- 7. Links
- ********
- This document is a compilation of techniques, whose more comprehensive
- descriptions can be found by following these links:
- - `Debootstrap <https://wiki.debian.org/EmDebian/CrossDebootstrap>`_
- - `Multiarch <https://wiki.debian.org/Multiarch/HOWTO>`_
- - `Building LLVM <https://llvm.org/docs/CMake.html>`_
- - `Cross-compiling the kernel <https://wiki.gentoo.org/wiki/Embedded_Handbook/General/Cross-compiling_the_kernel>`_
- - `QEMU s390x Guest Support <https://wiki.qemu.org/Documentation/Platforms/S390X>`_
- - `Plan 9 folder sharing over Virtio <https://wiki.qemu.org/Documentation/9psetup>`_
- - `Using GDB with QEMU <https://wiki.osdev.org/Kernel_Debugging#Use_GDB_with_QEMU>`_
|