Solarian Programmer

My programming ramblings

Using Clang as a cross compiler for Raspberry Pi

Posted on May 4, 2019 by Paul

In this article, I will show you how to cross compile C and C++ programs on a x86-64 machine for Raspberry Pi using Clang 8. The advantage of cross compiling on a x86-64 system for armhf is that, usually one has a beefy laptop or desktop computer that can speed up, by an order of magnitude or more, the compilation of C and C++ programs. This is more visible with large C++ programs, that can take hours to build on a Raspberry Pi 3 or days on a Raspberry Pi Zero.

We’ll start by building Clang as a cross compiler on the host x86-64 system. This will let us use the latest stable version of Clang, which at the time of this writing is 8. You will also be able to run the binaries compiled with this Clang version on all versions of Raspberry Pi.

As a side note, you can install the official Clang binary directly on your Raspberry Pi, check this article if you are interested. The problem with this approach is that the compiler will work only on Raspberry Pi 2 and up, no support for RPi 1 or Zero, and that it is really slow for large C++ projects.

I recommend that you do the build in a Debian Stretch virtual machine or a Docker container in order to not mess your system. If you decide to install Stretch in a virtual machine, make sure to use the minimal netinst system. It is really important that you start with a bare bone system, because we need to install armhf executables and libraries. By using a minimal system we avoid potential conflicts with the native x86-64 versions. Also, Clang uses a lot of memory when built from sources, make sure you have at least 4GB of RAM on your system.

I’ll assume that you have a clean Raspbian install. I’ve used the latest available desktop image. In principle you should be able to follow the article using Raspbian Lite, but I did all my tests using the full desktop version of Rasbian.

First, make sure your x86-64 Debian Stretch virtual machine or container is updated:

1 sudo apt update
2 sudo apt upgrade

Next, let’s enable the armhf architecture on the x86-64 machine:

1 sudo dpkg --add-architecture armhf
2 sudo apt update
3 sudo apt install qemu-user-static

At this point, you should be able to install armhf libraries and applications on your system and run them.

We’ll start by installing a few x86-64 prerequisites like a GCC compiler toolchain, version control systems, build systems, libraries and a few other applications:

1 sudo apt install build-essential subversion cmake git python3-dev libncurses5-dev libxml2-dev libedit-dev swig doxygen graphviz xz-utils ninja-build

We’ll also install the armhf counterpart of the above:

1 sudo apt install crossbuild-essential-armhf
2 sudo apt install python3-dev:armhf libncurses5-dev:armhf libxml2-dev:armhf libedit-dev:armhf

Next step, is to get the latest stable versions of Clang, LLVM, libc++ and a few other utilities:

 1 cd ~
 2 mkdir llvm_all && cd llvm_all
 3 svn co http://llvm.org/svn/llvm-project/llvm/tags/RELEASE_800/final llvm
 4 cd llvm/tools
 5 svn co http://llvm.org/svn/llvm-project/cfe/tags/RELEASE_800/final clang
 6 cd ../..
 7 cd llvm/projects
 8 svn co http://llvm.org/svn/llvm-project/compiler-rt/tags/RELEASE_800/final compiler-rt
 9 svn co http://llvm.org/svn/llvm-project/lld/tags/RELEASE_800/final lld
10 svn co http://llvm.org/svn/llvm-project/polly/tags/RELEASE_800/final polly
11 svn co http://llvm.org/svn/llvm-project/libunwind/tags/RELEASE_800/final libunwind
12 cd ~
13 cd llvm_all
14 svn co http://llvm.org/svn/llvm-project/libcxx/tags/RELEASE_800/final libcxx
15 svn co http://llvm.org/svn/llvm-project/libcxxabi/tags/RELEASE_800/final libcxxabi
16 svn co http://llvm.org/svn/llvm-project/openmp/tags/RELEASE_800/final openmp

Now, we can build what is in the llvm folder from above, depending on the speed of your computer, this could take from 40 minutes to a few hours:

1 mkdir build_llvm && cd build_llvm
2 cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DLLVM_BUILD_DOCS=OFF -DCMAKE_INSTALL_PREFIX=/usr/local/cross_armhf_clang_8.0.0 -DCMAKE_CROSSCOMPILING=True -DLLVM_DEFAULT_TARGET_TRIPLE=arm-linux-gnueabihf -DLLVM_TARGET_ARCH=ARM -DLLVM_TARGETS_TO_BUILD=ARM ../llvm
3 
4 ninja
5 sudo ninja install

Let’s add Clang to the system path:

1 echo 'export PATH=/usr/local/cross_armhf_clang_8.0.0/bin:$PATH' >> ~/.bashrc
2 echo 'export LD_LIBRARY_PATH=/usr/local/cross_armhf_clang_8.0.0/lib:$LD_LIBRARY_PATH' >> ~/.bashrc
3 source ~/.bashrc

Verify if you can use Clang by checking the compiler version, this is what I see on my machine:

1 ~/llvm_all/build_llvm $ clang --version
2 clang version 8.0.0 (tags/RELEASE_800/final 359901)
3 Target: arm-unknown-linux-gnueabihf
4 Thread model: posix
5 InstalledDir: /usr/local/cross_armhf_clang_8.0.0/bin
6 ~/llvm_all/build_llvm $

If you get a command not found error, close and reopen your Terminal and try again.

At this point, you should be able to compile C and C++ programs, but you will be limited to use the system standard libraries and headers which correspond to GCC 6 which is a bit old. Fortunately, we can use libc++ which is also provided by Clang. Let’s build the remaining libraries libc++, libc++abi and openmp:

 1 cd ..
 2 mkdir build_libcxxabi && cd build_libcxxabi
 3 cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local/cross_armhf_clang_8.0.0 -DLLVM_TARGETS_TO_BUILD=ARM -DCMAKE_C_COMPILER=/usr/local/cross_armhf_clang_8.0.0/bin/clang -DCMAKE_CXX_COMPILER=/usr/local/cross_armhf_clang_8.0.0/bin/clang++ ../libcxxabi
 4 ninja
 5 sudo ninja install
 6 
 7 cd ..
 8 mkdir build_libcxx && cd build_libcxx
 9 cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local/cross_armhf_clang_8.0.0 -DLLVM_TARGETS_TO_BUILD=ARM -DCMAKE_C_COMPILER=/usr/local/cross_armhf_clang_8.0.0/bin/clang -DCMAKE_CXX_COMPILER=/usr/local/cross_armhf_clang_8.0.0/bin/clang++ ../libcxx
10 ninja
11 sudo ninja install
12 
13 cd ..
14 mkdir build_openmp && cd build_openmp
15 cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local/cross_armhf_clang_8.0.0 -DCMAKE_C_COMPILER=arm-linux-gnueabihf-gcc -DCMAKE_CXX_COMPILER=arm-linux-gnueabihf-g++ -DLIBOMP_ARCH=arm ../openmp
16 ninja
17 sudo ninja install

We are done, now we have a complete Clang cross toolchain. Let’s see how you can use it by compiling some small programs. I will put the next three test programs in a folder named work_examples in my home directory:

1 cd ~
2 mkdir work_examples && cd work_examples

Copy or write the next three examples in the above folder:

1 // t0.c
2 #include <stdio.h>
3 
4 int main(void) {
5     printf("Hello armhf from cross Clang\n");
6 }
 1 // t1.cpp
 2 #include <iostream>
 3 #include <vector>
 4 
 5 int main() {
 6     std::vector<int> V{1,2,3,4,5,6};
 7 
 8     for(auto &e : V) {
 9         std::cout << e << ' ';
10     }
11     std::cout << '\n';
12 }
1 // t2.cpp
2 #include <iostream>
3 #include <filesystem>
4 
5 int main() {
6     for(auto &file : std::filesystem::recursive_directory_iterator("./")) {
7         std::cout << file.path() << '\n';
8     }
9 }

You can build the first example, t0.c, from above with this command:

1 clang -std=c17 -Wall -Wextra -pedantic t0.c -o t0

Side note, you can execute the generated binary directly on your host system because we’ve installed the qemu emulator, if your write:

1 ./t0

you should see the Hello armhf from cross Clang message.

You can convince yourself that t0 is an armhf binary by using the file command. This is what I see on my machine:

1 ~/work_examples$ file t0
2 t0: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, not stripped
3 ~/work_examples$

Next, we can build the t1.cpp example in two ways:

1 clang++ -std=c++17 -Wall -Wextra -pedantic t1.cpp -o t1a
2 clang++ -std=c++17 -stdlib=libc++ -Wall -Wextra -pedantic t1.cpp -o t1b -lc++ -lc++abi

t1a from above uses the system provided libstdc++ from GCC and t1b uses libc++. You can check what libraries needs each binary by using the next commands:

1 readelf -d t1a | grep 'NEEDED'
2 readelf -d t1b | grep 'NEEDED'

This is what I see on my machine:

 1 ~/work_examples$ readelf -d t1a | grep 'NEEDED'
 2  0x00000001 (NEEDED)                     Shared library: [libstdc++.so.6]
 3  0x00000001 (NEEDED)                     Shared library: [libm.so.6]
 4  0x00000001 (NEEDED)                     Shared library: [libgcc_s.so.1]
 5  0x00000001 (NEEDED)                     Shared library: [libc.so.6]
 6 ~/work_examples$ readelf -d t1b | grep 'NEEDED'
 7  0x00000001 (NEEDED)                     Shared library: [libc++.so.1]
 8  0x00000001 (NEEDED)                     Shared library: [libc++abi.so.1]
 9  0x00000001 (NEEDED)                     Shared library: [libm.so.6]
10  0x00000001 (NEEDED)                     Shared library: [libgcc_s.so.1]
11  0x00000001 (NEEDED)                     Shared library: [libc.so.6]
12 ~/work_examples$

For building t2.cpp use the next command:

1 clang++ -std=c++17 -stdlib=libc++ -Wall -Wextra -pedantic t2.cpp -o t2 -lc++ -lc++abi -lc++fs

Please note that t2.cpp needs the libc++, libc++abi and -libc++fs libraries. You simply can’t build it with the system libstdc++ library because it is too old and lacks C++17 filesystem support.

You can check what libraries are used by t2 like before. This is what I see on my machine:

1 ~/work_examples$ readelf -d t2 | grep 'NEEDED'
2  0x00000001 (NEEDED)                     Shared library: [libc++.so.1]
3  0x00000001 (NEEDED)                     Shared library: [libc++abi.so.1]
4  0x00000001 (NEEDED)                     Shared library: [libm.so.6]
5  0x00000001 (NEEDED)                     Shared library: [libgcc_s.so.1]
6  0x00000001 (NEEDED)                     Shared library: [libc.so.6]
7 ~/work_examples$

Next, we are ready to deploy our executables to a Raspberry Pi device. We’ll also need to copy the libc++ and openmp libraries to the RPi, because our compiled versions are not available in Raspbian. I will make a folder named clang_libs in which I will copy all the required libraries:

1 mkdir ~/clang_libs
2 cp /usr/local/cross_armhf_clang_8/lib/libc++* ~/clang_libs
3 cp /usr/local/cross_armhf_clang_8/lib/libgo* ~/clang_libs
4 cp /usr/local/cross_armhf_clang_8/lib/libiom* ~/clang_libs

Now, you’ll need to copy clang_libs and work_examples to your RPi device. Please note that clang_libs needs to be copied only once, while work_examples, or other binary that you’ve built, needs to be transferred every time you recompile the programs on your x86-64 Debian machine.

I will assume that you’ve transferred clang_libs and work_example in the home folder of your RPi device. For example, if you are using ssh, you can copy the folders with these commands:

1 cd ~
2 scp -r work_example pi@your_rpi_device_ip:
3 scp -r clang_libs pi@your_rpi_device_ip:

don’t forget to replace your_rpi_device_ip from above with your RPi device IP!

For the remaining of this article I will assume that you are on an RPi device at a Terminal prompt.

On your RPi move the clang_libs folder to /usr/local:

1 cd ~
2 sudo mv clang_libs /usr/local

Next, add the libraries from clang_libs to the system library path (this needs to be done only once):

1 echo 'export LD_LIBRARY_PATH=/usr/local/clang_libs:$LD_LIBRARY_PATH' >> ~/.bashrc
2 source ~/.bashrc

At this point you should be able to run, on your RPi, any C or C++ program compiled with your cross Clang compiler. Try to run the four executables:

1 cd ~/work_examples
2 ./t0
3 ./t1a
4 ./t1b
5 ./t2

If you see an error about missing a shared library when you run t1b or t2 try to close and reopen your Terminal on the RPi, or logout and login again if you are using ssh.

This is what I see on my RPi:

 1 pi@raspberrypi:~/work_examples $ ./t0
 2 Hello armhf from cross Clang
 3 pi@raspberrypi:~/work_examples $ ./t1a
 4 1 2 3 4 5 6
 5 pi@raspberrypi:~/work_examples $ ./t1b
 6 1 2 3 4 5 6
 7 pi@raspberrypi:~/work_examples $ ./t2
 8 "./t1b"
 9 "./t2"
10 "./t1.cpp"
11 "./t0.c"
12 "./t0"
13 "./t2.cpp"
14 "./t1a"
15 pi@raspberrypi:~/work_examples $

If you want to learn more about programming on the Raspberry Pi, a very good book is Exploring Raspberry Pi by Derek Molloy:


Show Comments