Skip to content

Installing gcc 10 and clang 10 on Linux (dependencies for libpy and the libpy wrapper for simdjson)

Jump to bottom Edit New page
Louis Maddox edited this page Jul 16, 2020 · 2 revisions

gcc and clang need to be installed for Python tools like libpy and by extension libpy-simdjson (for fast JSON parsing)

Clang is the simpler to install if you just want to satisfy the libpy dependency, but you need both for libpy-simdjson

clang

Clang was easy enough to install, simply download it from their website and run tar -xJf to extract the (unusual) .tar.xz archive and then cp -R /usr/local/ to place each of the subdirectories of the extracted archive into the subdirectories of /usr/local/ with the same directory names: e.g. the clang-*/lib/{files} become /usr/local/lib/{files} etc.

I followed this guidance (replacing the * with tab completion, and using wget).

wget <clang-binaries-tarball-url> #  or `curl -O <url>`
tar xf clang*
cd clang*
sudo cp -R * /usr/local/

The clang precompiled binaries are available here under "Pre-Built Binaries" (only for version 18.04 for Ubuntu at the time of writing).

gcc

  • apt only has version 7.4 of gcc, and the requirements for libpy are gcc >= 9
    • apt show gcc will verify this

Version 10 (released in May 2020) can be installed as follows.

Firstly, the PPA involved here has a test branch, the ppa branch packages:

updates from release branches (e.g. Ubuntu 14.04 LTS released with gcc-4.8.2 and the PPA has gcc-4.8.3).

I've not yet upgraded to Mint 20 (with Ubuntu 20.04) so I need the test branch of the PPA instead, which has a gcc-10 package for Ubuntu 18.04. Check for your system at the PPA links

The official guidance is posted at the PPA's reference URL, https://wiki.ubuntu.com/ToolChain#GCC, which says that:

The default (recommended) GCC version is packaged as the gcc-defaults package, building binary packages gcc, g++, gfortran, etc.

See Linuxize blog if you want to manage multiple versions for development.

Linuxize recommends installing software-properties-common:

sudo apt install software-properties-common

This software provides an abstraction of the used apt repositories. It allows you to easily manage your distribution and independent software vendor software sources.

I don't think this is necessary, see here for further discussion.

To install version 10 (presuming you're on a distro which didn't package gcc-9 or higher, else why would you be following this step), add the test branch of the PPA as described earlier:

sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt install gcc-10 g++-10

Then you need to set it as the default gcc: I made this by editing the version of the command given on the Linuxize blog post (linked above), where it was written for versions 7 to 9.

  • This command sets gcc as an alias for version 10, i.e. gcc-10, and likewise gcov-10 as the default gcov coverage testing version, and g++-10 as the default g++.
  • The --install flag means that gcc is the "master link"

--install link name path priority [--slave link name path]...

Add a group of alternatives to the system. link is the generic name for the master link, name is the name of its symlink in the alternatives directory, and path is the alternative being introduced for the master link. The arguments after --slave are the generic name, symlink name in the alternatives directory and the alternative path for a slave link. Zero or more --slave options, each followed by three arguments, may be specified
...

sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 100 --slave /usr/bin/g++ g++ /usr/bin/g++-10 --slave /usr/bin/gcov gcov /usr/bin/gcov-10

The master/slave terms mean that there is a synced group and that the one passed with the master flag will change the associated ones (e.g. removal of the master link will remove the rest).

libpy

Now pip install libpy works!

libpy-simdjson

Now pip install libpy-simdjson works!

A quick tab completion look around their namespaces:

>>> import libpy
>>> libpy.
libpy.VersionInfo(  libpy.ctypes        libpy.os            libpy.version_info
>>> libpy.ctypes
libpy.ctypes
>>> libpy.ctypes.
libpy.ctypes.ARRAY(                  libpy.ctypes.PYFUNCTYPE(             libpy.ctypes.c_bool(
libpy.ctypes.c_int64(                libpy.ctypes.c_uint16(               libpy.ctypes.c_wchar_p(
libpy.ctypes.pydll
libpy.ctypes.ArgumentError(          libpy.ctypes.PyDLL(                  libpy.ctypes.c_buffer(
libpy.ctypes.c_int8(                 libpy.ctypes.c_uint32(               libpy.ctypes.cast(
libpy.ctypes.pythonapi
libpy.ctypes.Array(                  libpy.ctypes.RTLD_GLOBAL             libpy.ctypes.c_byte(
libpy.ctypes.c_long(                 libpy.ctypes.c_uint64(               libpy.ctypes.cdll
libpy.ctypes.resize(
libpy.ctypes.BigEndianStructure(     libpy.ctypes.RTLD_LOCAL              libpy.ctypes.c_char(
libpy.ctypes.c_longdouble(           libpy.ctypes.c_uint8(
libpy.ctypes.create_string_buffer(   libpy.ctypes.set_errno(
libpy.ctypes.CDLL(                   libpy.ctypes.SetPointerType(         libpy.ctypes.c_char_p(
libpy.ctypes.c_longlong(             libpy.ctypes.c_ulong(
libpy.ctypes.create_unicode_buffer(  libpy.ctypes.sizeof(
libpy.ctypes.CFUNCTYPE(              libpy.ctypes.Structure(              libpy.ctypes.c_double(
libpy.ctypes.c_short(                libpy.ctypes.c_ulonglong(            libpy.ctypes.get_errno(
libpy.ctypes.string_at(
libpy.ctypes.DEFAULT_MODE            libpy.ctypes.Union(                  libpy.ctypes.c_float(
libpy.ctypes.c_size_t(               libpy.ctypes.c_ushort(               libpy.ctypes.memmove(
libpy.ctypes.wstring_at(
libpy.ctypes.LibraryLoader(          libpy.ctypes.addressof(              libpy.ctypes.c_int(
libpy.ctypes.c_ssize_t(              libpy.ctypes.c_void_p(               libpy.ctypes.memset(
libpy.ctypes.LittleEndianStructure(  libpy.ctypes.alignment(              libpy.ctypes.c_int16(
libpy.ctypes.c_ubyte(                libpy.ctypes.c_voidp(                libpy.ctypes.pointer(
libpy.ctypes.POINTER(                libpy.ctypes.byref(                  libpy.ctypes.c_int32(
libpy.ctypes.c_uint(                 libpy.ctypes.c_wchar(                libpy.ctypes.py_object(
>>> libpy.ctypes.
KeyboardInterrupt
>>> import libpy_simdjson
>>> libpy_simdjson.
libpy_simdjson.Array(   libpy_simdjson.Object(  libpy_simdjson.Parser(  libpy_simdjson.libpy
libpy_simdjson.load(    libpy_simdjson.loads(   libpy_simdjson.parser   
>>> libpy_simdjson.parser.
libpy_simdjson.parser.Array(   libpy_simdjson.parser.Object(  libpy_simdjson.parser.Parser(
libpy_simdjson.parser.load(    libpy_simdjson.parser.loads(   
>>> libpy_simdjson.libpy.
libpy_simdjson.libpy.VersionInfo(  libpy_simdjson.libpy.ctypes        libpy_simdjson.libpy.os
libpy_simdjson.libpy.version_info  
>>> libpy_simdjson.libpy.

Main page

Add a custom sidebar
Clone this wiki locally