Skip to content
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.

Commit 3f5e402

Browse files
committedAug 14, 2022
[test][bionic] Force python3 and utf-8 encoding
bionic's /usr/bin/python is python2. Encoding and python2/python3 is.... complicated. Those tests already run on py3 which is the default for Ubuntu Focal and Fedora 34. This change symlinks /usr/local/bin/python to python3 within the Bionic container and changes all tools to use "#!/usr/bin/env python" shebang instead of "#!/usr/bin/python" which allows to use the default python set by the environment. To add insult to injury. Python 3.6 still needs a bit of help to default to UTF-8 which can be achieved with the env var `PYTHONIOENCODING` to `utf-8`. Since python 3.7, [UTF-8 mode is enabled by default when the locale is C or POSIX](https://docs.python.org/3/whatsnew/3.7.html#whatsnew37-pep540). Some more details about this problem can be found here: https://vstinner.github.io/posix-locale.html#misconfigured-locales-in-docker-images Illustration: ``` root@bionic:/# locale LANG= LANGUAGE= LC_CTYPE="POSIX" LC_NUMERIC="POSIX" LC_TIME="POSIX" LC_COLLATE="POSIX" LC_MONETARY="POSIX" LC_MESSAGES="POSIX" LC_PAPER="POSIX" LC_NAME="POSIX" LC_ADDRESS="POSIX" LC_TELEPHONE="POSIX" LC_MEASUREMENT="POSIX" LC_IDENTIFICATION="POSIX" LC_ALL= root@bionic:/# python2 Python 2.7.17 (default, Jul 1 2022, 15:56:32) [GCC 7.5.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> print("%s" % u'\xb7') Traceback (most recent call last): File "<stdin>", line 1, in <module> UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 0: ordinal not in range(128) >>> root@bionic:/# python3 Python 3.6.9 (default, Jun 29 2022, 11:45:57) [GCC 8.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> print("%s" % u'\xb7') Traceback (most recent call last): File "<stdin>", line 1, in <module> UnicodeEncodeError: 'ascii' codec can't encode character '\xb7' in position 0: ordinal not in range(128) >>> root@bionic:/# PYTHONIOENCODING=utf-8 python3 Python 3.6.9 (default, Jun 29 2022, 11:45:57) [GCC 8.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> print("%s" % u'\xb7') · ``` On Focal: ``` root@focal:/# locale LANG= LANGUAGE= LC_CTYPE="POSIX" LC_NUMERIC="POSIX" LC_TIME="POSIX" LC_COLLATE="POSIX" LC_MONETARY="POSIX" LC_MESSAGES="POSIX" LC_PAPER="POSIX" LC_NAME="POSIX" LC_ADDRESS="POSIX" LC_TELEPHONE="POSIX" LC_MEASUREMENT="POSIX" LC_IDENTIFICATION="POSIX" LC_ALL= root@focal:/# python3 Python 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> print("%s" % u'\xb7') · ```
1 parent 1a3f8c2 commit 3f5e402

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

98 files changed

+101
-99
lines changed
 

‎docker/Dockerfile.tests

+1
Original file line numberDiff line numberDiff line change
@@ -72,3 +72,4 @@ RUN wget -O ruby-install-0.7.0.tar.gz \
7272

7373
RUN ruby-install --system ruby 2.6.0 -- --enable-dtrace
7474
RUN if [ ! -f "/usr/bin/python" ]; then ln -s /bin/python3 /usr/bin/python; fi
75+
RUN if [ ! -f "/usr/local/bin/python" ]; then ln -s /usr/bin/python3 /usr/local/bin/python; fi

‎tests/wrapper.sh.in

+4-3
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ kind=$1; shift
99
cmd=$1; shift
1010

1111
PYTHONPATH=@CMAKE_BINARY_DIR@/src/python/bcc-python3
12+
PYTHONIOENCODING=utf-8
1213
LD_LIBRARY_PATH=@CMAKE_BINARY_DIR@:@CMAKE_BINARY_DIR@/src/cc
1314

1415
ns=$name
@@ -32,15 +33,15 @@ function ns_run() {
3233
sudo ip netns exec $ns ethtool -K eth0 tx off
3334
sudo ip addr add dev $ns.out 172.16.1.1/24
3435
sudo ip link set $ns.out up
35-
sudo --preserve-env=PYTHON_TEST_LOGFILE env PYTHONPATH=$PYTHONPATH LD_LIBRARY_PATH=$LD_LIBRARY_PATH ip netns exec $ns $cmd "$@"
36+
sudo --preserve-env=PYTHON_TEST_LOGFILE env PYTHONIOENCODING=$PYTHONIOENCODING PYTHONPATH=$PYTHONPATH LD_LIBRARY_PATH=$LD_LIBRARY_PATH ip netns exec $ns $cmd "$@"
3637
return $?
3738
}
3839
function sudo_run() {
39-
sudo --preserve-env=PYTHON_TEST_LOGFILE env PYTHONPATH=$PYTHONPATH LD_LIBRARY_PATH=$LD_LIBRARY_PATH $cmd "$@"
40+
sudo --preserve-env=PYTHON_TEST_LOGFILE env PYTHONIOENCODING=$PYTHONIOENCODING PYTHONPATH=$PYTHONPATH LD_LIBRARY_PATH=$LD_LIBRARY_PATH $cmd "$@"
4041
return $?
4142
}
4243
function simple_run() {
43-
PYTHONPATH=$PYTHONPATH PYTHON_TEST_LOGFILE=$PYTHON_TEST_LOGFILE LD_LIBRARY_PATH=$LD_LIBRARY_PATH $cmd "$@"
44+
PYTHONIOENCODING=$PYTHONIOENCODING PYTHONPATH=$PYTHONPATH PYTHON_TEST_LOGFILE=$PYTHON_TEST_LOGFILE LD_LIBRARY_PATH=$LD_LIBRARY_PATH $cmd "$@"
4445
return $?
4546
}
4647

0 commit comments

Comments
 (0)
Please sign in to comment.