Cowrie fails to deploy

[himuura]

Hey there! I've been having issues with Cowrie while using the deploy script from MHN.
Changed pip references to pip3 and virtualenv as well. Supervisorctl shows nothing and MHN doesn't even register the probe.
Here's my deploy script:

#!/bin/bash

set -e
set -x

if [ $# -ne 2 ]
then
echo "Wrong number of arguments supplied."
echo "Usage: $0 <server_url> <deploy_key>."
exit 1
fi

apt-get update
apt-get install -y python

server_url=$1
deploy_key=$2

apt-get update
apt-get -y install python-dev git supervisor authbind openssl python3-virtualenv build-essential libgmp-dev libmpfr-dev libmpc-dev libssl-dev python3-pip libffi-dev

pip3 install gmpy
pip install -U supervisor
/etc/init.d/supervisor start || true

sed -i 's/#Port/Port/g' /etc/ssh/sshd_config
sed -i 's/Port 22$/Port 2222/g' /etc/ssh/sshd_config
service ssh restart
useradd -d /home/cowrie -s /bin/bash -m cowrie -g users

cd /opt
git clone https://github.com/micheloosterhof/cowrie.git cowrie
cd cowrie

Most recent known working version

git checkout 34f8464

Config for requirements.txt

cat > /opt/cowrie/requirements.txt <=17.1.0
cryptography>=2.1
configparser
pyopenssl
pyparsing
packaging
appdirs>=1.4.0
pyasn1_modules
attrs
service_identity
python-dateutil
tftpy
bcrypt
EOF

virtualenv --python=python3 cowrie-env #env name has changed to cowrie-env on latest version of cowrie
source cowrie-env/bin/activate

without the following, i get this error:

Could not find a version that satisfies the requirement csirtgsdk (from -r requirements.txt (line 10)) (from versions: 0.0.0a5, 0.0.0a6, 0.0.0a5.linux-x86_64, 0.0.0a6.linux-x86_64, 0.0.0a3)

pip install csirtgsdk==0.0.0a6
pip install -r requirements.txt

Register sensor with MHN server.

wget $server_url/static/registration.txt -O registration.sh
chmod 755 registration.sh

Note: this will export the HPF_* variables

. ./registration.sh $server_url $deploy_key "cowrie"

cd etc
cp cowrie.cfg.dist cowrie.cfg
sed -i 's/hostname = svr04/hostname = server/g' cowrie.cfg
sed -i 's/listen_endpoints = tcp:2222:interface=0.0.0.0/listen_endpoints = tcp:22:interface=0.0.0.0/g' cowrie.cfg
sed -i 's/version = SSH-2.0-OpenSSH_6.0p1 Debian-4+deb7u2/version = SSH-2.0-OpenSSH_6.7p1 Ubuntu-5ubuntu1.3/g' cowrie.cfg
sed -i 's/#[output_hpfeeds]/[output_hpfeeds]/g' cowrie.cfg
sed -i '/[output_hpfeeds]/!b;n;cenabled = true' cowrie.cfg
sed -i "s/#server = hpfeeds.mysite.org/server = $HPF_HOST/g" cowrie.cfg
sed -i "s/#port = 10000/port = $HPF_PORT/g" cowrie.cfg
sed -i "s/#identifier = abc123/identifier = $HPF_IDENT/g" cowrie.cfg
sed -i "s/#secret = secret/secret = $HPF_SECRET/g" cowrie.cfg
sed -i 's/#debug=false/debug=false/' cowrie.cfg
cd ..

chown -R cowrie:users /opt/cowrie/
touch /etc/authbind/byport/22
chown cowrie /etc/authbind/byport/22
chmod 770 /etc/authbind/byport/22

start.sh is deprecated on new Cowrie version and substituted by "bin/cowrie [start/stop/status]"

sed -i 's/AUTHBIND_ENABLED=no/AUTHBIND_ENABLED=yes/' bin/cowrie
sed -i 's/DAEMONIZE=""/DAEMONIZE="-n"/' bin/cowrie

Config for supervisor

cat > /etc/supervisor/conf.d/cowrie.conf <<EOF
[program:cowrie]
command=/opt/cowrie/bin/cowrie start
directory=/opt/cowrie
stdout_logfile=/opt/cowrie/var/log/cowrie/cowrie.out
stderr_logfile=/opt/cowrie/var/log/cowrie/cowrie.err
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=cowrie
EOF

supervisorctl update

What am i missing? The default script has 3 broken dependencies i think, python-virtualenv, gmpy and one other that i dont recall...any help?

[himuura]

Note, selecting 'python-dev-is-python2' instead of 'python-dev'
Package python-virtualenv is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'python-virtualenv' has no installation candidate
E: Unable to locate package python-gmpy2

Here's the original deploy script result. Ubuntu 20.04 Server.

[himuura]

After some tinkering, i got this:
python -c 'import json;obj=json.load(file("/tmp/deploy.json"));print obj["uuid"]'
File "", line 1
import json;obj=json.load(file("/tmp/deploy.json"));print obj["uuid"]
^
SyntaxError: invalid syntax
(the arrow is on the last obj)

[wrharding]

I just want to note that this project is not capable of using Python >2.7 in a number of locations. You've gone outside the norm by utilizing cowrie on Ubuntu 20.04 and with Python 3. This version of cowrie is utilized because it supports Python 2.7 still:

mhn/scripts/deploy_cowrie.sh

Line 35 in d15325c

git checkout 34f8464

[himuura]

Well, it "mostly" works until now...just this last part of print obj...it breaks here. the rest i think is pretty much straightforward, it's just the HPF variables and supervisorctl...any help on this?

[wrharding]

I changed the following and was able to successfully install cowrie on Ubuntu 20.04, but it's stuck in a "STARTING" state:

mhn/server/mhn/static/registration.txt

Line 36 in d15325c

uuid=$(python -c 'import json;obj=json.load(file("/tmp/deploy.json"));print obj["uuid"]')

python -c 'import json;f=open("/tmp/deploy.json");obj=json.load(f);print(obj["uuid"]);'

mhn/server/mhn/static/registration.txt

Line 52 in d15325c

export HPF_IDENT=$(python -c 'import json;obj=json.load(file("/tmp/deploy.json"));print obj["identifier"]')

python -c 'import json;f=open("/tmp/deploy.json");obj=json.load(f);print(obj["identifier"]);'

mhn/server/mhn/static/registration.txt

Line 53 in d15325c

export HPF_SECRET=$(python -c 'import json;obj=json.load(file("/tmp/deploy.json"));print obj["secret"]')

python -c 'import json;f=open("/tmp/deploy.json");obj=json.load(f);print(obj["secret"]);'

stuck :(

root@cowrie-2004:~# supervisorctl status
cowrie                           STARTING 

followed by a fail

root@cowrie-2004:~# supervisorctl status
cowrie                           BACKOFF   Exited too quickly (process log may have details)

I did this all by making a new registration.txt file and updating this part of the deploy_cowrie.sh script:

mhn/scripts/deploy_cowrie.sh

Line 62 in d15325c

wget $server_url/static/registration.txt -O registration.sh

After putting some time into this issue, I would suggest just using Ubuntu 18.04 since that is verified to work with MHN. The deploy_cowrie.sh script hasn't been updated in years so it's not the project's fault for this deployment failure, but some underlying system or library. If you can identify exactly what is causing cowrie to fail in this scenario then it would help the effort to move parts of MHN to Python 3.

[himuura]

@wrharding by all means mate! My "fault" for using ubuntu 20.04, not the MHN team! Yeah, i managed to do some work and figured it was something to do with the python command on the registration.txt file. Will try the ubuntu 16.04 with cowrie to see if it eases my pain!
Thank you again for all your help mate, you rock!

[himuura]

Well, ubuntu 16.04 isn't much better...

Collecting setuptools (from supervisor)
Downloading https://files.pythonhosted.org/packages/69/77/aee1ecacea4d0db740046ce1785e81d16c4b1755af50eceac4ca1a1f8bfd/setuptools-60.5.0.tar.gz (2.3MB)
100% |████████████████████████████████| 2.3MB 119kB/s
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "setuptools/init.py", line 8, in
import _distutils_hack.override # noqa: F401
File "_distutils_hack/override.py", line 1, in
import('_distutils_hack').do_override()
File "_distutils_hack/init.py", line 71, in do_override
ensure_local_distutils()
File "_distutils_hack/init.py", line 55, in ensure_local_distutils
importlib.import_module('distutils')
File "/usr/lib/python2.7/importlib/init.py", line 37, in import_module
import(name)
AttributeError: DistutilsMetaFinder instance has no attribute 'find_module'

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-W6SX8t/setuptools/

[premaldmehta]

Hi , Did you get solution of this?. I am stuck with the same error.