With ssl & digest support
- The only playbok at the moment is
zookeeper.yml - Java must be installed separately on Zookeeper hosts.
javaexecutable must be in the system path.
Seems, thatJAVA_HOMEcan be specified in theroles/zookeeper/templates/zookeeper-env.sh.j2 - Distributions & certs must be placed to the
roles/zookeeper/files/directory. Its current contents is:
tree roles/zookeeper/files
roles/zookeeper/files
├── cert
│ ├── fqdn-01.pem
│ ├── fqdn-02.pem
│ ├── fqdn-03..pem
│ └── truststore.pem
└── distrib
└── apache-zookeeper-3.7.2-bin.tar.gz -> /distrib/zookeeper/apache-zookeeper-3.7.2-bin.tar.gz
One may find the zookeeper distribution in the 10.0.100.149:/distrib/zookeeper/ nfs directory, for example.
- Certificates generation
- Node cert & key file in the PEM format must have the
<inventory_hostname>.pemname truststore.pemmay contain a single root CA cert or root CA & intermediate CA certs
- Node cert & key file in the PEM format must have the
- Installing / reconfiguring
ansible-playbook zookeeper.yml
- Stopping
ansible-playbook zookeeper.yml -e 'play_action=stop'
- Starting
ansible-playbook zookeeper.yml -e 'play_action=start'
- Configs on the 1-st ZK node of the inventory
ansible -m shell -a "ls -l /opt/zookeeper/cert" $(grep -A1 -F [zookeepers inv | tail -1)
ansible -m shell -a "cat /opt/zookeeper/conf/zoo.cfg" $(grep -A1 -F [zookeepers inv | tail -1)
ansible -m shell -a "cat /opt/zookeeper/conf/zookeeper-env.sh" $(grep -A1 -F [zookeepers inv | tail -1)
- Zookeeper processes
ansible -m shell -a "ps -u zookeeper -o args= | grep 'zoo\.cfg$'" all
ansible -m shell -a "ps -fwwp \$(cat /home/zookeeper/data/zookeeper_server.pid)" all
- Log files
On the 3-rd ZK node of the inventory
ansible -m shell -a "sh -c 'tail -100 \$(ls -tr /opt/zookeeper/logs/zookeeper-zookeeper-server-*.out | tail -1)'" $(grep -A3 -F [zookeepers inv | tail -1)
- Connection & status
cert=/distrib/cert/certs/fqdn-01-03/zk/fqdn-03.crt
key=/distrib/cert/certs/fqdn-01-03/zk/fqdn-03.key
cafile=roles/zookeeper/files/cert/truststore.pem
openssl s_client -connect fqdn-01:2281 \
-cert ${cert?} \
-key ${key?} \
-CAfile ${cafile?} \
-state -quiet
echo "stat" | nc --ssl --ssl-cert ${cert?} --ssl-key ${key?} fqdn-01 2281
- ACL
The/clickhousenode can be created for Clickhouse.
Suppose it has usernamechand passwordchsecret. We compute the corresponding digest (can be computed with JAVA as well as playbook does it):
echo -n ch:chsecret | openssl dgst -binary -sha1 | openssl base64
E/Elk/liGnzgGqtkPXslt5B5a6s=
and use as follows (superuser's username and password are specified via playbook parameters, see zk_superDigest):
/opt/zookeeper/bin/zkCli.sh -server fqdn-01:2281
addauth digest super:adminsecret
create /clickhouse
setAcl -R /clickhouse digest:ch:E/Elk/liGnzgGqtkPXslt5B5a6s=:crdwa
close
Now the ch user can connect and list this node:
/opt/zookeeper/bin/zkCli.sh -server fqdn-01:2281
addauth digest ch:chsecret
ls /clickhouse
- zkCli.sh
export CLIENT_JVMFLAGS="
-Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty
-Dzookeeper.client.secure=true
-Dzookeeper.ssl.keyStore.location=${PWD}/roles/zookeeper/files/cert/fqdn-01.pem
-Dzookeeper.ssl.keyStore.type=PEM
-Dzookeeper.ssl.trustStore.location=${PWD}/roles/zookeeper/files/cert/truststore.pem
-Dzookeeper.ssl.trustStore.type=PEM
"
/opt/zookeeper/bin/zkCli.sh -server fqdn-01:2281 whoami
Auth scheme: User
x509: CN=fqdn-01
ip: 10.0.100.149
2024-07-24 13:31:58,350 [myid:] - INFO [main:o.a.z.u.ServiceUtils@45] - Exiting JVM with code 0
/opt/zookeeper/bin/zkCli.sh -server fqdn-01:2281
addauth digest ch:chsecret
[zk: fqdn-01:2281(CONNECTED) 1] ls /clickhouse
[]
[zk: fqdn-01:2281(CONNECTED) 2] getAcl /clickhouse
'digest,'ch:E/Elk/liGnzgGqtkPXslt5B5a6s=
: cdrwa
TBD