Merge pull request #15 from marcb1/marcb/updating-docs

updating docs + k8s manifests

.env_example

@@ -1,10 +1,14 @@
-PORT=
-WEBHOOK_SECRET=
-LOG_LEVEL=
+PORT=80
+LOG_LEVEL=info
+KUBECTL_EXTERNAL_DIFF=
+#------------------------
+# the env variables are required for running k8s, check the README for deployment
 APP_ID=
-PRIVATE_KEY_PATH=
 GHE_HOST=
-ARGOCD_AUTH_TOKEN=
-ARGOCD_SERVER=
-GITHUB_TOKEN=
 GITHUB_REPO=
+GITHUB_TOKEN=
+WEBHOOK_SECRET=
+PRIVATE_KEY_PATH=
+ARGOCD_SERVER=
+ARGOCD_AUTH_TOKEN=
+#------------------------

.gitignore

@@ -4,4 +4,3 @@ gh_keys
 package-lock.json
 node_modules
 lib/
-deployment/install.yaml

Dockerfile

@@ -3,7 +3,7 @@ FROM argoproj/argocd:$ARGOCD_VERSION as argocd
 
 FROM node:11.10.1-slim
 
-RUN apt-get update && apt-get install -y git apt-utils sudo python make && \
+RUN apt-get update && apt-get install -y git apt-utils sudo python make vim procps && \
     apt-get clean && \
     rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
 
@@ -23,9 +23,13 @@ RUN groupadd -g 999 argocd && \
 RUN echo "argocd ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
 
 COPY --chown=argocd . ./
+COPY --chown=argocd deployment/diff_helper.sh /usr/local/bin/diff_helper
 #workaround https://github.com/golang/go/issues/14625
 ENV USER=argocd
 USER argocd
 
+# used by argocd cli tool, makes diff prettier for github markdown
+ENV KUBECTL_EXTERNAL_DIFF=/usr/local/bin/diff_helper
+
 # run npm as argocd user
 RUN npm install && npm run build && npm run test

README.md

@@ -50,8 +50,6 @@ These are commands that are supported by the bot.
 More commands might be added, run `argo help` on a PR, to view all supported commands.
 
 ## Deployment
-Docker based deployment is still a WIP.
-To run the bot for now follow the steps below:
 
 ### Create a Github App
 Create a new GitHub App [here](https://github.com/settings/apps/new).  
@@ -62,35 +60,35 @@ Create a new GitHub App [here](https://github.com/settings/apps/new).
 For more on creating Github apps [see](https://probot.github.io/docs/development/#manually-configuring-a-github-app)
 
 ### Update Config
-There is an `.env_example` file, that file should be renamed to `.env`. NodeJS will read that file and expose the variables to the bot.  
+There is an `.env_example` file that should be renamed to `.env`. NodeJS will read that file and expose the variables to the bot, when running locally.  
+When running in Kubernetes, there is a helper script to create k8s secrets from that file (more on this in the kubernetes deployment section).  
 Here is a description of each parameter:
 - `PORT` is the port that bot will listen on via HTTP.
-- `WEBHOOK_SECRET` is the secret configured when creating the Github app (can be left empty if no secret is specified).
 - `LOG_LEVEL` can be set to `trace`, `debug`, `info`, `warn`, `error`, or `fatal`.
+- `KUBECTL_EXTERNAL_DIFF` this is used by `argocd diff`, we pass a helper script to pretti-fy diffs posted on the PR.
 - `APP_ID` is the app id corresponding to the Github app (this is generated on app creation).
-- `PRIVATE_KEY_PATH` is the path to the private key generated for the Github app, this is usually a `.pem` file.
 - `GHE_HOST` for Github enterprise installations, specify the hostname. Otherwise leave blank, bot will use Github.com
-- `ARGOCD_AUTH_TOKEN` it is recommended to generate an automation token using the `/api/v1/projects/{project}/roles/{role}/token` API. For more information [see](https://github.com/argoproj/argo-cd/blob/master/docs/security.md#authentication)
-- `ARGOCD_SERVER`, this is the ip address/hostname of the argocd server.
-- `GITHUB_TOKEN` generate a Github token for the bot, and give it no scopes. This is just used to clone the repo.
 - `GITHUB_REPO` this is the repo that the bot will operate on.
-- `KUBECTL_EXTERNAL_DIFF` this is used by `argocd diff`, we pass a helper script to pretti-fy diffs posted on the PR.
+- `GITHUB_TOKEN` generate a Github token for the bot, and give it no scopes. This is just used to clone the repo.
+- `WEBHOOK_SECRET` is the secret configured when creating the Github app (can be left empty if no secret is specified).
+- `PRIVATE_KEY_PATH` is the path to the private key generated for the Github app, this is usually a `.pem` file.
+- `ARGOCD_SERVER`, this is the ip address/hostname of the argocd server.
+- `ARGOCD_AUTH_TOKEN` it is recommended to generate an automation token using the `/api/v1/projects/{project}/roles/{role}/token` API. For more information [see](https://github.com/argoproj/argo-cd/blob/master/docs/security.md#authentication)
 
-### Starting Server
-`npm install && npm start`
+### Kubernetes Deployment
+Docker images of `argocd-bot` are built [here](https://cloud.docker.com/repository/docker/marcb1/argocd-bot), they are provided as part of releases [here](https://github.com/marcb1/argocd-bot/releases)
 
+Check the config section above, once you have a `.env` file that's populated with the correct values run `./helper_scripts/create_kubectl_secrets.sh`.  
+This will generated a k8s secret `argocd-bot-secret` used by the deployment.  
 
-## Architecture
-See docs [here](./docs/architecture.md)
+Build manifests using `kustomize`:
+`npm run manifests`
 
+Create deployment from manifests:
+`kubectl create -f deployment/install.yaml`
+
+### Manual Deployment
+See docs [here](./docs/development.md#manual-deployment)
 
 ## Development/Contributing
 See docs [here](docs/development.md)
-
-
-## Minimal TODO
-- Deployment for Kubernetes
-
-## Future Work
-- tag releases in Github
-- look into Github deployment API

deployment/base/argocd-bot-deployment.yaml

@@ -16,62 +16,62 @@ spec:
       containers:
       - name: argocd-bot
         command: [npm, start]
-        image: argocd-bot:latest
+        image: marcb1/argocd-bot:v0.1
         env:
         - name: NODE_TLS_REJECT_UNAUTHORIZED
           value: "0"
-        - name: PRIVATE_KEY_PATH
-          value: "/data/key.pem"
-        - name: ARGO_CD_SERVER_IP
-          valueFrom:
-            secretKeyRef:
-              name: argocd-bot-secret
-              key: ARGOCD_SERVER
-        - name: ARGOCD_SERVER
+        - name: LOG_LEVEL
+          value: "info"
+        - name: PORT
+          value: "8080"
+        - name: APP_ID
           valueFrom:
             secretKeyRef:
               name: argocd-bot-secret
-              key: ARGOCD_SERVER
-        - name: ARGO_CD_API_TOKEN
+              key: APP_ID
+        # if using github.com, this is not needed
+        - name: GHE_HOST
           valueFrom:
             secretKeyRef:
               name: argocd-bot-secret
-              key: TOKEN
-        - name: ARGOCD_AUTH_TOKEN
+              key: GHE_HOST
+        - name: GITHUB_REPO
           valueFrom:
             secretKeyRef:
               name: argocd-bot-secret
-              key: TOKEN
-        - name: APP_ID
+              key: GITHUB_REPO
+        - name: GITHUB_TOKEN
           valueFrom:
             secretKeyRef:
               name: argocd-bot-secret
-              key: APP_ID
-        - name: GITHUB_TOKEN
+              key: GITHUB_TOKEN
+        - name: WEBHOOK_SECRET
           valueFrom:
             secretKeyRef:
               name: argocd-bot-secret
-              key: GITHUB_TOKEN
-        - name: GITHUB_REPO
+              key: WEBHOOK_SECRET
+        - name: PRIVATE_KEY_PATH
+          value: "/data/key.pem"
+        - name: ARGOCD_SERVER
           valueFrom:
             secretKeyRef:
               name: argocd-bot-secret
-              key: GITHUB_REPO
-        - name: WEBHOOK_SECRET
+              key: ARGOCD_SERVER
+        - name: ARGOCD_AUTH_TOKEN
           valueFrom:
             secretKeyRef:
               name: argocd-bot-secret
-              key: WEBHOOK_SECRET
+              key: ARGOCD_AUTH_TOKEN
         volumeMounts:
         - name: app-private-key
           mountPath: "/data"
           readOnly: true
         imagePullPolicy: Always
         ports:
-        - containerPort: 3000
+        - containerPort: 8080
         readinessProbe:
           tcpSocket:
-            port: 3000
+            port: 8080
           initialDelaySeconds: 5
           periodSeconds: 1
       volumes:

deployment/base/argocd-bot-service.yaml

@@ -4,9 +4,12 @@ metadata:
   labels:
     app.kubernetes.io/name: argocd-bot
   name: argocd-bot
+  annotations:
+    cloud.google.com/load-balancer-type: "Internal"
 spec:
+  type: LoadBalancer
   ports:
   - port: 80
-    targetPort: 3000
+    targetPort: 8080
   selector:
     app.kubernetes.io/name: argocd-bot

deployment/install.yaml

@@ -1,15 +1,18 @@
 apiVersion: v1
 kind: Service
 metadata:
+  annotations:
+    cloud.google.com/load-balancer-type: Internal
   labels:
     app.kubernetes.io/name: argocd-bot
   name: argocd-bot
 spec:
   ports:
   - port: 80
-    targetPort: 3000
+    targetPort: 8080
   selector:
     app.kubernetes.io/name: argocd-bot
+  type: LoadBalancer
 ---
 apiVersion: apps/v1
 kind: Deployment
@@ -33,58 +36,57 @@ spec:
         env:
         - name: NODE_TLS_REJECT_UNAUTHORIZED
           value: "0"
-        - name: PRIVATE_KEY_PATH
-          value: /data/key.pem
-        - name: ARGO_CD_SERVER_IP
-          valueFrom:
-            secretKeyRef:
-              key: ARGOCD_SERVER
-              name: argocd-bot-secret
-        - name: ARGOCD_SERVER
+        - name: LOG_LEVEL
+          value: info
+        - name: PORT
+          value: "8080"
+        - name: APP_ID
           valueFrom:
             secretKeyRef:
-              key: ARGOCD_SERVER
+              key: APP_ID
               name: argocd-bot-secret
-        - name: ARGO_CD_API_TOKEN
+        - name: GHE_HOST
           valueFrom:
             secretKeyRef:
-              key: TOKEN
+              key: GHE_HOST
               name: argocd-bot-secret
-        - name: ARGOCD_AUTH_TOKEN
+        - name: GITHUB_REPO
           valueFrom:
             secretKeyRef:
-              key: TOKEN
+              key: GITHUB_REPO
               name: argocd-bot-secret
-        - name: APP_ID
+        - name: GITHUB_TOKEN
           valueFrom:
             secretKeyRef:
-              key: APP_ID
+              key: GITHUB_TOKEN
               name: argocd-bot-secret
-        - name: GITHUB_TOKEN
+        - name: WEBHOOK_SECRET
           valueFrom:
             secretKeyRef:
-              key: GITHUB_TOKEN
+              key: WEBHOOK_SECRET
               name: argocd-bot-secret
-        - name: GITHUB_REPO
+        - name: PRIVATE_KEY_PATH
+          value: /data/key.pem
+        - name: ARGOCD_SERVER
           valueFrom:
             secretKeyRef:
-              key: GITHUB_REPO
+              key: ARGOCD_SERVER
               name: argocd-bot-secret
-        - name: WEBHOOK_SECRET
+        - name: ARGOCD_AUTH_TOKEN
           valueFrom:
             secretKeyRef:
-              key: WEBHOOK_SECRET
+              key: ARGOCD_AUTH_TOKEN
               name: argocd-bot-secret
-        image: argocd-bot:latest
+        image: marcb1/argocd-bot:v0.1
         imagePullPolicy: Always
         name: argocd-bot
         ports:
-        - containerPort: 3000
+        - containerPort: 8080
         readinessProbe:
           initialDelaySeconds: 5
           periodSeconds: 1
           tcpSocket:
-            port: 3000
+            port: 8080
         volumeMounts:
         - mountPath: /data
           name: app-private-key

docs/development.md

@@ -1,13 +1,15 @@
 # Argocd Bot Development
 
 ## Development
-This is entirely built using Javascript and a few helper scripts in bash.
+This is entirely built using Typescript and a few helper scripts in bash.
 
+### Building
+To install modules and build ts: `npm install && npm run build`
 
-## Tests
+### Tests
 The `test` folder contains all test cases, to run tests: `npm test`
 
-## Logging
+### Logging
 This uses probot logging, for further documentation [see](https://probot.github.io/docs/logging/)
 
 ## Notes
@@ -18,3 +20,10 @@ npm install  npm-check-updates
 ./node_modules/.bin/ncu -u
 npm install
 ```
+
+## Manual Deployment
+To run `argocd-bot` in this git repo, install required modules and build typescript:
+- `npm install && npm run build`
+
+### Starting Server
+`npm start`

helper_scripts/build_and_push.sh

@@ -1,9 +1,8 @@
+#!/usr/bin/env bash
 set -e
-
 # helper script to push to docker hub: https://hub.docker.com/r/marcb1/argocd-bot
 
 version=0.1
-
 docker build -f Dockerfile -t argocd-bot .
 docker tag argocd-bot marcb1/argocd-bot:v${version}
 docker push marcb1/argocd-bot:v${version}

helper_scripts/create_kubectl_secrets.sh

@@ -0,0 +1,12 @@
+#!/usr/bin/env bash
+export $(cat .env | xargs -L1)
+
+kubectl create secret generic argocd-bot-secret \
+    --from-literal=APP_ID=$APP_ID \
+    --from-literal=GHE_HOST=$GHE_HOST \
+    --from-literal=GITHUB_REPO=$GITHUB_REPO \
+    --from-literal=GITHUB_TOKEN=$GITHUB_TOKEN \
+    --from-literal=WEBHOOK_SECRET=$WEBHOOK_SECRET \
+    --from-file=key.pem=$PRIVATE_KEY_PATH \
+    --from-literal=ARGOCD_SERVER=$ARGOCD_SERVER \
+    --from-literal=ARGOCD_AUTH_TOKEN=$ARGOCD_AUTH_TOKEN

package.json

@@ -10,7 +10,7 @@
     }
   ],
   "scripts": {
-    "build": "tsc",
+    "build": "tsc && cp -r ./src/sh ./lib/",
     "test": "jest ./test/",
     "start": "probot run ./lib/index.js",
     "manifests": "kustomize build ./deployment/base > deployment/install.yaml"