New repo structure and Manifess

Signed-off-by: Eduardo Silva <[email protected]>

README.md

@@ -1,35 +1,26 @@
-# Help to Test!
+# Kubernetes Logging with Fluent Bit
 
-[Fluent Bit](http://fluentbit.io) v0.11 was just released at the end of March 2017 and it comes with support for Kubernetes logging needs: it can consume logs and enrich them with proper Kubernetes metadata, retrieved from the API server of course.
+[Fluent Bit](http://fluentbit.io) is a lightweight and extensible __Log Processor__ that comes with full support for Kubernetes:
 
-Our goal is to make logging cheaper in terms of memory consumption. In order to accomplish this we need to deal with different factors, not only processing incoming logs but also _formatting_ the data for our backends like Elasticsearch, which can be a little expensive as it requires a JSON representation.
+- Read Kubernetes/Docker log files from the file system or through Systemd Journal
+- Enrich logs with Kubernetes metadata
+- Deliver logs to third party storage services like Elasticsearch, InfluxDB, HTTP, etc.
 
-> Note: Fluent Bit uses a binary representation for logs, converting this to JSON requires enough memory for the process.
+This repository contains a set of Yaml files to deploy Fluent Bit which consider namespace, RBAC, Service Account, etc.
 
 ## Getting started
 
-Fluent Bit must be deployed as a DaemonSet, on that way it will be available on every node of your Kubernetes cluster.
+[Fluent Bit](http://fluentbit.io) must be deployed as a DaemonSet, so on that way it will be available on every node of your Kubernetes cluster. To get started:
 
-This repository contains two Yaml DaemonSet files:
+__FIXME__
 
-| Yaml file | Description |
-|-----------|-------------|
-| [fluent-bit-daemonset-elasticsearch](fluent-bit-daemonset-elasticsearch.yaml) | deploys a stable release of Fluent Bit. |
-
-The current DaemonSet points to this specific Docker Hub image:
-
-[0.11](https://hub.docker.com/r/fluent/fluent-bit-kubernetes-daemonset/tags/) fluent/fluent-bit-kubernetes-daemonset:0.11
-
-### Steps
-
-1. Make sure your Elasticsearch backend is running and can be reach through the hostname _elasticsearch-logging_. This value can be changed in the Yaml file
-
-2. Deploy the daemonset file from this repository:
-
-```bash
-$ kubectl apply -f fluent-bit-daemonset-elasticsearch.yaml
 ```
+$ kubectl create namespace logging
+$ kubectl create -f ./fluent-bit-service-account.yaml
+$ kubectl create -f ./fluent-bit-role.yaml
+$ kubectl create -f ./fluent-bit-role-binding.yaml
 
+```
 ## Details
 
 The default configuration of Fluent Bit makes sure of the following:

fluent-bit-role-binding.yaml

@@ -0,0 +1,12 @@
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRoleBinding
+metadata:
+  name: fluent-bit-read
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: fluent-bit-read
+subjects:
+- kind: ServiceAccount
+  name: fluent-bit
+  namespace: logging

fluent-bit-role.yaml

@@ -0,0 +1,10 @@
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: ClusterRole
+metadata:
+  name: fluent-bit-read
+rules:
+- apiGroups: [""]
+  resources:
+  - namespaces
+  - pods
+  verbs: ["get", "list", "watch"]

fluent-bit-service-account.yaml

@@ -0,0 +1,5 @@
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: fluent-bit
+  namespace: logging

output/elasticsearch/fluent-bit-ds.yaml

@@ -0,0 +1,49 @@
+apiVersion: extensions/v1beta1
+kind: DaemonSet
+metadata:
+  name: fluent-bit
+  namespace: logging
+  labels:
+    k8s-app: fluent-bit-logging
+    version: v1
+    kubernetes.io/cluster-service: "true"
+spec:
+  template:
+    metadata:
+      labels:
+        k8s-app: fluent-bit-logging
+        version: v1
+        kubernetes.io/cluster-service: "true"
+    spec:
+      containers:
+      - name: fluent-bit
+        image: fluent/fluent-bit:0.12.7
+        env:
+        - name: FLUENT_ELASTICSEARCH_HOST
+          value: "10.0.2.2"
+        - name: FLUENT_ELASTICSEARCH_PORT
+          value: "9200"
+        volumeMounts:
+        - name: varlog
+          mountPath: /var/log
+        - name: varlibdockercontainers
+          mountPath: /var/lib/docker/containers
+          readOnly: true
+        - name: fluent-bit-config
+          mountPath: /fluent-bit/etc/
+      terminationGracePeriodSeconds: 10
+      volumes:
+      - name: varlog
+        hostPath:
+          path: /var/log
+      - name: varlibdockercontainers
+        hostPath:
+          path: /var/lib/docker/containers
+      - name: fluent-bit-config
+        configMap:
+          name: fluent-bit-config
+      serviceAccountName: fluent-bit
+      tolerations:
+      - key: node-role.kubernetes.io/master
+        operator: Exists
+        effect: NoSchedule