What is Elasticsearch?
Elasticsearch is a distributed search engine based on Apache Lucene library.
It is multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents.
Elasticsearch is the central component of the ELK/EFK(Elasticsearch, Logstash/Fluentd, Kibana) Stack. It is very useful for managing logs of IT systems and applications.
What is Kibana?
Kibana is a frontend application that provides search and data visualization capabilities for data indexed in Elasticsearch.
Commonly known as the charting tool for the ELK/EFK Stack. Kibana also acts as the user interface for monitoring, managing, and securing an Elastic Stack cluster.
In this quick start demo, we are going to setup an Elasticsearch cluster on a Google Kubernetes Engine cluster.
Elasticsearch Cluster Setup
Prerequisites:
- A Kubernetes cluster with 3 nodes(For information on how to deploy a GKE cluster, see this post.)
- kubectl client library to connect to Kubernetes Cluster
- Admin privileges on Kubernetes Cluster
Connect to GKE cluster
gcloud container clusters get-credentials demo-k8s-clusterMake sure that the cluster has 3 nodes since we are going to deploy a Elasticsearch cluster with 3 pods.
cloudshell:~$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
gke-demo-cluster-demo-gke-node-pool-ce0ec65d-4zk7 Ready 4d23h v1.22.8-gke.202
gke-demo-cluster-demo-gke-node-pool-ce0ec65d-hr8h Ready 29h v1.22.8-gke.202
gke-demo-cluster-demo-gke-node-pool-ce0ec65d-sqfg Ready 3h8m v1.22.8-gke.202Create a Namespace for Elasticsearch Cluster
By running following command, we are going to create a namespace called “elk” for creating Elasticsearch Pods, Services, Storage Class and PVs.
cloudshell:~/elk$ kubectl create namespace elk
namespace/elk createdCreate a Storage Class for Persistence Storage
Since Elasticsearch needs storage persistency, we are going to use Kubernetes Statefulsets to manage pods.
Here you can find more information about Kubernetes Statefulsets.
Copy following configuration to a file called es-sc.yaml and apply it using kubectl command.
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: elastic-sc
namespace: elk
provisioner: kubernetes.io/gce-pd
volumeBindingMode: Immediate
allowVolumeExpansion: true
reclaimPolicy: Delete
parameters:
type: pd-standard
fstype: ext4
replication-type: nonecloudshell:~/elk$ kubectl apply -f es-sc.yaml
storageclass.storage.k8s.io/elastic-sc createdStatefulSet Deployment
Kubernetes StatefulSet requires a headless service to provide network identity to the pods it creates.
Copy following configuration to a file called es-service.yaml and apply it using kubectl command.
kind: Service
apiVersion: v1
metadata:
name: elasticsearch
namespace: elk
labels:
app: elasticsearch
spec:
selector:
app: elasticsearch
clusterIP: None
ports:
- port: 9200
name: rest
- port: 9300
name: inter-node
cloudshell:~/elk$ kubectl apply -f es-service.yaml
service/elasticsearch createdNow we are going to create Elasticsearch StatefulSets.
Copy following configuration to a file called es-statefulsets.yaml.
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: es-cluster
namespace: elk
spec:
serviceName: elasticsearch
replicas: 3
selector:
matchLabels:
app: elasticsearch
template:
metadata:
labels:
app: elasticsearch
spec:
containers:
- name: elasticsearch
image: elasticsearch:7.17.5
imagePullPolicy: Always
resources:
limits:
cpu: 1000m
requests:
cpu: 100m
ports:
- containerPort: 9200
name: rest
protocol: TCP
- containerPort: 9300
name: inter-node
protocol: TCP
volumeMounts:
- name: data
mountPath: /usr/share/elasticsearch/data
env:
- name: cluster.name
value: k8s-logs
- name: node.name
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: discovery.seed_hosts
value: "es-cluster-0.elasticsearch,es-cluster-1.elasticsearch,es-cluster-2.elasticsearch"
- name: cluster.initial_master_nodes
value: "es-cluster-0"
- name: ES_JAVA_OPTS
value: "-Xms512m -Xmx512m"
- name: network.host
value: "_site_"
initContainers:
- name: fix-permissions
image: busybox
command: ["sh", "-c", "chown -R 1000:1000 /usr/share/elasticsearch/data"]
securityContext:
privileged: true
volumeMounts:
- name: data
mountPath: /usr/share/elasticsearch/data
- name: increase-vm-max-map
image: busybox
command: ["sysctl", "-w", "vm.max_map_count=262144"]
securityContext:
privileged: true
- name: increase-fd-ulimit
image: busybox
command: ["sh", "-c", "ulimit -n 65536"]
securityContext:
privileged: true
volumeClaimTemplates:
- metadata:
name: data
labels:
app: elasticsearch
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: elastic-sc
resources:
requests:
storage: 100GiApply it using kubectl command.
When applied 3 init containers will run first to change permissions and kernel parameters. Each pod will have 100G persistence storage.
cloudshell:~/elk$ kubectl apply -f es-statefulsets.yaml
statefulset.apps/es-cluster createdList running pods, services, volumes using kubectl commands.
cloudshell:~/elk$ kubectl get pods -n elk
NAME READY STATUS RESTARTS AGE
es-cluster-0 1/1 Running 0 11m
es-cluster-1 1/1 Running 0 11m
es-cluster-2 1/1 Running 0 10m
cloudshell:~/elk$
cloudshell:~/elk$ kubectl get endpoints -n elk -o wide
NAME ENDPOINTS AGE
elasticsearch 172.22.1.5:9200,172.22.2.3:9200,172.22.2.4:9200 + 3 more… 21m
cloudshell:~/elk$
cloudshell:~/elk$ kubectl get pvc -n elk
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
data-es-cluster-0 Bound pvc-62de1f22-6d52-47c3-a538-7199496fee37 100Gi RWO elastic-sc 12m
data-es-cluster-1 Bound pvc-942287cc-0cfa-4c1b-9479-ae7638232b2e 100Gi RWO elastic-sc 11m
data-es-cluster-2 Bound pvc-a54edd05-e7fe-4cd0-ab23-82306209efe9 100Gi RWO elastic-sc 11mDeploying Kibana
Now let’s deploy Kibana on GKE cluster to search Elasticsearch cluster using Web interface.
Copy following configuration to a file called kibana.yaml.
apiVersion: apps/v1
kind: Deployment
metadata:
name: kb-deployment
namespace: elk
labels:
app: kibana
spec:
replicas: 1
selector:
matchLabels:
app: kibana
template:
metadata:
labels:
app: kibana
spec:
containers:
- name: kibana
image: kibana:7.17.5
imagePullPolicy: Always
resources:
limits:
cpu: 2000m
memory: "2048Mi"
requests:
cpu: 500m
memory: "1024Mi"
env:
- name: ELASTICSEARCH_URL
value: http://elasticsearch.elk.svc.cluster.local:9200
ports:
- containerPort: 5601
---
apiVersion: v1
kind: Service
metadata:
name: kb-svc
namespace: elk
labels:
app: kibana
spec:
ports:
- port: 80
targetPort: 5601
selector:
app: kibana
type: LoadBalancer
Apply above configuration using kubectl command.
This will create a Kibana deployment and a service with external load balancer to access from internet.
This configuration will point Kibana to elasticsearch.elk.svc.cluster.local:9200 which we created in earlier steps.
cloudshell:~/elk$ kubectl apply -f kibana.yaml
deployment.apps/kb-deployment created
service/kb-svc createdAfter couple of minutes you can access Kibana web UI using external load balancer IP address from GKE’s ‘Services & Ingress’ console.
We can check Elasticsearch cluster health status from Kibana console. Go to ‘Management’ -> ‘Dev Tools’ section.
Run GET request against ‘/_cluster/health’ endpoint to get Elasticsearch cluster health status. In the following screen shot you can see our ‘k8s-logs‘ status is green.
After sending some log messages using Fluentd(refer this post) we can create an index pattern and access them from ‘Analytics’ -> ‘Discover’ console.
To create index pattern go to ‘Stack Mangement’ -> ‘index pattern’ from Kibana main console, then click on ‘Create index pattern’
Since we are using Fluentd for log shipping it will create index with “logstash” prefix by default.
Give a name to index pattern, since our log indexes have “logstash” prefix to match with them we need to use “logstatsh-*” as index pattern name.
Then select “@timestamp” as Timestamp field, then click on “Create index pattern” button.
Now go to ‘Analytics’ -> ‘Discovr’ console to see and search log messages stored in Elasticsearch cluster.
Conclusion
In this quick start demo we have deployed Elasticsearch cluster and Kibana on a GKE cluster and accessed Kibana web UI and verified Elasticsearch cluster health status. We have also created a index pattern and access logs stored in Elasticsearch cluster. You can find more information about Elastic stack in official documentation.