Export data from Elasticsearch into a CSV file
This runbook aims to guide you through exporting data from Elasticsearch into a CSV file.
The CSV Export feature in Kibana is a proprietary feature of Elasticsearch that requires X-Pack to be configured and installed in our Elasticsearch cluster which is not provided and supported in the Elasticsearch that is offered by AWS.
Cloud-platform-live Elasticsearch endpoint (https://search-cloud-platform-live-dibidbfud3uww3lpxnhj2jdws4.eu-west-2.es.amazonaws.com) is accessible within the cluster.
kubectl exec in to any of the fluentd pod in logging namespace and install es2csv
kubectl -n logging exec -ti fluentd-es-xxxxx -- /bin/bash
You also need Python 2.7.x and pip
apt update apt install python-pip pip install es2csv
You can view the installation instructions here
es2csv -q 'kubernetes.namespace_name:starter-pack' -u https://<ES_endpoint>:443 -i logstash-* -f kubernetes_cluster -o kibanatest.csv --verify-certs Here, the parameters are : -q Query -u URL -i Indices -f Fields to be preserved -o Output file name
For more usage examples please refer this
Note: Once the export is completed, delete the fluentd pod you installed es2csv