Export data from Elasticsearch into a CSV file
This runbook aims to guide you through exporting data from Elasticsearch into a CSV file.
The CSV Export feature in Kibana is a proprietary feature of Elasticsearch that requires X-Pack to be configured and installed in our Elasticsearch cluster which is not provided and supported in the Elasticsearch that is offered by AWS.
The workaround, is using es2csv, which is a third party tool that saves the search result to a CSV file. Please refer this blog page for more information.
Cloud-platform-live Elasticsearch endpoint (https://search-cloud-platform-live-dibidbfud3uww3lpxnhj2jdws4.eu-west-2.es.amazonaws.com) is accessible within the cluster.
kubectl exec in to any of the fluentd pod in logging namespace and install es2csv
kubectl -n logging exec -ti fluentd-es-xxxxx -- /bin/bash
You also need Python 2.7.x and pip
apt update apt install python-pip pip install es2csv
You can view the installation instructions here
es2csv -q 'kubernetes.namespace_name:starter-pack' -u https://<ES_endpoint>:443 -i logstash-* -f kubernetes_cluster -o kibanatest.csv --verify-certs Here, the parameters are : -q Query -u URL -i Indices -f Fields to be preserved -o Output file name
For more usage examples please refer this
Note: Once the export is completed, delete the fluentd pod you installed es2csv