DevOps

How to Delete Older Elasticsearch indices using Curator

In this tutorial, we explain how to delete older Elasticsearch indices using curator, there was a requirement in one of our project to have an opensource tool which will do log aggregation and monitoring and we got the best tool i.e., ELK stack (Elasticsearch Logstash Kibana) and it is Opensource.How to Delete Older Elasticsearch indices using Curator

If you are new to this ELK stack then check this ELK stack tutorial.

How to install Curator on Linux using pip command?

Verify Curator installed properly:

A good practice is to install on Elasticsearch machine itself.

If your log size is more and you want to keep old data for 5days as per your requirement then you need to delete old Elasticsearch indices where all logs get stored and these results free up some disk space for newly generated logs. And you will be knowing Logstash create new index every day this is default configuration.

If you don’t want to delete old indices then simple increase your disk space of Elasticsearch cluster.

Steps to delete old data/indices from Elasticsearch

This is very simple to do, follow mention steps:

Step 1: Install Curator and configure it to delete indices x days old with specific pattern.

Step 2: Now, Configure Curator

Step 3: Now we need to tell the curator what action needs to be done. Some of the action is mention below:

  • Alias
  • Allocation
  • Close
  • Cluster Routing
  • Create Index
  • Delete Indices
  • Delete Snapshots
  • Open
  • forceMerge
  • Replicas
  • Restore
  • Snapshot

In this tutorial, we will use delete indices action.

Sample Action file delete-indices.yml which will delete indices older than 5Days

Command to check what pattern are the indices using?

Step 4: Now, Goto the location where you have created “delete-indices.yml” action file and run this action file with mention curator command.

Check which all indices are going to delete with dry-run option. Dry-run option is used to test action file it will not delete the index

Curator command to delete old index

To cleanup old indices run below command:

You can also configure this in cronjob using crontab –e.

If you have sudo permission then use this crontab entry:

This cronjob run at 12:00 you can change the time as per your requirement.

Tips to change time check Screenshot:

How to Delete Older Elasticsearch indices using Curator

Comment us below if you have any queries. If you like these tutorials please share with your friends.

Check this also:

Compare ELK vs Splunk

ELK STACK ARCHITECTURE

 

Show More

Related Articles

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Close