Elasticsearch — How to Optimize Index and Search Performance

Index performance tuning

1. Refresh interval
   Tune refresh interval according to your search requirements.
   Example: If you don’t have a use case to read data from index in near-real time, change refresh interval to >1sec (default)
2. Disable replicas/start with small
   Disabling replicas can greatly improves indexing performance but this not always possible because of the HA requirements but we can always start with small number and increase this values depending on search/indexing use case.
   Example: Ideal use case will be write heavy indexes with less to no reads. In this case you can adjust your replicas to minimum.
3. _id filed
   Don’t use custom values for _id field as this can lead to read before write to validate the uniqueness of of the _id. It’s highly recommended to leave this value to Elasticsearch unless there is strong reason.
4. Use SSD’s
   Avoid NAS/SAN based storage and HDD.
5. Use bulk API
   Use bulk API to load a bunch of documents to increase the throughput. Make sure to check the limitations of bulk load before using.
6. Use wait_for/avoid explicit refreshes
   If you have a requirement to read the documents immediately, don’t use explicit refreshes. Instead use wait_for while indexing the document to allow the refresh to trigger at set interval.
7. Index mapping
   Choose your index fields carefully, don’t use text (default) filed if they are not used in search. This saves space (to store inverted indexes) and unnecessary analysis cost.
   Example: Avoid using text field for ip addresses.
   Recommendation: Always pre-define your mapping in the form of index templates or use dynamic mapping techniques (if you’re unsure about the fields)
8. Choose analysers carefully
   Read analysers section of Elasticsearch documentation before choosing finalising the analyser for your use case. Analysers like ngrams utilises significant amount of resources and slow down the indexing speed.

Search performance tuning

The main reason for slow search performance could be related to queries from application and cluster configuration. In this section we will focus on some of the points which we can tune to reduce the search latency.

• Small shards

Having small shards could cause lot of network roundtrips and as a result which can impact your search performance even for a small datasets.
Recommendation:
It’s fine to leave number of shards for a small data set with 1 and increase replicas count > 1 to improve search performance.

number_of_shards is fixed and cannot be changed later without reindexing, so keep this in mind before finalising on this count.

• Shards

Having right number of shards is very important for search performance and also to balance your cluster performance. As explained above, having too many small shards or shard with >50GB size is not recommended. The optimal size of a shard is between 30–50GB but this also depends on cluster capacity (nodes capacity).

To avoid over sharding or under sharding, Always estimate your storage requirements and size your shards based on projected data growth.
Example:
If you it’s expected to have 100Gb of accumulated data per month/day, it’s reasonable to create that respective index with 2 shards.

• Filters

Use search filters where ever possible to cache your search results. Repeated search results would get benefited with cached results hence less IO.

• Wildcard/Regex

Avoid wildcard/regex queries specially leading wildcard searches.

1. Wildcard
   Avoid beginning patterns with * or ?. This can increase the iterations needed to find matching terms and slow search performance.
   https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-wildcard-query.html#wildcard-query-field-params
2. Regex
   The performance of the regexp query can vary based on the regular expression provided. To improve performance, avoid using wildcard patterns, such as .* or .*?+, without a prefix or suffix.
   https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-regexp-query.html#regexp-query-field-params

• Size parameter/max_result_window

Always limit your query hits by small size values. Assigning huge value to size parameter cause Elasticsearch to compute large amount of documents/hits.
Recommendation:
Limit size parameter value to small values and use batches instead of single query with huge value limit.

• Heavy aggregations

Avoid performing heavy aggregations with huge size limits. As explained above number of hits performed by single query can impact search performance.

• Routing

Use custom routing while indexing your data with a specific custom filed to route your request to specific host in the cluster to avoid scattered searches. This will greatly improves your search performance.

• Force Merge

Use force merge api to merge all the segments of read only indices and reduce number of shards (if required). This would greatly help search performance of the queries by reducing number of IO’s involved.

• ILM (hot-warm-cold-delete)

It’s a best practice to automate index life cycle management with ILM policy to move read only indices to commodity hardware.

• Adaptive replica selection

Use cluster.routing.use_adaptive_replica_selection setting on clusters with versions 6.1 and above to re-route request from overloaded nodes.

• Slow search logs

Having slow query log enabled at index level might not contribute directly to improve the search performance but this is the only source of truth to identify rogue queries. So, always enable slow search logging with appropriate SLA’s.

• Avoid frequent updates

Frequent updates can mark old records as deleted and creates lot of garbage across several segment files and these files may not get merged all the time to cleanup the garbage. As a results search must go through several segment files an it can slow down the search process.
Recommendation:
If possible collect all the updates in application logic and send consolidated results in intervals to avoid frequent updates.

Reindexing tuning

In this section, we will see how to improve reindexing time.

• Disable replicas and tune/disable refresh interval

Disable replicas and tune refresh interval when creating a new index from scratch if it’s not serving any live traffic. Replica count and refresh interval settings can be adjusted dynamically at the end of re-indexing.

• Disable merge throttling

Merge throttling setting can be disabled to speed up reindexing process

• Increase index buffer size

By default 10% of the heap size is allocated for all shards in the cluster and every shard has its indexing buffer shared from 10% of the heap. Having more memory allocated to this buffer can reduce number segments generated from reindexing hence less IO activity with improved indexing performance

• Use reindex API

Use reindex API to change analysers or to use custom routing options