i’m asking this probably for the second time as most resources out there are identical and I need some more answers/help to determine the proper way of improving my ES cluster performance.
We are using ES 5.4.0 and our cluster consist of client nodes, master nodes and 4 data nodes.
Also, 2 LS instances are streaming log events to ES.
My issues/questions are as follows:
- I have done most of the suggested options to increase performance but don't seem to be able to go beyond primary indexing rate of 4k events/s, and this causes a back pressure on LS. LS without the ES output plugin can do about 10k/s, but with ES plugin output enabled it drops by 50% or more.
- I changed the flush size and other settings on LS but still no big change.
- I have monitored the ES queues and some of them are barely going beyond 35.
- How much of a difference can replication set 0 make in such case ?
- Does all my none data nodes need to be set to node.ingest = false ? does this affect performance if not set to false ?
- Disks are not SSD is this a big factor in the indexing performance ?
- Does setting index.merge.scheduler.max_thread_count: 1 in my case help ?