Re: a problem about elasticsearch
Vincent Praveen <vincent2...@...>
Hi Jason,toggle quoted messageShow quoted text
We are also facing the same issue and we want to find out the root cause for this these are the steps we do
We ingest 1000 edges in a single commit, I understand the error comes on the ES side. We assume that the ES cannot complete the indexing of the 1000 inserted records in 30 seconds or 1000 record batch size is too large for it to process ( Note : we always had 1000 edge batches and there was no problem earlier we have about 200m+ edges inside already )
We want to know what could have triggered this message, and what steps happen in which order so we can dig deeper.
1. After we ingest the 1000 edges and run commit, ES takes over and runs the index if this step fails will the indexing not happen at all for these 1000 records ?
2. We noticed a recent slowness in our ingestion speed, could this be due to this ES index issue , if so anything we can do to overcome it ?
3. For each batch commit, does the JG wait for the ES to complete the indexing before moving on to the next batch?
4. We have multiple error messages in our logs and want to know whats the impact on the index or any possible data loss?
On Wednesday, August 8, 2018 at 10:07:03 PM UTC+8 Jason Plurad wrote: