1. Topics
  2. JanusGraph Management keeps force-closed instances as "open"

JanusGraph Management keeps force-closed instances as "open"

f.gri...@...
 

Hello there,

in our team, we recently encountered a behavior of the JanusGraph management system, that we didn't expect.

Scenario:
We have some (currently 24) JanusGraph instances running in a Docker swarm. Every now and then, some of these instances are restarted, e.g. for maintenance or when healthchecks fail.

Behavior:
For us, it was expected that
mgmt.getOpenInstances()
should return a list of those 24 instances, that are currently running. This is not the case. Instead, over the course of months or even years, JanusGraph had accumulated tens of thousands of instances, that are seemingly still unclosed.

Why is this relevant?
In our case, we needed to stop all open instances in order to apply a change to a GLOBAL_OFFLINE config. Even though it was possible to close all instances via the management API, this process took about half an hour.

We thought it would be much cleaner if we found a way to automatically remove killed instances from the internal list of open instances. Does anybody know a way to accomplish this? Or does something like this require additional features in JanusGraph to be implemented?

Regards,
Florian
sparshneel chanchlani <sparshneel...@...>
 

Did you commit the management transaction ?after u force closed instances

-Sparshneel

On Wed, Aug 12, 2020, 6:02 PM <f.gri...@...> wrote:
Hello there,

in our team, we recently encountered a behavior of the JanusGraph management system, that we didn't expect.

Scenario:
We have some (currently 24) JanusGraph instances running in a Docker swarm. Every now and then, some of these instances are restarted, e.g. for maintenance or when healthchecks fail.

Behavior:
For us, it was expected that
mgmt.getOpenInstances()
should return a list of those 24 instances, that are currently running. This is not the case. Instead, over the course of months or even years, JanusGraph had accumulated tens of thousands of instances, that are seemingly still unclosed.

Why is this relevant?
In our case, we needed to stop all open instances in order to apply a change to a GLOBAL_OFFLINE config. Even though it was possible to close all instances via the management API, this process took about half an hour.

We thought it would be much cleaner if we found a way to automatically remove killed instances from the internal list of open instances. Does anybody know a way to accomplish this? Or does something like this require additional features in JanusGraph to be implemented?

Regards,
Florian

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgra...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/cb749c90-9dcc-4d23-a6b1-8fc791ae008co%40googlegroups.com.
Florian Grieskamp <f.gri...@...>
 

Did you commit the management transaction ?after u force closed instances
Which management transaction do you mean? Just to be clear, I did not force close the instances by the use of
mgmt.forceCloseInstance()

They are just closed by the container orchestrator. The situation should be about the same as if one machine unexpectedly died.
sparshneel chanchlani <sparshneel...@...>
 

Florian,
Best way is to manually close those
 You could do it using gremlin console as well. Hope this helps


On Thu, Aug 13, 2020, 10:17 AM Florian Grieskamp <f.gri...@...> wrote:
Did you commit the management transaction ?after u force closed instances
Which management transaction do you mean? Just to be clear, I did not force close the instances by the use of
mgmt.forceCloseInstance()

They are just closed by the container orchestrator. The situation should be about the same as if one machine unexpectedly died.

--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgra...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/6d274d99-021f-4e3b-a782-f1d02e7bf748o%40googlegroups.com.
1 - 4 of 4