Use of mgmt.setTTL to expire edges


ke...@...
 

Hi,

I am experimenting with using the mgmt.setTTL option to automatically expire edges after 7 days - which is working well and generating much less overhead then trying to execute a task that finds and drops the edges.  However this option does not drop the entries from external indexes in Elasticsearch. I imaging that this is because the setTTL is being passed directly to the storage backend to handle (Cassandra in my case) and so only happens there.

Is this working as expected? Are there plans to sync the deletions in Elasticsearch? Or should I plan to manually run a purge from Elasticsearch in line with the 7 day expiry in Cassandra?

Thanks for any of your views!

Kevin


Jerry He <jerr...@...>
 

JanusGraph's documentation [1] shows both ES and Solr have document index TTL.

But looking at the code, I only see Solr has it on:
    private static final IndexFeatures SOLR_FEATURES = new IndexFeatures.Builder()
            .supportsDocumentTTL()

The code to support sync of entry TTL to index document TTL is also in place for Solr (and for ES as well except the last mile).
We need to look into why it is missing there.

[1]  http://docs.janusgraph.org/latest/index-backends.html

Thanks,

Jerry


On Monday, July 3, 2017 at 8:41:50 AM UTC-7, k...@... wrote:
Hi,

I am experimenting with using the mgmt.setTTL option to automatically expire edges after 7 days - which is working well and generating much less overhead then trying to execute a task that finds and drops the edges.  However this option does not drop the entries from external indexes in Elasticsearch. I imaging that this is because the setTTL is being passed directly to the storage backend to handle (Cassandra in my case) and so only happens there.

Is this working as expected? Are there plans to sync the deletions in Elasticsearch? Or should I plan to manually run a purge from Elasticsearch in line with the 7 day expiry in Cassandra?

Thanks for any of your views!

Kevin


Samik Raychaudhuri <sam...@...>
 

Hi,
Curious: when an edge is deleted based on TTL, are the nodes that are connected only to those edges also deleted? Or do I need to specify TTL for nodes as well?
Same question the other way round: if some nodes are deleted through TTL, would the incident edges be deleted as well?
Regards.
-Samik

On 04-Jul-17 11:49 PM, Jerry He wrote:

JanusGraph's documentation [1] shows both ES and Solr have document index TTL.

But looking at the code, I only see Solr has it on:
    private static final IndexFeatures SOLR_FEATURES = new IndexFeatures.Builder()
            .supportsDocumentTTL()

The code to support sync of entry TTL to index document TTL is also in place for Solr (and for ES as well except the last mile).
We need to look into why it is missing there.


Jerry


On Monday, July 3, 2017 at 8:41:50 AM UTC-7, k...@... wrote:
Hi,

I am experimenting with using the mgmt.setTTL option to automatically expire edges after 7 days - which is working well and generating much less overhead then trying to execute a task that finds and drops the edges.  However this option does not drop the entries from external indexes in Elasticsearch. I imaging that this is because the setTTL is being passed directly to the storage backend to handle (Cassandra in my case) and so only happens there.

Is this working as expected? Are there plans to sync the deletions in Elasticsearch? Or should I plan to manually run a purge from Elasticsearch in line with the 7 day expiry in Cassandra?

Thanks for any of your views!

Kevin
--
You received this message because you are subscribed to the Google Groups "JanusGraph users list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgra...@....
For more options, visit https://groups.google.com/d/optout.


Jerry He <jerr...@...>
 

Please refer to the documentation here: 
http://docs.janusgraph.org/latest/advanced-schema.html

Thanks,

Jerry


On Wednesday, July 5, 2017 at 4:33:57 AM UTC-7, Samik R wrote:
Hi,
Curious: when an edge is deleted based on TTL, are the nodes that are connected only to those edges also deleted? Or do I need to specify TTL for nodes as well?
Same question the other way round: if some nodes are deleted through TTL, would the incident edges be deleted as well?
Regards.
-Samik

On 04-Jul-17 11:49 PM, Jerry He wrote:
JanusGraph's documentation [1] shows both ES and Solr have document index TTL.

But looking at the code, I only see Solr has it on:
    private static final IndexFeatures SOLR_FEATURES = new IndexFeatures.Builder()
            .supportsDocumentTTL()

The code to support sync of entry TTL to index document TTL is also in place for Solr (and for ES as well except the last mile).
We need to look into why it is missing there.


Jerry


On Monday, July 3, 2017 at 8:41:50 AM UTC-7, k...@... wrote:
Hi,

I am experimenting with using the mgmt.setTTL option to automatically expire edges after 7 days - which is working well and generating much less overhead then trying to execute a task that finds and drops the edges.  However this option does not drop the entries from external indexes in Elasticsearch. I imaging that this is because the setTTL is being passed directly to the storage backend to handle (Cassandra in my case) and so only happens there.

Is this working as expected? Are there plans to sync the deletions in Elasticsearch? Or should I plan to manually run a purge from Elasticsearch in line with the 7 day expiry in Cassandra?

Thanks for any of your views!

Kevin
--
You received this message because you are subscribed to the Google Groups "JanusGraph users list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-use...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.