Potential complexities of making secondary persistence atomic

florian.caesar <florian.caesar@...>


Currently JanusGraph's philosophy is to fail transactions only if the primary persistence fails and allow failures in secondary persistence.
According to https://docs.janusgraph.org/advanced-topics/recovery/#transaction-failure, the recommended approach for dealing with secondary persistence failures is maintaining and continuously recovering from a write-ahead log.
This results in temporary inconsistencies between primary storage and mixed indices.

For my use case, potential inconsistencies are not acceptable. Currently there is no way of making JanusGraph fail a transaction if either primary or secondary persistence fails. Looking at the source code (starting at https://github.com/JanusGraph/janusgraph/blob/a73d39d8a071f8d5340aede4917653629ab595cb/janusgraph-core/src/main/java/org/janusgraph/graphdb/database/StandardJanusGraph.java#L743), it should be possible to re-order persistence to commit both secondary and primary storage at the same time. Naturally there is some cleanup to do in case of transaction failure, but it seems manageable to me.

What are the potential complexities of pursuing this? How does this mess with JanusGraph's logic & assumptions in other places?

If it seems fine, it should be an easy change. If nobody else wants to take it, I would also be happy to start a PR for this feature myself at some point.

Thanks for your input and time

Join janusgraph-dev@lists.lfaidata.foundation to automatically receive all group messages.