Hi!

Currently JanusGraph's philosophy is to fail transactions only if the primary persistence fails and allow failures in secondary persistence.
According to https://docs.janusgraph.org/advanced-topics/recovery/#transaction-failure, the recommended approach for dealing with secondary persistence failures is maintaining and continuously recovering from a write-ahead log.
This results in temporary inconsistencies between primary storage and mixed indices.

For my use case, potential inconsistencies are not acceptable. Currently there is no way of making JanusGraph fail a transaction if either primary or secondary persistence fails. Looking at the source code (starting at https://github.com/JanusGraph/janusgraph/blob/a73d39d8a071f8d5340aede4917653629ab595cb/janusgraph-core/src/main/java/org/janusgraph/graphdb/database/StandardJanusGraph.java#L743), it should be possible to re-order persistence to commit both secondary and primary storage at the same time. Naturally there is some cleanup to do in case of transaction failure, but it seems manageable to me.

What are the potential complexities of pursuing this? How does this mess with JanusGraph's logic & assumptions in other places?

If it seems fine, it should be an easy change. If nobody else wants to take it, I would also be happy to start a PR for this feature myself at some point.

Thanks for your input and time
Florian

Hi Florian!

That definitely sounds like a promising idea. You are most likely not the only one requiring consistency between storage and index data. In general, my advice is to start with a test case which explicitly addresses the situation where the primary persistence succeeds and the secondary persistence fails. From there, see what it needs to let the transaction fail. Once you have a solution for that, you can start the discussion by opening a PR. We will then see which issues could arise and try to solve them. The first prototype does not have to be perfect, a proof of concept is fine!

Best Regards,
Florian