Re: Anyone with experience of adding new Storage backend for JanusGraph ? [Help needed w.r.t SnowFlake]
Debasish Kanhar <d.k...@...>
toggle quoted messageShow quoted text
Sorry about the late response. I was working on this project part time only till last week when we moved into full time dev for this PoC. Really thanks to your pointers and Jason's that we have been able to start with the development works and we have some ground work to start with :-)
So,we are modelling SnowFlake (Which is like SQL File store) as a Key-Value store by creating two columns namely "Key" and "Value" in each tables. We are going to define the data type as binary here (Or Stringified Binary) so that arbitrary data can be dumped (I feel its of type StaticBuffer Key and StaticBuffer value. Is that correct? )
Since, we are modelling SnowFlake as Key-Value store, it makes sense to have a SnowFlakeManager class implement OrderedKeyValueStore like for BerkleyJE? Is that correct understanding?
Updates are that we have almost finished development of SnowFlakeManager class. The required methods needed are implemented like beginTransaction, openDatabase though one particular function not done is mutateMany is not done, but it will be done as it in turn calls KeyValueStore.insert() method.
Also, a lot of basic functions in KeyValueStore is also done like insert (Insert binary key-value), get (Get from binary key), delete (Delete a row using binary key). We are kinda stuck at the function getSlice(). What does it do?
We are kinda wondering how getSlice operates? I know that the function is used when querying Janusgraph for gremlin queries (Read operations) (https://github.com/BillBaird/delftswa-aurelius-titan/blob/master/SA-doc/Operations.md#queries) . We see that a sliceQuery is generated which is then executed againt backend to get results.
Now, my question here is that, slice query is used while queryingfor properties for vertices (edges/properties) by slicing the relations of vertex and slicing them based on filters/conditions. The following steps are followed in getSlice function (BerkleyKeyValueStore - berkleydb & ColumnValueStore - inmemory) :
My question is related to last step. Since my data in DB is just "Binary Key-Binary Value", how can we apply another constraints (slice conditions) in query? It just doesn't have any additional meta data to apply slice on as I just have 2 columns in my table.
Hope my explaination was clear for you to understand. I want to know primarily how the last step would work in the data model I described above (Having 2 columns, one for Key and other for Value. And each of stringified binary data type). And, is the data model selected good enough?
Thanks in advance. And I promise this time my replies will be quicker :-)
On Friday, 25 October 2019 03:17:24 UTC+5:30, Dmitry Kovalev wrote: