Re: Multiple writers cause inconsistent vertices


Ajay Srivastava <Ajay.Sr...@...>
 

Taking the lock on property and index solved the consistency problem but it took 10 mins. to ingest data from three clients while this data can be added from one client in less than 2 min.
Our system will eventually be 99% read and 1% write. Single write client would be enough for that. But I need to ingest millions of files to initialise the graph database. With this speed, it will take years to ingest data. 


Regards,
Ajay


On 24-Sep-2017, at 5:04 PM, Robert Dale <rob...@...> wrote:

Have you read through this section on data consistency?  http://docs.janusgraph.org/latest/eventual-consistency.html

Robert Dale

On Sun, Sep 24, 2017 at 7:33 AM, Robert Dale <rob...@...> wrote:
Enabled is good.

Robert Dale

On Sun, Sep 24, 2017 at 5:47 AM, Ajay Srivastava <Ajay.Sr...@...> wrote:
Thanks Robert.
I have commit and await in my code. Here is more information -
scala> val mgt = graph.openManagement()
mgt: org.janusgraph.core.schema.JanusGraphManagement = org.janusgraph.graphdb.database.management.ManagementSystem@46f31564

scala> val index = mgt.getGraphIndexes(classOf[Vertex]).iterator.next
index: org.janusgraph.core.schema.JanusGraphIndex = UniqueURI

scala> val properties = index.getFieldKeys
properties: Array[org.janusgraph.core.PropertyKey] = Array(URI)

scala> properties.toList
res30: List[org.janusgraph.core.PropertyKey] = List(URI)

scala> index.getIndexStatus(properties(0))
res29: org.janusgraph.core.schema.SchemaStatus = ENABLED


So, the status of index is “ENABLED”. Should it be “REGISTERED" ?
I deleted db and recreated schema again, the awaitGraphIndexStatus call times out -
14:03:04,435  INFO GraphIndexStatusWatcher:81 - Some key(s) on index UniqueURI do not currently have status REGISTERED: URI=ENABLED
14:03:04,435  INFO GraphIndexStatusWatcher:90 - Timed out (PT1M) while waiting for index UniqueURI to converge on status REGISTERED

I waited for half an hour but the status remains as “ENABLED”.
Note that there are no records in db and I create all vertex/edge properties and indexes in one transaction. I have tried creating only vertex properties, labels and index in one transaction and that also is not working.


Regards,
Ajay


On 23-Sep-2017, at 5:22 PM, Robert Dale <rob...@...> wrote:

Do you `mgt.commit()`?  Do you `mgt.awaitGraphIndexStatus(graph, 'UniqueURI').call()`?

You can use this script in the console to help see the state of the index - https://gist.github.com/robertdale/ad4c63910009dd1118abe67b33ce41e1


Robert Dale

On Sat, Sep 23, 2017 at 6:36 AM, Ajay Srivastava <Ajay.Sr...@...> wrote:
Hi,

I am using janusgraph-0.1.1 with HBase.
The data is being loaded in graph using three clients connecting to same gremlin server. The clients are executing same code that checks if vertex is not already present in the graph then it inserts the vertex.
I was verifying the data and found following problem -

scala> graph.V().hasLabel("Root").toList
15:27:22,361  WARN StandardJanusGraphTx:1273 - Query requires iterating over all vertices [(~label = Root)]. For better performance, use indexes
res11: List[gremlin.scala.Vertex] = List(v[737304], v[4136], v[442432])
Results is three vertices.

scala> graph.V().hasLabel("Root").properties("URI").toList
15:27:52,275  WARN StandardJanusGraphTx:1273 - Query requires iterating over all vertices [(~label = Root)]. For better performance, use indexes
res13: List[gremlin.scala.Property[Any]] = List(vp[URI->Root], vp[URI->Root], vp[URI->Root])
Result is three vertices having same URI.

scala> val uri = Key[String]("URI")
scala> graph.V().has(uri, "Root").toList
res12: List[gremlin.scala.Vertex] = List(v[442432])
Since vertices are uniquely indexed on URI, this result is correct. Janusgraph should not have allowed to insert vertices having same URI but it did as displayed in above two outputs.

I am new to janusgraph and have many questions -

1) What am I doing wrong here ?
2) Multiple clients writing to same gremlin server may create problem ?
3) How to read back the schema created by me ?
4) Below is the code for creating schema. Is this correct ?

/* Creating three types of vertices having same properties and indexed on same property URI */
    def createVertexSchema : Boolean = {
        val vertexLabels = Array("Root", "Lang", "Cocpt")

        val GUID     = mgt.makePropertyKey("GUID").dataType(classOf[String]).make
        val Name = mgt.makePropertyKey("Name").dataType(classOf[String]).make
        val URI      = mgt.makePropertyKey("URI").dataType(classOf[String]).make

        vertexLabels.foreach {
            vertexLabel =>
                val vLabel   = mgt.makeVertexLabel(vertexLabel).make
        }

        mgt.buildIndex("UniqueURI", classOf[Vertex]).addKey(URI).unique().buildCompositeIndex()
        true
    }

Regards,
Ajay


--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/D8F1F502-3482-4A8E-AB9A-5021273DC697%40guavus.com.
For more options, visit https://groups.google.com/d/optout.



--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/CC7B1514-1089-4B7A-9CEC-200619E64DC9%40guavus.com.
For more options, visit https://groups.google.com/d/optout.



--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsu...@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/CABed_4ocmXy0-fMdPG1-WihDaU9Qha%2B_PYehRkt5Tq8K9j8K0g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Join janusgraph-users@lists.lfaidata.foundation to automatically receive all group messages.