|
Re: [BLOG] Configuring JanusGraph for spark-yarn
HadoopMarc <bi...@...>
Hi ... and others, I have been offline for a few weeks enjoying a holiday and will start looking into your questions and make the suggested corrections. Thanks for following the recipes and helping others with it. ..., did you run the recipe on the same HDP sandbox and same Tinkerpop version? I remember (from 4 weeks ago) that copying the zookeeper.znode.parent property from the hbase configs to the janusgraph configs was essential to get janusgraph's HBaseInputFormat working (that is: read graph data for the spark tasks). Cheers, Marc Op maandag 24 juli 2017 10:12:13 UTC+2 schreef spi...@...:
|
|
|
|
Re: Best practice setup for Go driver development & identifying the websocket serialization format
Ray Scott <raya...@...>
Do you have a reference for that setting the response format? The driver documentation doesn't mention it, only that you can specify the format of the request. There is an example response in JSON, but it's nothing like what I receive as a response. This is in the Tinkerpop docs. Janus seem to have removed driver development documentation from their release.
http://tinkerpop.apache.org/docs/3.2.5/dev/provider/#_graph_driver_provider_requirements
|
|
|
|
Re: Best practice setup for Go driver development & identifying the websocket serialization format
loh...@...
The server serializes the response in whichever format has been requested by the client. I'd imagine what you're seeing would be the unmarshalled version of the returned JSON that your go websockets library created.
On Friday, August 4, 2017 at 3:40:44 PM UTC-4, Ray Scott wrote:
|
|
|
|
Best practice setup for Go driver development & identifying the websocket serialization format
Ray Scott <raya...@...>
I want to develop a driver in Go that connects to Gremlin Server using a websocket, runs a parameterized Groovy script and parses the response. At this stage all I need to do is perform basic queries and modify the graph. I've read through the documentation on driver development, and looked through some source code for existing drivers. Connecting and sending the data is the easy part. What I can not find anywhere, is an explanation of what I can expect to receive back in terms of the serialised format. I'm actually using JanusGraph straight out the box. I've looked at the yaml config and read some posts on the serializers listed therein. I've read a little about GraphSON and GraphML and Kryo and all I'm really looking for is a way to setup the server so that it returns a response thats some sort of official spec'd format that I can work with in Go. The only other thing I need to do, is be able to use the console as normal. As an example, if I send this query... I receive this... What format is that? How do other driver developers handle this? Do I need to change the settings of the serializers in the yaml config? Do I use a writer in the Groovy script to serialize the result into a format of my choice? I don't want to perform any unnecessary serialization. Thanks.
|
|
|
|
Potential Fix for Indexes stuck in `INSTALLED` state
David Pitera <piter...@...>
Hey guys, I know there have been a bunch of questions lately about indexes getting stuck in the `installed` state, and I recently discovered some more interesting potential causes for the problem; please see #5 in my StackOverflow answer here: https://stackoverflow.com/questions/40585417/titan-db-ignoring-index/40591478#40591478 TLDR; you might have phantom JanusGraph nodes that are unable to acknowledge index existence, and thus the index will never move to `REGISTERED`. You may also have issues with backfilling of the queue, but I would definitely expect the former first. Good luck!
|
|
|
|
Re: how is janusgraph data stored in Cassandra
Jerry He <jerr...@...>
The edges and properties are serialized, encoded and optionally compressed in the backend table. Raw scan on the backend table will not easily show what they are. The things you may be able to see in clear text, for example, are the configuration settings stored in the backend store. Having said that, I wonder if it is feasible or usable to provide such a tool to look at or examine the raw data in the backend table. Thanks.
On Friday, August 4, 2017 at 6:53:45 AM UTC-7, Suny wrote:
|
|
|
|
Re: how is janusgraph data stored in Cassandra
Suny <sahithiy...@...>
Thanks, Under Janusgraph keyspace in cassandra i see some tables with information stored as blob. Is there a way to find the row (containing data from janusgraph) in any table ?
On Thursday, August 3, 2017 at 5:15:56 PM UTC-4, Kelvin Lawrence wrote:
|
|
|
|
about janusgraph use spark(yarn-client) compute
liuzhip...@...
1、configuration " conf/hadoop-graph/read-cassandra.properties " file ，as follow： --------------------------------------------------------------------------------------------------- gremlin.graph=org.apache.tinkerpop.gremlin.hadoop.structure.HadoopGraph gremlin.hadoop.graphInputFormat=org.janusgraph.hadoop.formats.hbase.HBaseInputFormat gremlin.hadoop.graphOutputFormat=org.apache.tinkerpop.gremlin.hadoop.structure.io.gryo.GryoOutputFormat gremlin.hadoop.jarsInDistributedCache=true gremlin.hadoop.inputLocation=none gremlin.hadoop.outputLocation=output janusgraphmr.ioformat.conf.storage.backend=hbase janusgraphmr.ioformat.conf.storage.hostname=testhadoop001.ppdapi.com,testhadoop002.ppdapi.com,testhadoop003.ppdapi.com janusgraphmr.ioformat.conf.storage.keyspace=janusgraph spark.master=yarn-client spark.serializer=org.apache.spark.serializer.KryoSerializer spark.yarn.services=org.apache.spark.deploy.yarn.history.YarnHistoryService spark.yarn.historyServer.address=http://testhadoop002.ppdapi.com:18088 spark.history.provider=org.apache.spark.deploy.yarn.history.YarnHistoryProvider spark.history.ui.port=18088 gremlin.spark.persistContext=true 2、start gremlin.sh，problem is here
view the spark task by web-ui it seems to miss a jar package， but i put guava-16.0.1.jar into the lib
|
|
|
|
DynamoDB autoscaling for JanusGraph
sanjana....@...
Ram
|
|
|
|
Re: how is janusgraph data stored in Cassandra
Kelvin Lawrence <kelvin....@...>
Janus graph uses an adjacency list model. Each vertex, its properties and its adjacent edges are stored as a row in Cassandra. You might find this part of the documentation of use. http://docs.janusgraph.org/latest/data-model.html HTH Kelvin
On Thursday, August 3, 2017 at 3:51:58 PM UTC-5, Suny wrote:
|
|
|
|
how is janusgraph data stored in Cassandra
Suny <sahithiy...@...>
Can someone explain how janusgraph data is stored in Cassandra ? Are there any specific tables in Cassandra that i can look at for data from janusgraph ?
|
|
|
|
Do We Need Specialized Graph Databases? Benchmarking Real-Time Social Networking Applications
rcanz...@...
Has everyone seen this article out of the University of Waterloo, which concludes TinkerPop 3 to be not ready for prime time? Do We Need Specialized Graph Databases? Benchmarking
Real-Time Social Networking Applications Anil Pacaci, Alice Zhou, Jimmy Lin, and M. Tamer Özsu 10.1145/3078447.3078459 https://event.cwi.nl/grades/2017/12-Apaci.pdf Interested to know what other folks think of this testing setup and set of conclusions.
|
|
|
|
Re: hi how can i use janusGraph api to connect gremlin-server
Robert Dale <rob...@...>
There are various ways to connect to gremlin-server and depends on the server configuration and host language. You can start here http://docs.janusgraph.org/latest/server.html#_connecting_to_janusgraph_via_gremlin_server There are more details here http://tinkerpop.apache.org/docs/current/reference/#connecting-via-console Robert Dale
On Wed, Aug 2, 2017 at 11:51 PM, 李平 <lipin...@...> wrote:
|
|
|
|
Re: Janusgraph with ES as index backend
Kelvin Lawrence <kelvin....@...>
Yes - sorry if that was not clear. Kelvin
On Thursday, August 3, 2017 at 10:28:11 AM UTC-5, Suny wrote:
|
|
|
|
Re: Janusgraph with ES as index backend
Suny <sahithiy...@...>
Thanks for you response. By ' If you tell Janus about the indexed properties using the management API' you mean, creating property keys and index using management API right ?
On Wednesday, August 2, 2017 at 6:50:25 PM UTC-4, Kelvin Lawrence wrote:
|
|
|
|
Re: janus cassandra limitations
mirosla...@...
Ok so i get it a bit wrong in my initial assumption. 1. "vertexindex" stores values for all properties for all vertices. In my case key=0x00 is 'false' and this value is stored in 90% of my vertices. so still in theory you could have so many vertices as titan schema allows but you could not store same value for any property more than 2^30 times. 2. "edgestorage" contains information about all vertices with all properties values references and all edges per vertex this means one vertex could have in theory maximum of 2^30 edges 3. Request to janusgraph designers:
On Thursday, August 3, 2017 at 12:58:29 AM UTC+2, Kelvin Lawrence wrote:
|
|
|
|
hi how can i use janusGraph api to connect gremlin-server
李平 <lipin...@...>
i want to use janusGraph api to connect my gremlin server another question ,how to build a unique vertex if the vertex exists ,and then return the vertex
|
|
|
|
Re: janus cassandra limitations
Kelvin Lawrence <kelvin....@...>
Hi Mirosław, Janus graph uses an adjacency list model for storing vertices and edges. A vertex, its properties and all of its adjacent edges are stored in a single Cassandra row, The Janus Graph documentation goes into these issues in some detail. http://docs.janusgraph.org/latest/index.html You are using a very old version of Titan BTW. It would be worth upgrading if you can. Cheers, Kelvin
On Wednesday, August 2, 2017 at 10:36:39 AM UTC-5, Mirosław Głusiuk wrote:
|
|
|
|
Re: Janusgraph with ES as index backend
Kelvin Lawrence <kelvin....@...>
If you tell Janus about the indexed properties using the management API it will use them automatically when you run Gremlin queries. You only need to use indexQuery for cases where you want to read from the index directly for other reasons. HTH Kelvin
On Wednesday, August 2, 2017 at 12:46:37 PM UTC-5, Suny wrote:
|
|
|
|
Re: I'm starting a new startup big project, should I use Janus as main database to store all my data?
Kelvin Lawrence <kelvin....@...>
Hi there, I don't think it would be appropriate to make definitive recommendations as to whether or not to use Janus in production for your needs. The best way to decide on that is to install it and run some tests. What I do know is that on this list a number of people have indicated they either already are or plan to build solutions that include Janus Graph. As to your other questions here are some answers. Janus graph supports the Gremlin query and traversal language that let's you add, delete, update nodes and edges to a graph. Janus supports numerous back end stores that include Cassandra, HBase and Berkley DB and it can also run just in memory which is good for testing. The graph data is persisted to the back end store. Deciding which back end store to use will depend on many factors. You will want to consider things like number of users and whether you care more about consistency or availability when making that choice. I would encourage you to install Janus and run some tests and see what works best for your needs. I'm sure people on this list can help if you encounter issues as you experiment. HTH Kelvin
On Wednesday, August 2, 2017 at 8:56:53 AM UTC-5, Augusto Will wrote:
|
|
|
