|
DynamoDB autoscaling for JanusGraph
sanjana....@...
Ram |
|
|
|
Re: how is janusgraph data stored in Cassandra
Kelvin Lawrence <kelvin....@...>
Janus graph uses an adjacency list model. Each vertex, its properties and its adjacent edges are stored as a row in Cassandra. You might find this part of the documentation of use. http://docs.janusgraph.org/latest/data-model.html HTH Kelvin On Thursday, August 3, 2017 at 3:51:58 PM UTC-5, Suny wrote:
|
|
|
|
how is janusgraph data stored in Cassandra
Suny <sahithiy...@...>
Can someone explain how janusgraph data is stored in Cassandra ? Are there any specific tables in Cassandra that i can look at for data from janusgraph ? |
|
|
|
Do We Need Specialized Graph Databases? Benchmarking Real-Time Social Networking Applications
rcanz...@...
Has everyone seen this article out of the University of Waterloo, which concludes TinkerPop 3 to be not ready for prime time? Do We Need Specialized Graph Databases? Benchmarking
Real-Time Social Networking Applications Anil Pacaci, Alice Zhou, Jimmy Lin, and M. Tamer Özsu 10.1145/3078447.3078459 https://event.cwi.nl/grades/2017/12-Apaci.pdf Interested to know what other folks think of this testing setup and set of conclusions. |
|
|
|
Re: hi how can i use janusGraph api to connect gremlin-server
Robert Dale <rob...@...>
There are various ways to connect to gremlin-server and depends on the server configuration and host language. You can start here http://docs.janusgraph.org/latest/server.html#_connecting_to_janusgraph_via_gremlin_server There are more details here http://tinkerpop.apache.org/docs/current/reference/#connecting-via-console Robert Dale On Wed, Aug 2, 2017 at 11:51 PM, 李平 <lipin...@...> wrote:
|
|
|
|
Re: Janusgraph with ES as index backend
Kelvin Lawrence <kelvin....@...>
Yes - sorry if that was not clear. Kelvin On Thursday, August 3, 2017 at 10:28:11 AM UTC-5, Suny wrote:
|
|
|
|
Re: Janusgraph with ES as index backend
Suny <sahithiy...@...>
Thanks for you response. By ' If you tell Janus about the indexed properties using the management API' you mean, creating property keys and index using management API right ? On Wednesday, August 2, 2017 at 6:50:25 PM UTC-4, Kelvin Lawrence wrote:
|
|
|
|
Re: janus cassandra limitations
mirosla...@...
Ok so i get it a bit wrong in my initial assumption. 1. "vertexindex" stores values for all properties for all vertices. In my case key=0x00 is 'false' and this value is stored in 90% of my vertices. so still in theory you could have so many vertices as titan schema allows but you could not store same value for any property more than 2^30 times. 2. "edgestorage" contains information about all vertices with all properties values references and all edges per vertex this means one vertex could have in theory maximum of 2^30 edges 3. Request to janusgraph designers: On Thursday, August 3, 2017 at 12:58:29 AM UTC+2, Kelvin Lawrence wrote:
|
|
|
|
hi how can i use janusGraph api to connect gremlin-server
李平 <lipin...@...>
i want to use janusGraph api to connect my gremlin server another question ,how to build a unique vertex if the vertex exists ,and then return the vertex |
|
|
|
Re: janus cassandra limitations
Kelvin Lawrence <kelvin....@...>
Hi Mirosław, Janus graph uses an adjacency list model for storing vertices and edges. A vertex, its properties and all of its adjacent edges are stored in a single Cassandra row, The Janus Graph documentation goes into these issues in some detail. http://docs.janusgraph.org/latest/index.html You are using a very old version of Titan BTW. It would be worth upgrading if you can. Cheers, Kelvin On Wednesday, August 2, 2017 at 10:36:39 AM UTC-5, Mirosław Głusiuk wrote:
|
|
|
|
Re: Janusgraph with ES as index backend
Kelvin Lawrence <kelvin....@...>
If you tell Janus about the indexed properties using the management API it will use them automatically when you run Gremlin queries. You only need to use indexQuery for cases where you want to read from the index directly for other reasons. HTH Kelvin On Wednesday, August 2, 2017 at 12:46:37 PM UTC-5, Suny wrote:
|
|
|
|
Re: I'm starting a new startup big project, should I use Janus as main database to store all my data?
Kelvin Lawrence <kelvin....@...>
Hi there, I don't think it would be appropriate to make definitive recommendations as to whether or not to use Janus in production for your needs. The best way to decide on that is to install it and run some tests. What I do know is that on this list a number of people have indicated they either already are or plan to build solutions that include Janus Graph. As to your other questions here are some answers. Janus graph supports the Gremlin query and traversal language that let's you add, delete, update nodes and edges to a graph. Janus supports numerous back end stores that include Cassandra, HBase and Berkley DB and it can also run just in memory which is good for testing. The graph data is persisted to the back end store. Deciding which back end store to use will depend on many factors. You will want to consider things like number of users and whether you care more about consistency or availability when making that choice. I would encourage you to install Janus and run some tests and see what works best for your needs. I'm sure people on this list can help if you encounter issues as you experiment. HTH Kelvin On Wednesday, August 2, 2017 at 8:56:53 AM UTC-5, Augusto Will wrote:
|
|
|
|
Re: [BLOG] Configuring JanusGraph for spark-yarn
Joe Obernberger <joseph.o...@...>
Could this be a networking issue? Maybe a firewall is enabled, or selinux is preventing a connection? I've been able to get this to work, but running a simple count - g.V().count() on anything but a very small graph takes a very very long time (hours). Are there any cache settings, or other resources that could be modified to better the performance? The YARN container logs are filled withe debug lines about 'Created dirty vertex map with initial size 32', 'Created vertex cache with max size 20000', and 'Generated HBase Filter ColumnRange Filter'. Can any of these things be adjusted in the properties file? Thank you! -Joe On 7/24/2017 4:12 AM,
spirit...@... wrote:
|
|
|
|
Janusgraph with ES as index backend
Suny <sahithiy...@...>
Hi, I am using Janusgraph with ES as index backend. I created a mixed index on some vertex attributes. This mixed index is saved in ES. Now if i query for that vertices based on index, will janusgraph use ES internally to perform search operation ? Or do i need to use IndexQuery to perform search on ES directly ? |
|
|
|
janus cassandra limitations
mirosla...@...
Hi all, from what I know janus is fork of titan which means if it does not have different storage impl it could have problems with bigger data count. "janusgraph/titan can store up to a quintillion edges (2^60) and half as many vertices. " "The maximum number of cells (rows x columns) in a single partition is 2 billion." 2 billions is about (2^31) in cassandra schema we always have 2 columns per table so you could store about (2^30) values per key so if not mistaken "half as many vertices" is not for cassandra storage backend? I'm using titan 0.4.4 and after having like 50M+ vertices I have spot cassandra started to complain about "Compacting large partition titan/vertexindex:00". So my question is what is real janusgraph/titan limit for cassandra backend which will not "kill" cassandra? Btw I also spot that some keys from "edgestore" table for "supernodes" are also bigger than 1GB with my current graph. Could anyone explain how janusgraph stores data in cassandra and how to configure it to prevent storing huge rows? |
|
|
|
open source graph meetup NYC Aug 22
Jason Plurad <plu...@...>
We're getting another open source graph meetup organized in New York City later this month, Tuesday August 22 https://www.meetup.com/graphs/events/241136321/ If anybody in the community is doing something with graphs that you'd like to present, let me know (pluradj/gmail if you don't want to post up here. We picked the date and location to be close to the first ever JupyterCon conference, so if you have a topic with data science, machine learning, Python, etc, those might work too. Thanks! -- Jason |
|
|
|
I'm starting a new startup big project, should I use Janus as main database to store all my data?
Augusto Will <pw...@...>
I'm thinking about learn Janus to use in my new big project but i can't understand some things. Janus
can be used like any database and supports "insert", "update", "delete"
operations so Janus will write data into Cassandra or other database
to store these data, right? Where Janus store the Nodes, Edges, Attributes etc, it will write these into database, right? These data should be loaded in memory by Janus or will be read from Cassandra all the time? The data that Janus read, must be load in Janus in every query or it will do selects in database to retrieve the data I need? The data retrieved in database is only what I need or Janus will read all records in database all the time? Should I use Janus in my project in production or should I wait until it becomes production ready? I'm
developing some kind of social network that need to store friendship,
posts, comments, user blocks and do some elasticsearch too, in this
case, what database backend should I use? Thank you. |
|
|
|
Re: Failed to load many nodes & edges
Jason Plurad <plu...@...>
Robert Dale posted an answer over on gremlin-users https://groups.google.com/d/msg/gremlin-users/PGtuWvG8UNs/AKtvy9ipAwAJ On Tuesday, August 1, 2017 at 10:02:04 AM UTC-4, Ohad Pinchevsky wrote:
|
|
|
|
Savvi and The Graph
Jay F <na...@...>
(x-post to gremlin-users) Fellow traversers! For the last 3 years at Savvi, we've been doing our bit towards reifying the Universal Graph Theory[1] by building extremely high-fidelity Graphs for our telecoms customers. These graphs span the gamut from active physical infrastructure (the modem at your home, copper cables, sheaths, segments, pits, pillars and so forth, the DSLAM, the rack, the chassis, the line card, the SFP, the ethernet cable... on and on it goes...), to the logical constructs on top (virtual circuits, trunks/LAGs, satellite beam apertures etc.). We then annotate that structural graph with time-series data (link utilisation, for example) - yes, we store this in graph, although hopefully not for too much longer! - as well as rendering more ephemeral data such as incident tickets, alarms/alerts as well. We've been running Titan 0.5.4 in Production since January 2015, and we even worked with the Aurelius team in the background in the early days (at least until they were snarfed up by Datastax!). Since then we've been progressively adding more and more fidelity to the graph as new datasets became available to us. Despite initial teething problems with Titan (principally around HA/DR), Titan has been rock solid for us, running in a mission critical environment and powering a number of applications. One of the more interesting ones is real-time field force optimisation, whereby our automatons will identify common infrastructural elements across incidents, and migrate a field engineer to look at that element as opposed to visiting a number of households. This kinds of application of the graph is extremely valuable, and yet took us just 2 months from concept to Production - possible only because of the expressivity of the Gremlin/Groovy language (80% of that application is an - admittedly very complex - Gremlin query). A big thank you to all of the Tinkerpop, Titan and now JanusGraph developers for contributing to this awesome project and keeping it stable amidst a large amount of change! We are naturally now in the process of migrating to JanusGraph, and expanding our use of Graph into a number of other areas as well. I'm poking my head out as we're now looking for more Graph wizards to join our team, and I thought some of you may find the above interesting even if you're not looking for work - hopefully I won't be hung, drawn and quartered for advertising as a result ;) If you'd like to come play with us, please take a look at the role[2] and either apply or fire an email to firstc...@.... Happy to discuss any of the above on here too. Thanks! [1] https://www.youtube.com/watch?v=aRNWhpEPOOA [2] https://savvi.workable.com/jobs/113135 Best Regards, Jay Fenton (@jfenton / skype:jfenton) Founder & CTO, Savvi Inc. Level 3, 455 Bourke Street Melbourne CBD, VIC, Australia |
|
|
|
Re: janusgraph solr cassandra GraphOfTheGodsFactory
Adam Holley <holl...@...>
I deleted my previous post as it was not correct. You do not need to create a core for each mixedIndex. Assuming you are using Solr cloud mode, and following the instructions for Option 1 (http://docs.janusgraph.org/latest/solr.html#_solr_collections) you just need to manually copy the configset, and then add the initial core. Here's the relevant section from my janusgraph-cassandra-solr.properties file index.search.backend=solr index.search.solr.mode=cloud index.search.solr.zookeeper-url=localhost:2181 index.search.solr.configset=janusgraph On Thursday, July 13, 2017 at 10:35:47 PM UTC-5, s...@... wrote:
|
|
|
