Date   

Re: [VOTE] JanusGraph 0.6.0 release

Florian Hockmann
 

I just did a quick test of the two distribution archives which just consists of starting JanusGraph Server and connecting to it via the remote console to create a vertex. For the full distribution, I then also restarted the server to verify that the vertex was persisted in Cassandra. This unfortunately failed for 0.6.0. The graph was completely empty after executing ‘bin/janusgraph.sh stop’ and ‘start’.

I’m not sure what’s causing this, but I think that we should look into it. At a first glance, the problem seems to be related to Cassandra as that didn’t create a directory ‘db/Cassandra/data/janusgraph’ for the JanusGraph keyspace. I only see directories for the system keyspaces there, but for 0.5.3 where this still worked, there was also a janusgraph keyspace directory.

 

Other then that, I also noticed that the Changelog mentions the removal of Thrift and refers to this repository where the Thrift code should be moved to: https://github.com/JanusGraph/janusgraph-cassandra

Unfortunately, the PR that would actually add the code to master branch there is still open: https://github.com/JanusGraph/janusgraph-cassandra/pull/3

I don’t think that this should stop the release, but we should try to merge it at least soon after the release.

 

Von: janusgraph-dev@... <janusgraph-dev@...> Im Auftrag von Boxuan Li
Gesendet: Dienstag, 31. August 2021 09:32
An: janusgraph-dev@...
Betreff: Re: [janusgraph-dev] [VOTE] JanusGraph 0.6.0 release

 

I did the following tests:

  1. Connected to AWS keyspace, did some simple traversals
  2. Compatibility test: using 0.5.3 CQL + ES to create a graph with 10 million nodes (https://gist.github.com/li-boxuan/d8059d1c40652c3c5308670ce383e96a), able to load using 0.6.0
  3. Did some simple performance comparison (using the graph created in the previous test) and saw significant performance improvements (up to 2x) in some queries, compared to 0.5.3:

0.5.3 (query.batch=true):

gremlin>  clock(10) { graph.traversal().V().has("name", "inner0").toList() }

==>478.1922667999999

gremlin>  clock(10) { graph.traversal().V().has("age", 0).toList() }

==>17873.4094548

gremlin> clock(10) {graph.traversal().V().has("name", "outer").out().out().toList()}

==>14500.0969486

 

 

0.6.0 (query.batch = true, query.limit-batch-size=true)

gremlin>  clock(10) { graph.traversal().V().has("name", "inner0").toList() }

==>267.27335619999997

gremlin> clock(10) { graph.traversal().V().has("age", 0).toList() }

==>7432.816540099999

gremlin> clock(10) {graph.traversal().V().has("name", "outer").out().out().toList()}

==>11786.6717737

 

  1. Basic SparkGraphComputer traversal works in Spark-local, Spark standalone cluster, and Spark yarn cluster modes. I also wrote a blog to record all configurations needed: https://li-boxuan.medium.com/configure-janusgraph-0-6-0-for-spark-64bd7628fb77

 

My vote is +1.


Re: [VOTE] JanusGraph 0.6.0 release

Boxuan Li
 

I did the following tests:

  1. Connected to AWS keyspace, did some simple traversals
  2. Compatibility test: using 0.5.3 CQL + ES to create a graph with 10 million nodes (https://gist.github.com/li-boxuan/d8059d1c40652c3c5308670ce383e96a), able to load using 0.6.0
  3. Did some simple performance comparison (using the graph created in the previous test) and saw significant performance improvements (up to 2x) in some queries, compared to 0.5.3:

0.5.3 (query.batch=true):

gremlin>  clock(10) { graph.traversal().V().has("name", "inner0").toList() }

==>478.1922667999999

gremlin>  clock(10) { graph.traversal().V().has("age", 0).toList() }

==>17873.4094548

gremlin> clock(10) {graph.traversal().V().has("name", "outer").out().out().toList()}

==>14500.0969486

 

 

0.6.0 (query.batch = true, query.limit-batch-size=true)

gremlin>  clock(10) { graph.traversal().V().has("name", "inner0").toList() }

==>267.27335619999997

gremlin> clock(10) { graph.traversal().V().has("age", 0).toList() }

==>7432.816540099999

gremlin> clock(10) {graph.traversal().V().has("name", "outer").out().out().toList()}

==>11786.6717737

 

  1. Basic SparkGraphComputer traversal works in Spark-local, Spark standalone cluster, and Spark yarn cluster modes. I also wrote a blog to record all configurations needed: https://li-boxuan.medium.com/configure-janusgraph-0-6-0-for-spark-64bd7628fb77

My vote is +1.


Re: [VOTE] JanusGraph 0.6.0 release

Oleksandr Porunov
 

I also executed all TinkerPop tests again with `mvn clean install -Dtest.skip.tp=false -DskipTests=true`. All tests passed.


Re: [VOTE] JanusGraph 0.6.0 release

Oleksandr Porunov
 

I realized that I was building with Java 11 instead of Java 8. I switched to the Java 8 (OpenJDK 1.8.0_302), dropped previous staging repository, rebuilt JanusGraph using Java 8, re-uploaded new artifacts to the staging repository and to GitHub release. Checked that `graph = JanusGraphFactory.open("inmemory")` now works in Gremlin console. I believe the above issue should be fixed now.


Re: [VOTE] JanusGraph 0.6.0 release

Oleksandr Porunov
 

Thank you Boxuan! Confirm this issue. Most likely I made a mistake during the build process because I can reproduce this issue with my build but can't reproduce this issue with the next build (https://github.com/JanusGraph/janusgraph/actions/runs/1177782203) see `distribution-builds` artifact. I will check that tomorrow and will try to fix that.
I think we will need to automate the releasing process soon to eliminate such issues in the future and build releases in deterministic way.


Re: [VOTE] JanusGraph 0.6.0 release

Boxuan Li
 

I downloaded janusgraph-0.6.0.zip and got this error when using the gremlin console:

gremlin> graph = JanusGraphFactory.open("inmemory")
java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer;
Type ':help' or ':h' for help.
Display stack trace? [yN]y
java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer;
at org.janusgraph.diskstorage.util.WriteByteBuffer.getStaticBufferFlipBytes(WriteByteBuffer.java:148)


Re: [VOTE] JanusGraph 0.6.0 release

Oleksandr Porunov
 

I dropped previous staging release from Sonatype account and reuploaded a new staring release.
I checked and successfully closed the staging release.
GitHub tag v0.6.0 has been updated to point on the latest commit.
All the GitHub artifacts has been rebuilt and updated.

The issue discovered by Boxuan Li has been resolved as well as several other issues. Moreover one new feature has been added to the release (possibility to configure internal Cassandra driver).

I'm reopening the voting process.

This [VOTE] will open for the next 3 days --- closing Tuesday, August 31, 2021 at 11:55 PM GMT+3.
All are welcome to review and vote on the release, but only votes from TSC members are binding.
My vote is +1.

Thank you,
Oleksandr Porunov


Re: [DISCUSSION] Release snapshot versions on each commit

Oleksandr Porunov
 

Hi Jan,

Thanks for pointing on that! That indeed uploads distribution for a commit but it doesn't provide you with the maven nexus repository to use those distributions. Basically, it means, that you will need to physically download `distribution-builds` and run it. In case you would like to run JanusGraph embedded into the application, it becomes not as easy to be used due to necessity to include transitive dependencies manually.
What I mean is to publish staging artifacts to the maven repository such as GitHub actions. If so, people could just include latest JanusGraph master snapshot releases simply defining something like:
<dependency>
  <groupId>org.janusgraph</groupId>
  <artifactId>janusgraph-core</artifactId>
  <version>0.6.0-SNAPSHOT-12345</version>
</dependency>

Ignore `version` as for now because I didn't think about versioning yet but you can see a simple example here: https://github.com/orgs/mapped/packages?repo_name=janusgraph
See the commit here: https://github.com/mapped/janusgraph/commit/a94a0c03cb9d1b1445901c6735aff3e66b8692ac

Basically, I think it's possible to configure different publishing flows (either to sonatype for official releases or to GitHub Packages for unofficial releases).

Moreover, about the link which you shared. Did we document it anywhere? If no, I think it would be good to create a separate documentation page which tells about release artifacts for each commit and potentially releases to GitHub Packages for each commit.  

Best regards,
Oleksandr


Re: [DISCUSSION] Release snapshot versions on each commit

Jansen, Jan
 

Hi Porunov

We already build release package for any master commit and this release package can be download by anyone.

Disclaimer:

It doesn't follow our default release guildlines, it just builds it and publishes as artifact (https://github.com/JanusGraph/janusgraph/actions/runs/1155506290).


I think it would great if we fully automate the release any step.

Greetings,
Jan


Von: janusgraph-dev@... <janusgraph-dev@...> im Auftrag von Oleksandr Porunov <alexandr.porunov@...>
Gesendet: Montag, 16. August 2021 15:09:26
An: janusgraph-dev@...
Betreff: [janusgraph-dev] [DISCUSSION] Release snapshot versions on each commit
 
I would like to propose to make simple GitHub jar releases on each commit in `master` or `v*` branches. Those releases would be simple GitHub packages and their purpose wouldn't be mostly for easier testing. 
This would also allow users to use latest features from `master` branch without need to build their own version of JanusGraph to use those snapshot versions.

Would be great to hear other opinions about it. Do you think this would be useful for the community or not?

Best regards,
Oleksandr


Re: [VOTE] JanusGraph 0.6.0 release

Oleksandr Porunov
 

Hi @vinay1331,

This is expected. You must upgrade your Gremlin Server client first. See upgrade instructions regarding your issue here: https://docs.janusgraph.org/master/changelog/#serialization-of-janusgraph-predicates-has-changed
Also, for gremlin settings breaking changes see the following instructions: 
https://docs.janusgraph.org/master/changelog/#breaking-change-for-gremlin-server-configs
https://docs.janusgraph.org/master/changelog/#introduction-of-a-janusgraph-server-startup-class-as-a-replacement-for-gremlin-server-startup
https://docs.janusgraph.org/master/changelog/#automatic-configurations-of-dynamic-graph-binding

Also, notice, the binaries will be updated soon. Thus, you *may* potentially need to update your gremlin server multiple times.

Thanks for taking part in testing.

Best regards,
Oleksandr


Re: [VOTE] JanusGraph 0.6.0 release

vinay1331@...
 

After the upgrade to testing getting following exception using remote traversal. Used tinkerpop 3.5.1 with janusgrah drive-0.6.0


WARN  org.apache.tinkerpop.gremlin.server.handler.WsGremlinBinaryRequestDecoder  - 
Gremlin Server is not configured with a serializer for the 
requested mime type [application/vnd.gremlin-v1.0+gryo] 
- using org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1 by default
2889684 [gremlin-server-worker-1]
 WARN  org.apache.tinkerpop.gremlin.server.handler.OpSelectorHandler  - 
 Invalid OpProcessor requested [null]
org.apache.tinkerpop.gremlin.server.op.OpProcessorException: Invalid OpProcessor requested [null]
        at org.apache.tinkerpop.gremlin.server.handler.OpSelectorHandler.decode(OpSelectorHandler.java:85)
        at org.apache.tinkerpop.gremlin.server.handler.OpSelectorHandler.decode(OpSelectorHandler.java:48)
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
        at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
        at io.netty.handler.codec.http.websocketx.Utf8FrameValidator.channelRead(Utf8FrameValidator.java:82)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324)


[DISCUSSION] Release snapshot versions on each commit

Oleksandr Porunov
 

I would like to propose to make simple GitHub jar releases on each commit in `master` or `v*` branches. Those releases would be simple GitHub packages and their purpose wouldn't be mostly for easier testing. 
This would also allow users to use latest features from `master` branch without need to build their own version of JanusGraph to use those snapshot versions.

Would be great to hear other opinions about it. Do you think this would be useful for the community or not?

Best regards,
Oleksandr


Re: [VOTE] JanusGraph 0.6.0 release

Boxuan Li
 

Hi thanks for the quick responses. Followed by Clement’s advice, I made a fix https://github.com/li-boxuan/janusgraph/commit/aec47d44e6b0a57eac5fb5acab4c11d365d0ecac that worked on my local laptop. If CI pass I will submit a PR.

Best regards,
Boxuan

On Aug 14, 2021, at 8:15 PM, Oleksandr Porunov <alexandr.porunov@...> wrote:

Thank you Boxuan for the catch!

I will try to look at it today's evening. I have no luck I hope Clement will be able to check it on Monday.

Thus, I'm extending the voting process until the issue is fixed. After the fix I will re-upload artifacts to Sonatype and update release artifacts and a tag.

Best regards,
Oleksandr


Re: [VOTE] JanusGraph 0.6.0 release

Oleksandr Porunov
 

Thank you Boxuan for the catch!

I will try to look at it today's evening. I have no luck I hope Clement will be able to check it on Monday.

Thus, I'm extending the voting process until the issue is fixed. After the fix I will re-upload artifacts to Sonatype and update release artifacts and a tag.

Best regards,
Oleksandr


Re: [VOTE] JanusGraph 0.6.0 release

Clement de Groc
 

Hey Boxuan,

Good catch !

To the best of my knowledge, this type of error usually means there is a mix of scala 2.11 and scala 2.12 libraries.

TinkerPop 3.5 switched to Spark 3 and Scala 2.12 (https://github.com/apache/tinkerpop/blob/master/spark-gremlin/pom.xml#L88).

But I see (at least one) reference to Scala 2.11 libraries in janusgraph-hadoop (https://github.com/JanusGraph/janusgraph/blob/master/janusgraph-hadoop/pom.xml#L77).

I would try chasing down libraries pulling Scala 2.11 and upgrading them.
I can look into that on Monday if noone has before.

Best,
Clement

Le sam. 14 août 2021 à 06:36, Boxuan Li <liboxuan@...> a écrit :
I downloaded https://github.com/JanusGraph/janusgraph/releases/download/v0.6.0/janusgraph-0.6.0.zip, tried a simple traversal using SparkGraphComputer on my laptop but failed:

:plugin use tinkerpop.hadoop
:plugin use tinkerpop.spark
graph = GraphFactory.open('conf/hadoop-graph/read-cql.properties')
g = graph.traversal().withComputer(SparkGraphComputer)
g.V().count()
with error `java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutable/ArrayOps;`

I don't have scala or Spark installed locally. The same traversal worked well on the 0.5.3 version. I personally never ran any workload on Spark before, so I am not sure if this is expected, or due to a library conflict in Janusgraph distribution. If anyone familiar with Spark can shed some light or test by themselves, it would be very helpful. Otherwise, I am afraid 0.6.0 might not work well with Spark.

Best,
Boxuan


Re: [VOTE] JanusGraph 0.6.0 release

Boxuan Li
 

I downloaded https://github.com/JanusGraph/janusgraph/releases/download/v0.6.0/janusgraph-0.6.0.zip, tried a simple traversal using SparkGraphComputer on my laptop but failed:

:plugin use tinkerpop.hadoop
:plugin use tinkerpop.spark
graph = GraphFactory.open('conf/hadoop-graph/read-cql.properties')
g = graph.traversal().withComputer(SparkGraphComputer)
g.V().count()
with error `java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutable/ArrayOps;`

I don't have scala or Spark installed locally. The same traversal worked well on the 0.5.3 version. I personally never ran any workload on Spark before, so I am not sure if this is expected, or due to a library conflict in Janusgraph distribution. If anyone familiar with Spark can shed some light or test by themselves, it would be very helpful. Otherwise, I am afraid 0.6.0 might not work well with Spark.

Best,
Boxuan


Re: [VOTE] JanusGraph 0.6.0 release

Clement de Groc
 

Hey,

I've performed the following tests:
- Built the source code and a docker image from janusgraph-dist/Dockerfile.
- Updated an existing configuration/graph. Followed the upgrade instructions.
- Started JanusGraph and performed simple requests (hitting composite and mixed indices).
- Enabled and checked the presence of CQL metrics and threadpool metrics.

+1 from me.

Clement

On Wed, Aug 11, 2021 at 6:17 PM Rafael Fernandes <luizrafael@...> wrote:
+1
Tested the binaries in our docker instances..

Rafa


On Wed, Aug 11, 2021 at 8:43 AM Oleksandr Porunov <alexandr.porunov@...> wrote:
Hello,

We are happy to announce that JanusGraph 0.6.0 is ready for release.

The release artifacts can be found at this location:
        https://github.com/JanusGraph/janusgraph/releases/tag/v0.6.0

A full binary distribution is provided for user convenience:
        https://github.com/JanusGraph/janusgraph/releases/download/v0.6.0/janusgraph-full-0.6.0.zip
 
A truncated binary distribution is provided:
        https://github.com/JanusGraph/janusgraph/releases/download/v0.6.0/janusgraph-0.6.0.zip
 
The GPG key used to sign the release artifacts is available at:
        https://github.com/JanusGraph/janusgraph/releases/download/v0.6.0/KEYS

The docs can be found here:
        https://github.com/JanusGraph/janusgraph/releases/download/v0.6.0/janusgraph-0.6.0-doc.zip

The release tag in Git can be found here:
        https://github.com/JanusGraph/janusgraph/tree/v0.6.0

The release notes are available here:
        https://github.com/JanusGraph/janusgraph/blob/master/docs/changelog.md#version-060-release-date-august-11-2021

This [VOTE] will open for the next 3 days --- closing Saturday, August 14, 2021 at 3:45 PM GMT+3.
All are welcome to review and vote on the release, but only votes from TSC members are binding.
My vote is +1.

Thank you,
Oleksandr Porunov


Re: [VOTE] JanusGraph 0.6.0 release

Rafael Fernandes
 

+1
Tested the binaries in our docker instances..

Rafa


On Wed, Aug 11, 2021 at 8:43 AM Oleksandr Porunov <alexandr.porunov@...> wrote:
Hello,

We are happy to announce that JanusGraph 0.6.0 is ready for release.

The release artifacts can be found at this location:
        https://github.com/JanusGraph/janusgraph/releases/tag/v0.6.0

A full binary distribution is provided for user convenience:
        https://github.com/JanusGraph/janusgraph/releases/download/v0.6.0/janusgraph-full-0.6.0.zip
 
A truncated binary distribution is provided:
        https://github.com/JanusGraph/janusgraph/releases/download/v0.6.0/janusgraph-0.6.0.zip
 
The GPG key used to sign the release artifacts is available at:
        https://github.com/JanusGraph/janusgraph/releases/download/v0.6.0/KEYS

The docs can be found here:
        https://github.com/JanusGraph/janusgraph/releases/download/v0.6.0/janusgraph-0.6.0-doc.zip

The release tag in Git can be found here:
        https://github.com/JanusGraph/janusgraph/tree/v0.6.0

The release notes are available here:
        https://github.com/JanusGraph/janusgraph/blob/master/docs/changelog.md#version-060-release-date-august-11-2021

This [VOTE] will open for the next 3 days --- closing Saturday, August 14, 2021 at 3:45 PM GMT+3.
All are welcome to review and vote on the release, but only votes from TSC members are binding.
My vote is +1.

Thank you,
Oleksandr Porunov


Re: [DISCUSS] JanusGraph 0.6.0 release

Oleksandr Porunov
 

The voting is now opened for 0.6.0 release at this location: https://lists.lfaidata.foundation/g/janusgraph-dev/topic/vote_janusgraph_0_6_0/84815207

Best regards,
Oleksandr


[VOTE] JanusGraph 0.6.0 release

Oleksandr Porunov
 

Hello,

We are happy to announce that JanusGraph 0.6.0 is ready for release.

The release artifacts can be found at this location:
        https://github.com/JanusGraph/janusgraph/releases/tag/v0.6.0

A full binary distribution is provided for user convenience:
        https://github.com/JanusGraph/janusgraph/releases/download/v0.6.0/janusgraph-full-0.6.0.zip
 
A truncated binary distribution is provided:
        https://github.com/JanusGraph/janusgraph/releases/download/v0.6.0/janusgraph-0.6.0.zip
 
The GPG key used to sign the release artifacts is available at:
        https://github.com/JanusGraph/janusgraph/releases/download/v0.6.0/KEYS

The docs can be found here:
        https://github.com/JanusGraph/janusgraph/releases/download/v0.6.0/janusgraph-0.6.0-doc.zip

The release tag in Git can be found here:
        https://github.com/JanusGraph/janusgraph/tree/v0.6.0

The release notes are available here:
        https://github.com/JanusGraph/janusgraph/blob/master/docs/changelog.md#version-060-release-date-august-11-2021

This [VOTE] will open for the next 3 days --- closing Saturday, August 14, 2021 at 3:45 PM GMT+3.
All are welcome to review and vote on the release, but only votes from TSC members are binding.
My vote is +1.

Thank you,
Oleksandr Porunov

41 - 60 of 1585