Re: [DISCUSS] JanusGraph 0.6.1 release
The PR for automatic releases is here: https://github.com/JanusGraph/janusgraph/pull/2941
Best regards, Oleksandr
|
|
JG Schema - addConnection seem to create duplicate connections
Peter Molnar
Hi All,
I have a strange behaviour while using the addConnection method for creating JG schema constraints. It seems it creates duplicated connections in some cases. Please see below how to reproduce this with JG v0.6 and Cassandra v3.11.11 backend. I just used the below snippets in Gremlin Console 3.5.1 to connect to JG remotely. The graph is supposed model transactions and parties involved in the transaction. It has transaction, person and entity nodes. A transaction is supposed to have one "from party" (either person or entity) connected with FROM edge and one "to party" (either person or entity) connected with TO edge. For example (Person A) --FROM--> (Transaction #1) --TO--> (Entity A) for expressing that "Transaction #1" was performed between "Person A" and "Entity A" and the source of the transaction was "Person A". Creating dynamic graph and schema ------- map = new HashMap();
map.put("storage.backend","cql");
map.put("storage.hostname","cassandra");
map.put("query.force-index", "false");
map.put("schema.default", "default");
map.put("schema.constraints", "false");
map.put("graph.graphname", "transactionGraph")
ConfiguredGraphFactory.createConfiguration(new MapConfiguration(map));
graph = ConfiguredGraphFactory.open("transactionGraph");
mgmt = graph.openManagement();
transaction = mgmt.makeVertexLabel('transaction').make();
person = mgmt.makeVertexLabel('person').make();
entity = mgmt.makeVertexLabel('entity').make();
fromEdge = mgmt.makeEdgeLabel('FROM').multiplicity(ONE2ONE).make()
mgmt.addConnection(fromEdge, person, transaction)
mgmt.addConnection(fromEdge, entity, transaction)
toEdge = mgmt.makeEdgeLabel('TO').multiplicity(ONE2ONE).make()
mgmt.addConnection(toEdge, transaction, person)
mgmt.addConnection(toEdge, transaction, entity)
mgmt.commit()
Checking connections of the schema ---------------- mgmt = graph.openManagement()
edges = mgmt.getRelationTypes(EdgeLabel.class)
fromEdge = edges[0]
toEdge = edges[1]
fromEdge.mappedConnections().size() // as I would expect, it has two connections
toEdge.mappedConnections().size() // why 4 connections are here? I would expect only two connections similarly to the FROM edge
mgmt.close() -------------- Could you please have a look and let me know if this is a feature or a bug? Thanks, Peter
|
|
Re: [DISCUSS] JanusGraph 0.6.1 release
Hi Florian,
It will definitely help me to try some parts of automation for 0.6.1 release. I will work on it today's evening and tomorrow. If I bump into the problem and can't start a release till Monday (January 17), please, go ahead with the release without waiting for me. Best regards, Oleksandr
|
|
Re: [DISCUSS] JanusGraph 0.6.1 release
Florian Hockmann
Hi Oleksandr,
automating the release process or at least parts of it would definitely be great!
Right now, we have all issues and PRs closed that are linked to the milestone for 0.6.1 so we could in general proceed with the release process, but I think it wouldn’t be a big problem if we’d delay the release a bit so we could already use an automated release process.
Do you think that it makes sense to wait with the 0.6.1 release for this? Would it maybe even help if you could try out some of the automation directly during the release process? Or would this delay the release too much?
Von: janusgraph-dev@... <janusgraph-dev@...> Im Auftrag von Oleksandr Porunov
Hi,
|
|
Re: [DISCUSS] JanusGraph 0.6.1 release
Hi,
I'm in favor of releasing 0.6.1 version. That said, it would make sense to automate the releasing process as Jan mentioned. I'm planning to try to work on automation or partial automation this week. The main reason for that is to have deterministic way of making those builds. At this moment when we are creating `jar`s which we are publishing everywhere - we are creating it locally on a release manager's computer with their own `java` installation. Yes, the release manager signs every resource but we have no idea what version of Java was used to create this build. In theory if the release manager have vulnerable machine it could lead to infecting that `jar` with malicious code. I think, it would be better to make all the builds in GH actions and use only them for release artifacts. I will try to work on that this week but I'm good if you release 0.6.1 using the current release process. Just use openjdk 1.8.<latest> for the release. Best regards, Oleksandr
|
|
Re: [DISCUSS] JanusGraph 0.6.1 release
Jansen, Jan
Hi
Von: janusgraph-dev@... <janusgraph-dev@...> im Auftrag von Florian Hockmann <fh@...>
Gesendet: Mittwoch, 5. Januar 2022 11:38:48 An: janusgraph-dev@... Betreff: Re: [janusgraph-dev] [DISCUSS] JanusGraph 0.6.1 release Given that there haven’t been any responses, I assume that there are no major objections to releasing 0.6.1 soon but that there is also no high interest in getting it out as a hotfix release as soon as possible (which is OK in my opinion given that probably not many users are using the janusgraph.sh script to start JG+Cassandra+ES all at once). So, I suggest that we just treat it as a normal maintenance release that will also contain the workaround for the Log4j CVE in ES among other fixes.
Are there any open issues / PRs that you think should be included in 0.6.1?
PR #2899 is already linked to the milestone v0.6.1 and it fixes a NullPointerException so I’d at least wait for that PR to be merged for 0.6.1: https://github.com/JanusGraph/janusgraph/pull/2899
Does anyone want to volunteer to be release manager for this release? Otherwise, I can also do it.
Von: janusgraph-dev@... <janusgraph-dev@...>
Im Auftrag von Florian Hockmann
Hi,
we wanted to release patch versions shortly given the CVE in Log4j to mitigate it in the Elasticsearch version we distribute with the pre-packaged distribution: https://lists.lfaidata.foundation/g/janusgraph-dev/message/1554
I just merged a PR to the v0.6 branch with a simple mitigation: https://github.com/JanusGraph/janusgraph/pull/2892
As I explained in a comment in that PR, the mitigation now only landed in v0.6 and not in v0.5 as we don’t have any continuous integration for that branch and in general, I’m not sure it’s worth the effort to release a patch release for that branch, given that it only helps users who start JanusGraph together with ES from the `bin/janusgraph.sh` script.
So, we could release 0.6.1 now with the simple mitigation for ES. But if we want to release soon, then the release date would be in the middle of the holiday season. That would at least open the question whether someone is available over the holiday period to handle the release. Since I will not be available myself, I personally suggest that we postpone the release to the beginning of January. If, however someone is willing to act as release manager during the holiday period, then we can still do the release soon. We would of course also need at least 3 TSC members for the VOTE, but I guess that should be possible if we could start the VOTE already in the beginning of next week.
Apart from the release date and who will be the release manager, are there any important issues still open that really need to be fixed for 0.6.1? I think we should try to release 0.6.1 as soon as possible so new users don’t start with a vulnerable Elasticsearch installation and we can release 0.6.2 soon after, if necessary, but if you see any issues that we should really include, then we can discuss that of course.
Other questions that we could discuss for this release:
Best regards, Florian
[2]: https://github.com/JanusGraph/janusgraph/pull/2892#issuecomment-993536460
|
|
Re: [DISCUSS] JanusGraph 0.6.1 release
Florian Hockmann
Given that there haven’t been any responses, I assume that there are no major objections to releasing 0.6.1 soon but that there is also no high interest in getting it out as a hotfix release as soon as possible (which is OK in my opinion given that probably not many users are using the janusgraph.sh script to start JG+Cassandra+ES all at once). So, I suggest that we just treat it as a normal maintenance release that will also contain the workaround for the Log4j CVE in ES among other fixes.
Are there any open issues / PRs that you think should be included in 0.6.1?
PR #2899 is already linked to the milestone v0.6.1 and it fixes a NullPointerException so I’d at least wait for that PR to be merged for 0.6.1: https://github.com/JanusGraph/janusgraph/pull/2899
Does anyone want to volunteer to be release manager for this release? Otherwise, I can also do it.
Von: janusgraph-dev@... <janusgraph-dev@...> Im Auftrag von Florian Hockmann
Hi,
we wanted to release patch versions shortly given the CVE in Log4j to mitigate it in the Elasticsearch version we distribute with the pre-packaged distribution: https://lists.lfaidata.foundation/g/janusgraph-dev/message/1554
I just merged a PR to the v0.6 branch with a simple mitigation: https://github.com/JanusGraph/janusgraph/pull/2892
As I explained in a comment in that PR, the mitigation now only landed in v0.6 and not in v0.5 as we don’t have any continuous integration for that branch and in general, I’m not sure it’s worth the effort to release a patch release for that branch, given that it only helps users who start JanusGraph together with ES from the `bin/janusgraph.sh` script.
So, we could release 0.6.1 now with the simple mitigation for ES. But if we want to release soon, then the release date would be in the middle of the holiday season. That would at least open the question whether someone is available over the holiday period to handle the release. Since I will not be available myself, I personally suggest that we postpone the release to the beginning of January. If, however someone is willing to act as release manager during the holiday period, then we can still do the release soon. We would of course also need at least 3 TSC members for the VOTE, but I guess that should be possible if we could start the VOTE already in the beginning of next week.
Apart from the release date and who will be the release manager, are there any important issues still open that really need to be fixed for 0.6.1? I think we should try to release 0.6.1 as soon as possible so new users don’t start with a vulnerable Elasticsearch installation and we can release 0.6.2 soon after, if necessary, but if you see any issues that we should really include, then we can discuss that of course.
Other questions that we could discuss for this release:
Best regards, Florian
[2]: https://github.com/JanusGraph/janusgraph/pull/2892#issuecomment-993536460
|
|
[DISCUSS] JanusGraph 0.6.1 release
Florian Hockmann
Hi,
we wanted to release patch versions shortly given the CVE in Log4j to mitigate it in the Elasticsearch version we distribute with the pre-packaged distribution: https://lists.lfaidata.foundation/g/janusgraph-dev/message/1554
I just merged a PR to the v0.6 branch with a simple mitigation: https://github.com/JanusGraph/janusgraph/pull/2892
As I explained in a comment in that PR, the mitigation now only landed in v0.6 and not in v0.5 as we don’t have any continuous integration for that branch and in general, I’m not sure it’s worth the effort to release a patch release for that branch, given that it only helps users who start JanusGraph together with ES from the `bin/janusgraph.sh` script.
So, we could release 0.6.1 now with the simple mitigation for ES. But if we want to release soon, then the release date would be in the middle of the holiday season. That would at least open the question whether someone is available over the holiday period to handle the release. Since I will not be available myself, I personally suggest that we postpone the release to the beginning of January. If, however someone is willing to act as release manager during the holiday period, then we can still do the release soon. We would of course also need at least 3 TSC members for the VOTE, but I guess that should be possible if we could start the VOTE already in the beginning of next week.
Apart from the release date and who will be the release manager, are there any important issues still open that really need to be fixed for 0.6.1? I think we should try to release 0.6.1 as soon as possible so new users don’t start with a vulnerable Elasticsearch installation and we can release 0.6.2 soon after, if necessary, but if you see any issues that we should really include, then we can discuss that of course.
Other questions that we could discuss for this release:
Best regards, Florian
[2]: https://github.com/JanusGraph/janusgraph/pull/2892#issuecomment-993536460
|
|
JanusGraph and Apache Log4J CVE-2021-44228 (aka "log4shell")
Florian Hockmann
A critical security vulnerability was reported in Apache Log4j 2 on December 9, 2021: CVE-2021-44228 [1]. JanusGraph itself is not directly affected by this vulnerability as it still uses log4j 1.2 and the CVE only affects log4j2.
However, the two index backends Elasticsearch and Apache Solr are affected by the CVE. Both projects have already published reports on how they are affected and how the vulnerability can be mitigated:
Apache HBase, Apache TinkerPop, Apache Spark, and Apache Hadoop are all also still on log4j 1 and are therefore also not affected.
We recommend all JanusGraph users who use JanusGraph together with Elasticsearch or Apache Solr to follow the recommendations listed in the reports linked above. This mostly comes down to updating the backends once an update with a fix is available and to set the JVM option
We also plan to release patch versions for 0.5 and 0.6 shortly for the Elasticsearch version that is distributed as part of the pre-packed (“full”) distribution of JanusGraph to adopt these recommendations by default for users who start JanusGraph via the `bin/janusgraph.sh` script which also starts Elasticsearch.
Given that Log4j 1 already reached EOL (end of life) and is also affected by other CVEs, we also plan to move away from that version in a future release of JanusGraph.
Best regards, The JanusGraph Technical Steering Committee
|
|
TinkerPop Discord Server
Florian Hockmann
Hi,
In case you haven’t seen it already: TinkerPop now has a Discord server, and that Discord has its own channel for implementers which of course includes graph databases like JanusGraph. I think it could be interesting for us to discuss everything related to TinkerPop there directly with the TinkerPop contributors or with developers from other graph databases.
Here is the invite link for the Discord Server:
Best regards, Florian
|
|
[ANNOUNCE] JanusGraph 0.6.0 Release
The JanusGraph Technical Steering Committee is excited to announce the release of JanusGraph 0.6.0.
JanusGraph is an Apache TinkerPop enabled property graph database with support for a variety of storage and indexing backends. Thank you to all of the contributors. Notable new features in this release include:
The release artifacts can be found at this location: https://github.com/JanusGraph/janusgraph/releases/tag/v0.6.0 A full binary distribution is provided for user convenience: https://github.com/JanusGraph/janusgraph/releases/download/v0.6.0/janusgraph-full-0.6.0.zip A truncated binary distribution is provided:
https://github.com/JanusGraph/janusgraph/releases/download/v0.6.0/janusgraph-0.6.0.zip The online docs can be found here: https://docs.janusgraph.org To view the resolved issues and commits check the milestone here:
https://github.com/JanusGraph/janusgraph/milestone/17?closed=1Thank you very much,
Oleksandr Porunov
|
|
[RESULT][VOTE] JanusGraph 0.6.0 release
This vote is now closed with a total of 5 +1s, no +0s and no -1s. The results are:
BINDING VOTES: +1 (3 -- Oleksandr Porunov, Boxuan Li, Florian Hockmann) 0 (0) -1 (0) NON-BINDING VOTES: +1 (Rafael Fernandes, Clement de Groc) 0 (0) -1 (0) Thank you very much, Oleksandr Porunov
|
|
Re: [VOTE] JanusGraph 0.6.0 release
Florian Hockmann
Thanks for fixing this!
I performed the same tests again and now everything looks good. So:
VOTE +1
Von: janusgraph-dev@... <janusgraph-dev@...> Im Auftrag von Oleksandr Porunov
I re-uploaded a new staring release.
|
|
Re: [VOTE] JanusGraph 0.6.0 release
I re-uploaded a new staring release.
I checked and successfully closed the staging release (1035). GitHub tag v0.6.0 has been updated to point on the latest commit. All the GitHub artifacts has been rebuilt and updated. The issue found by Florian Hockmann is now fixed. I'm reopening the voting process. This [VOTE] will open for the next 3 days --- closing Monday, September 6, 2021 at 3:45 PM GMT+3. All are welcome to review and vote on the release, but only votes from TSC members are binding. My vote is +1. Thank you, Oleksandr Porunov
|
|
Re: [VOTE] JanusGraph 0.6.0 release
Opened a PR here which fixes the issue with `janusgraph.sh`: https://github.com/JanusGraph/janusgraph/pull/2781
|
|
Re: [VOTE] JanusGraph 0.6.0 release
Confirming the issue which is found by Florian Hockmann.
I was trying to load `GraphOfTheGodsFactory.load(graph)` and couldn't: GraphOfTheGodsFactory.load(graph)
The indexing backend with name "search" is not defined. Specify an existing indexing backend or use GraphOfTheGodsFactory.loadWithoutMixedIndex(graph,true) to load without the use of an indexing backend.
So, it looks like `janusgraph-server.sh` is using inmemory option without index backend instead of using Cassandra + ElasticSearch as requested by `janusgraph.sh` script. Quickly checking `janusgraph.sh` script I see it starts `janusgraph-server.sh` with the following options: ``` "$BIN"/janusgraph-server.sh console conf/gremlin-server/gremlin-server-cql-es.yaml & ``` Checking `janusgraph-server.sh` script I see the following code: ``` if [[ -z "$JANUSGRAPH_YAML" ]]; then
JANUSGRAPH_YAML="$JANUSGRAPH_CONF/gremlin-server/gremlin-server.yaml"
fi
if [[ ! -r "$JANUSGRAPH_YAML" ]]; then
# try relative to conf
JANUSGRAPH_YAML="$JANUSGRAPH_CONF/gremlin-server/gremlin-server.yaml"
if [[ ! -r "$JANUSGRAPH_YAML" ]]; then
echo WARNING: $JANUSGRAPH_YAML is unreadable
fi
fi
```... if [[ -n "$1" ]] ; then
if [[ -r "$1" ]]; then
JANUSGRAPH_YAML="$1"
startForeground
elif [[ -r "$JANUSGRAPH_HOME/$1" ]] ; then
JANUSGRAPH_YAML="$JANUSGRAPH_HOME/$1"
startForeground
fi
echo Configuration file not found.
fi
... So, it looks like this script never expects path to be passed as a second argument. Thus, it looks like a second argument `conf/gremlin-server/gremlin-server-cql-es.yaml` is simply ignored and the default `$JANUSGRAPH_CONF/gremlin-server/gremlin-server.yaml` is used. But `$JANUSGRAPH_CONF/gremlin-server/gremlin-server.yaml` is using inmemory option by default. I believe this is the root cause for this problem but didn't had a chance to actually verify that yet. If anyone has time today to fix this issue, please, submit a PR, otherwise I will try to fix it in a day or two.
|
|
Re: [VOTE] JanusGraph 0.6.0 release
Florian Hockmann
I just did a quick test of the two distribution archives which just consists of starting JanusGraph Server and connecting to it via the remote console to create a vertex. For the full distribution, I then also restarted the server to verify that the vertex was persisted in Cassandra. This unfortunately failed for 0.6.0. The graph was completely empty after executing ‘bin/janusgraph.sh stop’ and ‘start’. I’m not sure what’s causing this, but I think that we should look into it. At a first glance, the problem seems to be related to Cassandra as that didn’t create a directory ‘db/Cassandra/data/janusgraph’ for the JanusGraph keyspace. I only see directories for the system keyspaces there, but for 0.5.3 where this still worked, there was also a janusgraph keyspace directory.
Other then that, I also noticed that the Changelog mentions the removal of Thrift and refers to this repository where the Thrift code should be moved to: https://github.com/JanusGraph/janusgraph-cassandra Unfortunately, the PR that would actually add the code to master branch there is still open: https://github.com/JanusGraph/janusgraph-cassandra/pull/3 I don’t think that this should stop the release, but we should try to merge it at least soon after the release.
Von: janusgraph-dev@... <janusgraph-dev@...> Im Auftrag von Boxuan Li
I did the following tests:
0.5.3 (query.batch=true): gremlin> clock(10) { graph.traversal().V().has("name", "inner0").toList() } ==>478.1922667999999 gremlin> clock(10) { graph.traversal().V().has("age", 0).toList() } ==>17873.4094548 gremlin> clock(10) {graph.traversal().V().has("name", "outer").out().out().toList()} ==>14500.0969486
0.6.0 (query.batch = true, query.limit-batch-size=true) gremlin> clock(10) { graph.traversal().V().has("name", "inner0").toList() } ==>267.27335619999997 gremlin> clock(10) { graph.traversal().V().has("age", 0).toList() } ==>7432.816540099999 gremlin> clock(10) {graph.traversal().V().has("name", "outer").out().out().toList()} ==>11786.6717737
My vote is +1.
|
|
Re: [VOTE] JanusGraph 0.6.0 release
Boxuan Li
I did the following tests:
0.5.3 (query.batch=true): gremlin> clock(10) { graph.traversal().V().has("name", "inner0").toList() } ==>478.1922667999999 gremlin> clock(10) { graph.traversal().V().has("age", 0).toList() } ==>17873.4094548 gremlin> clock(10) {graph.traversal().V().has("name", "outer").out().out().toList()} ==>14500.0969486
0.6.0 (query.batch = true, query.limit-batch-size=true) gremlin> clock(10) { graph.traversal().V().has("name", "inner0").toList() } ==>267.27335619999997 gremlin> clock(10) { graph.traversal().V().has("age", 0).toList() } ==>7432.816540099999 gremlin> clock(10) {graph.traversal().V().has("name", "outer").out().out().toList()} ==>11786.6717737
My vote is +1.
|
|
Re: [VOTE] JanusGraph 0.6.0 release
I also executed all TinkerPop tests again with `mvn clean install -Dtest.skip.tp=false -DskipTests=true`. All tests passed.
|
|
Re: [VOTE] JanusGraph 0.6.0 release
I realized that I was building with Java 11 instead of Java 8. I switched to the Java 8 (OpenJDK 1.8.0_302), dropped previous staging repository, rebuilt JanusGraph using Java 8, re-uploaded new artifacts to the staging repository and to GitHub release. Checked that `graph = JanusGraphFactory.open("inmemory")` now works in Gremlin console. I believe the above issue should be fixed now.
|
|