Hi Boxuan, Issue seems to occurs when edge properties are retrieved: cache has expected size with g.V().outE().id() an not when I do g.V().outE().valueMap(); I am able to reproduce with following groovy script : - gremlin console
( launched with JAVA_OPTS="-Xmx1G -Xms1G" ./bin/gremlin.sh) on JG 0.5.3
- conf/janusgraph-cache.properties:
gremlin.graph=org.janusgraph.core.JanusGraphFactory
storage.backend=cql
storage.hostname=127.0.0.1
storage.port=9042
schema.default=logging
cache.db-cache: true
cache.db-cache-size: 50000000
cache.db-cache-time: 6000000
graph = JanusGraphFactory.open('conf/janusgraph-cache.properties')
g = graph.traversal()
// Schema creation graph.tx().rollback()
mgmt = g.getGraph().openManagement()
try {
deviceLabel = mgmt.makeVertexLabel('device').make()
nameProperty = mgmt.makePropertyKey("name").dataType(java.lang.String).cardinality(org.janusgraph.core.Cardinality.SINGLE).make()
mgmt.addProperties(deviceLabel, nameProperty)
measurementLabel = mgmt.makeEdgeLabel('measurement').unidirected().make()
deviceNameProperty = mgmt.makePropertyKey("deviceName").dataType(java.lang.String).cardinality(org.janusgraph.core.Cardinality.SINGLE).make()
physicalQuantityProperty = mgmt.makePropertyKey("physicalQuantity").dataType(java.lang.String).cardinality(org.janusgraph.core.Cardinality.SINGLE).make()
valueProperty = mgmt.makePropertyKey("value").dataType(java.lang.Double).cardinality(org.janusgraph.core.Cardinality.SINGLE).make()
timestampProperty = mgmt.makePropertyKey("timestamp").dataType(java.util.Date).cardinality(org.janusgraph.core.Cardinality.SINGLE).make()
mgmt.addProperties(measurementLabel, deviceNameProperty, physicalQuantityProperty, valueProperty, timestampProperty)
mgmt.buildIndex("deviceByName", Vertex.class).indexOnly(deviceLabel).addKey(nameProperty).buildCompositeIndex();
//mgmt.buildEdgeIndex(measurementLabel, 'measurementByTimestamp', Direction.OUT, Order.decr, timestampProperty);
mgmt.commit()
} catch (Exception e) {
mgmt.rollback();
throw e;
}
// Load data random = new Random();
startTs = System.currentTimeMillis();
for (i = 0; i < 100; i++) {
deviceId = g.addV("device").property("name", "device-" + i).id().next();
for (k = 0; k < 5000; k++) {
g.V(deviceId).addE("measurement").
property("deviceName", "device-" + i).
property("physicalQuantity", "physicalQuantity-" + random.nextInt(10)).
property("value", random.nextDouble()).
property("timestamp", new Date(startTs + k * 1000)).
iterate();
if (k % 1000 == 0) {
g.tx().commit();
}
}
log.info("Done i={}",i);
}
g.tx().commit();
// Request data for (i = 0; i < 100; i++) {
measurementsList = g.V().has("device", "name", "device-" + i).outE().valueMap().toList();
log.info("Got {} measurements for {}", measurementsList.size(), i);
}
g.tx().commit();
Le sam. 9 janv. 2021 à 05:21, BO XUAN LI < liboxuan@...> a écrit : Hi Nicolas,
Looks interesting. Your configs look fine and I couldn’t reproduce your problem. Could you provide some sample code to reproduce it?
Best regards, Boxuan
Hi Boxuan,
I have configured janusgraph with:
cache.db-cache-time: 600000
cache.db-cache: true
cache.db-cache-size: 50000000 index.search.elasticsearch.create.ext.number_of_replicas: 0
storage.buffer-size: 1024
index.search.elasticsearch.create.ext.number_of_shards: 1
cache.cache.db-cache-time: 0
index.search.index-name: dcbrain
index.search.backend: elasticsearch
storage.port: 9042
ids.block-size: 1000000
schema.default: logging
storage.cql.batch-statement-size: 50
index.search.hostname: dfe-elasticsearch
storage.backend: cql
storage.hostname: dfe-cassandra
storage.cql.local-max-requests-per-connection: 4096
index.search.port: 9200
I have load some data on the graph and dump memory. When I import this dump with jvisualVM, retained size for ExpirationKCVSCache 257 Mb when the limit should be 50 Mb.
<image.png>
Regards, Nicolas Le lun. 4 janv. 2021 à 13:11, BO XUAN LI < liboxuan@...> a écrit : Hi Nicolas,
Can you provide your configurations and the memory usage you observed?
Regards, Boxuan
Hi, I try to use Database Level Caching as described in https://docs.janusgraph.org/basics/cache/ but it seems to use more memory than the configured threshold (
cache.db-cache-size ). Does anyone use such a feature ? Is it production ready ?
Regards, Nicolas 
Ce message et ses pièces jointes peuvent contenir des informations confidentielles ou privilégiées et ne doivent donc pas être diffusés, exploités ou copiés sans autorisation. Si vous avez reçu ce message par erreur, veuillez le signaler a l'expéditeur et le détruire ainsi que les pièces jointes. Les messages électroniques étant susceptibles d'altération, DCbrain décline toute responsabilité si ce message a été altéré, déformé ou falsifié. Merci.
This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, DCbrain is not liable for messages that have been modified, changed or falsified. Thank you.
--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/CAD7qnB4SYvXq5A3vkzu44fERkySr2kPhsoZC-5%3DbBoz9KvzPnw%40mail.gmail.com.
--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/3B5EFE52-BE38-437B-B399-05AF4899F398%40connect.hku.hk.
-- Nicolas Trangosi Lead back +33 (0)6 77 86 66 44   
Ce message et ses pièces jointes peuvent contenir des informations confidentielles ou privilégiées et ne doivent donc pas être diffusés, exploités ou copiés sans autorisation. Si vous avez reçu ce message par erreur, veuillez le signaler a l'expéditeur et le détruire ainsi que les pièces jointes. Les messages électroniques étant susceptibles d'altération, DCbrain décline toute responsabilité si ce message a été altéré, déformé ou falsifié. Merci.
This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, DCbrain is not liable for messages that have been modified, changed or falsified. Thank you.
--
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/CAD7qnB7bY3bNwf3PVCLuuU%2BOT%2BAmGWnoEGtT00i6LQ8%2Bu5sHzw%40mail.gmail.com.
You received this message because you are subscribed to the Google Groups "JanusGraph users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to janusgraph-users+unsubscribe@....
To view this discussion on the web visit https://groups.google.com/d/msgid/janusgraph-users/D7009A96-D9BB-4F05-A792-372EB3F5CE34%40connect.hku.hk.
-- 
Nicolas Trangosi Lead back +33 (0)6 77 86 66 44
Ce message et ses pièces jointes peuvent contenir des informations confidentielles ou privilégiées et ne doivent donc pas être diffusés, exploités ou copiés sans autorisation. Si vous avez reçu ce message par erreur, veuillez le signaler a l'expéditeur et le détruire ainsi que les pièces jointes. Les messages électroniques étant susceptibles d'altération, DCbrain décline toute responsabilité si ce message a été altéré, déformé ou falsifié. Merci.
This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation. If you have received this email in error, please notify the sender and delete this message and its attachments. As emails may be altered, DCbrain is not liable for messages that have been modified, changed or falsified. Thank you.
|
