Reminder: Marquez community meeting

When:
07/28/2022
11:00am to 12:00pm
(UTC-07:00) America/Los Angeles

Where:
https://us02web.zoom.us/j/89344845719?pwd=Y09RZkxMZHc2U3pOTGZ6SnVMUUVoQT09

Organizer: Marquez TSC

View Event

Description:
Marquez community meeting:

•  introductions
•  community topic discussion

Hi Team,

Here are the results from the July 2022 license scan of the Marquez project. The scan was performed using the Linux Foundation Fossology server. Licenses and copyrights were examined.

The key findings (if any) and license summary can be found in the HTML report, the list of files in the spreadsheet, and also find the SPDX file listed below:

REPORTS:

llfai/marquez, code pulled 2022-07-02
- report: https://lfscanning.org/reports/lfai/marquez-2022-07-02-3d1e5990-01ba-4086-895e-3f9c805bbc78.html
- xlsx: https://lfscanning.org/reports/lfai/marquez-2022-07-02-3d1e5990-01ba-4086-895e-3f9c805bbc78.xlsx
- spdx: https://github.com/lfscanning/spdx-lfai/tree/master/marquez/2022-07/marquez-2022-07-02.spdx

NOTE: The key findings listed are not new, they were found during one of the PREVIOUS scans.

Please feel free to contact me with any questions about the scan results. Be sure to reply to me directly as I may not get an email sent directly to the distribution list.

Thanks, Jeff

Jeff Shapiro
408-910-7792
jshapiro@...

Branch: refs/heads/renovate/jdbi3version
Home: https://github.com/MarquezProject/marquez
Commit: eea9e43bef622ddb002bac8798b838398ab36204
https://github.com/MarquezProject/marquez/commit/eea9e43bef622ddb002bac8798b838398ab36204
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date: 2022-07-16 (Sat, 16 Jul 2022)

Changed paths:
M api/build.gradle

Log Message:
-----------
chore(deps): update jdbi3version to v3.30.0

Signed-off-by: Renovate Bot <bot@...>

Branch: refs/heads/renovate/postgresql-11.x
Home: https://github.com/MarquezProject/marquez
Commit: bac1fab66372f48f74fbe4c0487c1db99a0d562e
https://github.com/MarquezProject/marquez/commit/bac1fab66372f48f74fbe4c0487c1db99a0d562e
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date: 2022-07-16 (Sat, 16 Jul 2022)

Changed paths:
M chart/Chart.lock
M chart/Chart.yaml

Log Message:
-----------
chore(deps): update helm release postgresql to v11.6.16

Signed-off-by: Renovate Bot <bot@...>

Branch: refs/heads/renovate/common-1.x
Home: https://github.com/MarquezProject/marquez
Commit: 7993c6d86174cb5af5e58a3be465166aad6a455b
https://github.com/MarquezProject/marquez/commit/7993c6d86174cb5af5e58a3be465166aad6a455b
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date: 2022-07-16 (Sat, 16 Jul 2022)

Changed paths:
M chart/Chart.lock
M chart/Chart.yaml

Log Message:
-----------
chore(deps): update helm release common to v1.16.1

Signed-off-by: Renovate Bot <bot@...>

Branch: refs/heads/renovate/org.dhatim-dropwizard-sentry-2.x
Home: https://github.com/MarquezProject/marquez
Commit: 706e4bc6b9a86c6bf32b6857547371d4c6d370ca
https://github.com/MarquezProject/marquez/commit/706e4bc6b9a86c6bf32b6857547371d4c6d370ca
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date: 2022-07-16 (Sat, 16 Jul 2022)

Changed paths:
M api/build.gradle

Log Message:
-----------
chore(deps): update dependency org.dhatim:dropwizard-sentry to v2.1.1-2

Signed-off-by: Renovate Bot <bot@...>

Branch: refs/heads/renovate/patch-testcontainersversion
Home: https://github.com/MarquezProject/marquez
Commit: cb922da3a3a41e6ae39d31ef601cbed7bfa5a4b8
https://github.com/MarquezProject/marquez/commit/cb922da3a3a41e6ae39d31ef601cbed7bfa5a4b8
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date: 2022-07-16 (Sat, 16 Jul 2022)

Changed paths:
M api/build.gradle

Log Message:
-----------
chore(deps): update testcontainersversion to v1.17.3

Signed-off-by: Renovate Bot <bot@...>

Branch: refs/heads/renovate/postgresql-11.1.x
Home: https://github.com/MarquezProject/marquez
Commit: 9f0895bacd1784d6c7f79812e6811ac62c10cf11
https://github.com/MarquezProject/marquez/commit/9f0895bacd1784d6c7f79812e6811ac62c10cf11
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date: 2022-07-16 (Sat, 16 Jul 2022)

Changed paths:
M chart/Chart.lock
M chart/Chart.yaml

Log Message:
-----------
chore(deps): update helm release postgresql to v11.1.29

Signed-off-by: Renovate Bot <bot@...>

Branch: refs/heads/renovate/patch-dropwizard-monorepo
Home: https://github.com/MarquezProject/marquez
Commit: 3678f556613fbb8bfc9c0aee4db67bf8e78c6c46
https://github.com/MarquezProject/marquez/commit/3678f556613fbb8bfc9c0aee4db67bf8e78c6c46
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date: 2022-07-16 (Sat, 16 Jul 2022)

Changed paths:
M build.gradle

Log Message:
-----------
chore(deps): update dropwizard monorepo to v2.1.1

Signed-off-by: Renovate Bot <bot@...>

Branch: refs/heads/renovate/patch-postgresqlversion
Home: https://github.com/MarquezProject/marquez
Commit: f1a54145888868540981aeb1223b18f4f7fa90d1
https://github.com/MarquezProject/marquez/commit/f1a54145888868540981aeb1223b18f4f7fa90d1
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date: 2022-07-16 (Sat, 16 Jul 2022)

Changed paths:
M build.gradle

Log Message:
-----------
chore(deps): update dependency org.postgresql:postgresql to v42.3.6

Signed-off-by: Renovate Bot <bot@...>

Branch: refs/heads/renovate/org.flywaydb-flyway-core-8.5.x
Home: https://github.com/MarquezProject/marquez
Commit: c3b5b58da15bfaa716ee0b6f2822a8f5348ad211
https://github.com/MarquezProject/marquez/commit/c3b5b58da15bfaa716ee0b6f2822a8f5348ad211
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date: 2022-07-16 (Sat, 16 Jul 2022)

Changed paths:
M api/build.gradle

Log Message:
-----------
chore(deps): update dependency org.flywaydb:flyway-core to v8.5.13

Signed-off-by: Renovate Bot <bot@...>

Branch: refs/heads/renovate/patch-sentryversion
Home: https://github.com/MarquezProject/marquez
Commit: 283c718e28bd13a1145076222b886406c97eb224
https://github.com/MarquezProject/marquez/commit/283c718e28bd13a1145076222b886406c97eb224
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date: 2022-07-16 (Sat, 16 Jul 2022)

Changed paths:
M api/build.gradle

Log Message:
-----------
chore(deps): update dependency io.sentry:sentry to v5.7.4

Signed-off-by: Renovate Bot <bot@...>

Branch: refs/heads/namespace_no_conflict
Home: https://github.com/MarquezProject/marquez

Branch: refs/heads/main
Home: https://github.com/MarquezProject/marquez
Commit: 78eeae02c63018d6e1c11e7d17fbdb181950fde4
https://github.com/MarquezProject/marquez/commit/78eeae02c63018d6e1c11e7d17fbdb181950fde4
Author: Michael Collado <40346148+collado-mike@...>
Date: 2022-07-15 (Fri, 15 Jul 2022)

Changed paths:
M CHANGELOG.md
M api/src/main/java/marquez/db/JobContextDao.java
M api/src/main/java/marquez/db/JobVersionDao.java
M api/src/main/java/marquez/db/NamespaceDao.java
M api/src/main/java/marquez/db/OpenLineageDao.java
M api/src/main/java/marquez/db/RunArgsDao.java

Log Message:
-----------
Avoid locking common records and remove transaction annotation (#2036)

* Fix NamespaceDao to stop updating namespace record on conflict

Signed-off-by: Michael Collado <collado.mike@...>

* Update JobContextDao to stop locking job context on conflict

Signed-off-by: Michael Collado <collado.mike@...>

* Update RunArgsDao to stop locking run args on conflict

Signed-off-by: Michael Collado <collado.mike@...>

* Remove transaction annotations

Signed-off-by: Michael Collado <collado.mike@...>

* Update changelog

Signed-off-by: Michael Collado <collado.mike@...>

Branch: refs/heads/namespace_no_conflict
Home: https://github.com/MarquezProject/marquez
Commit: cdee3314abf7acc8c6e18946e9382d775b1fba2e
https://github.com/MarquezProject/marquez/commit/cdee3314abf7acc8c6e18946e9382d775b1fba2e
Author: Sam Holmberg <sam@...>
Date: 2022-07-15 (Fri, 15 Jul 2022)

Changed paths:
M spec/openapi.yml

Log Message:
-----------
Fix dataset openapi model (#2038)

Signed-off-by: Sam Holmberg <sam@...>


Commit: dfb0aee21c7ffa8755fa8a3a01dd72098a5bdf6e
https://github.com/MarquezProject/marquez/commit/dfb0aee21c7ffa8755fa8a3a01dd72098a5bdf6e
Author: Willy Lulciuc <willy@...>
Date: 2022-07-15 (Fri, 15 Jul 2022)

Changed paths:
M spec/openapi.yml

Log Message:
-----------
Merge branch 'main' into namespace_no_conflict


Compare: https://github.com/MarquezProject/marquez/compare/0d1f82d9f46f...dfb0aee21c7f

Branch: refs/heads/main
Home: https://github.com/MarquezProject/marquez
Commit: cdee3314abf7acc8c6e18946e9382d775b1fba2e
https://github.com/MarquezProject/marquez/commit/cdee3314abf7acc8c6e18946e9382d775b1fba2e
Author: Sam Holmberg <sam@...>
Date: 2022-07-15 (Fri, 15 Jul 2022)

Changed paths:
M spec/openapi.yml

Log Message:
-----------
Fix dataset openapi model (#2038)

Signed-off-by: Sam Holmberg <sam@...>

Branch: refs/heads/fix_recursive_views_perf
Home: https://github.com/MarquezProject/marquez
Commit: 41834d550b3d88a6e02bbd9dd13d1a7243f43ced
https://github.com/MarquezProject/marquez/commit/41834d550b3d88a6e02bbd9dd13d1a7243f43ced
Author: Michael Collado <collado.mike@...>
Date: 2022-07-14 (Thu, 14 Jul 2022)

Changed paths:
M api/src/main/java/marquez/db/JobDao.java
M api/src/main/java/marquez/db/OpenLineageDao.java
A api/src/main/resources/marquez/db/migration/V45__update_jobs_view_rule.sql
M api/src/test/java/marquez/MarquezAppIntegrationTest.java
M api/src/test/java/marquez/db/BackfillTestUtils.java
M api/src/test/java/marquez/service/OpenLineageServiceIntegrationTest.java

Log Message:
-----------
Update JobsDao to stop updating existing jobs unnecessarily

Signed-off-by: Michael Collado <collado.mike@...>


Commit: 7313f1d98c7f4e115e6ea202e524966f68b9c41c
https://github.com/MarquezProject/marquez/commit/7313f1d98c7f4e115e6ea202e524966f68b9c41c
Author: Michael Collado <collado.mike@...>
Date: 2022-07-14 (Thu, 14 Jul 2022)

Changed paths:
M api/src/main/java/marquez/db/JobVersionDao.java
M api/src/main/java/marquez/db/OpenLineageDao.java
M api/src/main/java/marquez/db/migrations/V44_2__BackfillAirflowParentRuns.java
M api/src/main/java/marquez/db/migrations/V44_3_BackfillJobsWithParents.java
M api/src/main/java/marquez/service/RunService.java
M api/src/test/java/marquez/db/JobVersionDaoTest.java
M api/src/test/java/marquez/db/RunDaoTest.java

Log Message:
-----------
Remove need to query JobRow on run completion

Signed-off-by: Michael Collado <collado.mike@...>


Commit: 3e664d7b9d7dc8103b210e600965adc4be8c9b80
https://github.com/MarquezProject/marquez/commit/3e664d7b9d7dc8103b210e600965adc4be8c9b80
Author: Michael Collado <collado.mike@...>
Date: 2022-07-14 (Thu, 14 Jul 2022)

Changed paths:
M api/src/main/java/marquez/db/JobDao.java
M api/src/main/resources/marquez/db/migration/V45__update_jobs_view_rule.sql

Log Message:
-----------
Remove insert rule for job dao

Signed-off-by: Michael Collado <collado.mike@...>


Commit: 35366ff7302b4f188dfaccacebc12239040b0e96
https://github.com/MarquezProject/marquez/commit/35366ff7302b4f188dfaccacebc12239040b0e96
Author: Michael Collado <collado.mike@...>
Date: 2022-07-14 (Thu, 14 Jul 2022)

Changed paths:
M api/src/main/java/marquez/db/JobDao.java

Log Message:
-----------
Change jobdao to query before insert

Signed-off-by: Michael Collado <collado.mike@...>


Commit: 42d4c670f10dc1e9308d05d6d4e2d8b4155cd390
https://github.com/MarquezProject/marquez/commit/42d4c670f10dc1e9308d05d6d4e2d8b4155cd390
Author: Michael Collado <collado.mike@...>
Date: 2022-07-14 (Thu, 14 Jul 2022)

Changed paths:
M api/src/main/java/marquez/db/RunDao.java
M api/src/main/java/marquez/service/models/Run.java

Log Message:
-----------
Stop writing job context to runs

Signed-off-by: Michael Collado <collado.mike@...>


Commit: aa2e623ff83dd12fbecbfaac84482a957d941d09
https://github.com/MarquezProject/marquez/commit/aa2e623ff83dd12fbecbfaac84482a957d941d09
Author: Michael Collado <collado.mike@...>
Date: 2022-07-14 (Thu, 14 Jul 2022)

Changed paths:
M api/src/main/java/marquez/db/JobDao.java

Log Message:
-----------
Fix job dao conflict for parent id string

Signed-off-by: Michael Collado <collado.mike@...>


Commit: be602138a3d329fd23e38d17aa0d5b40ae281bf4
https://github.com/MarquezProject/marquez/commit/be602138a3d329fd23e38d17aa0d5b40ae281bf4
Author: Michael Collado <collado.mike@...>
Date: 2022-07-14 (Thu, 14 Jul 2022)

Changed paths:
M api/src/main/java/marquez/db/JobDao.java

Log Message:
-----------
Undid job query before insert

Signed-off-by: Michael Collado <collado.mike@...>


Commit: 5169781b6200b06a2939c693e6c145af38528670
https://github.com/MarquezProject/marquez/commit/5169781b6200b06a2939c693e6c145af38528670
Author: Michael Collado <collado.mike@...>
Date: 2022-07-14 (Thu, 14 Jul 2022)

Changed paths:
M api/src/main/java/marquez/db/OpenLineageDao.java
M api/src/main/java/marquez/db/RunDao.java
A api/src/main/java/marquez/db/mappers/RunRowMapper.java
M api/src/main/java/marquez/db/models/ExtendedRunRow.java
M api/src/main/java/marquez/db/models/RunRow.java
M api/src/main/java/marquez/service/DatasetService.java
M api/src/main/java/marquez/service/JobService.java
M api/src/main/java/marquez/service/OpenLineageService.java
M api/src/main/java/marquez/service/RunService.java
M api/src/test/java/marquez/db/BackfillTestUtils.java
M api/src/test/java/marquez/db/DbTestUtils.java
M api/src/test/java/marquez/db/JobVersionDaoTest.java
M api/src/test/java/marquez/db/RunDaoTest.java
M api/src/test/java/marquez/db/migrations/V44_3_BackfillJobsWithParentsTest.java

Log Message:
-----------
Changed RunDao to use simple RunRow rather than ExtendedRunRow where possible

Signed-off-by: Michael Collado <collado.mike@...>


Commit: c5a7dc350d8b07df51b0f5c3b61c776f5175f70f
https://github.com/MarquezProject/marquez/commit/c5a7dc350d8b07df51b0f5c3b61c776f5175f70f
Author: Michael Collado <collado.mike@...>
Date: 2022-07-15 (Fri, 15 Jul 2022)

Changed paths:
M api/src/main/java/marquez/db/JobDao.java
M api/src/main/java/marquez/db/OpenLineageDao.java
M api/src/main/java/marquez/db/RunDao.java
M api/src/main/resources/marquez/db/migration/V45__update_jobs_view_rule.sql
M api/src/test/java/marquez/MarquezAppIntegrationTest.java
M api/src/test/java/marquez/db/BackfillTestUtils.java
M api/src/test/java/marquez/db/JobDaoTest.java
M api/src/test/java/marquez/db/migrations/V44_3_BackfillJobsWithParentsTest.java
M api/src/test/java/marquez/service/OpenLineageServiceIntegrationTest.java

Log Message:
-----------
Refactor jobs_view to use job_fqn table

Signed-off-by: Michael Collado <collado.mike@...>


Compare: https://github.com/MarquezProject/marquez/compare/41834d550b3d%5E...c5a7dc350d8b

Branch: refs/heads/renovate/common-1.x
Home: https://github.com/MarquezProject/marquez
Commit: 4744503dfbb08f19b5c899e5f4690c3875132581
https://github.com/MarquezProject/marquez/commit/4744503dfbb08f19b5c899e5f4690c3875132581
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date: 2022-07-14 (Thu, 14 Jul 2022)

Changed paths:
M chart/Chart.lock
M chart/Chart.yaml

Log Message:
-----------
chore(deps): update helm release common to v1.16.1

Signed-off-by: Renovate Bot <bot@...>

Branch: refs/heads/namespace_no_conflict
Home: https://github.com/MarquezProject/marquez
Commit: 0d1f82d9f46f95733fbf2bfde96a151b5e1f3d32
https://github.com/MarquezProject/marquez/commit/0d1f82d9f46f95733fbf2bfde96a151b5e1f3d32
Author: Michael Collado <collado.mike@...>
Date: 2022-07-14 (Thu, 14 Jul 2022)

Changed paths:
M CHANGELOG.md

Log Message:
-----------
Update changelog

Signed-off-by: Michael Collado <collado.mike@...>

Branch: refs/heads/namespace_no_conflict
Home: https://github.com/MarquezProject/marquez
Commit: b60579494e9d4b87c6f1cbfa1f8bbe93ab823d16
https://github.com/MarquezProject/marquez/commit/b60579494e9d4b87c6f1cbfa1f8bbe93ab823d16
Author: Michael Collado <collado.mike@...>
Date: 2022-07-14 (Thu, 14 Jul 2022)

Changed paths:
M api/src/main/java/marquez/db/NamespaceDao.java

Log Message:
-----------
Fix NamespaceDao to stop updating namespace record on conflict

Signed-off-by: Michael Collado <collado.mike@...>


Commit: a3a7e694c9a022d39b65ad362edefa0f26842d91
https://github.com/MarquezProject/marquez/commit/a3a7e694c9a022d39b65ad362edefa0f26842d91
Author: Michael Collado <collado.mike@...>
Date: 2022-07-14 (Thu, 14 Jul 2022)

Changed paths:
M api/src/main/java/marquez/db/JobContextDao.java

Log Message:
-----------
Update JobContextDao to stop locking job context on conflict

Signed-off-by: Michael Collado <collado.mike@...>


Commit: d22fa1083ab9bf3d556e43d084d2b3ef3a17f140
https://github.com/MarquezProject/marquez/commit/d22fa1083ab9bf3d556e43d084d2b3ef3a17f140
Author: Michael Collado <collado.mike@...>
Date: 2022-07-14 (Thu, 14 Jul 2022)

Changed paths:
M api/src/main/java/marquez/db/RunArgsDao.java

Log Message:
-----------
Update RunArgsDao to stop locking run args on conflict

Signed-off-by: Michael Collado <collado.mike@...>


Commit: b882e50a8c9761dcecc4646ea37c79faa76b9904
https://github.com/MarquezProject/marquez/commit/b882e50a8c9761dcecc4646ea37c79faa76b9904
Author: Michael Collado <collado.mike@...>
Date: 2022-07-14 (Thu, 14 Jul 2022)

Changed paths:
M api/src/main/java/marquez/db/JobVersionDao.java
M api/src/main/java/marquez/db/OpenLineageDao.java

Log Message:
-----------
Remove transaction annotations

Signed-off-by: Michael Collado <collado.mike@...>


Compare: https://github.com/MarquezProject/marquez/compare/fafe4f21a1b0...b882e50a8c97