Move Marquez integrations to OpenLineage


willy@...
 

Hello Marquez Community!
 
As part of the evolution of Marquez, the lineage ingestion API specification was moved to a standalone project as OpenLineage under LFAI & Data foundation umbrella. (This is very akin to OpenTelemetry in the CNCF but for data). Marquez is now officially the reference implementation of the OpenLineage API and all the integrations adopt the OpenLineage standard.
 
The next logical step is to move the Marquez integrations to OpenLineage as they are part of the wider OpenLineage ecosystem. The Marquez API, UI and clients remain part of the Marquez project. The same functionality will be available to Marquez, just the main difference is that integration packages will get a new name (openlineage-* instead of marquez-*) but otherwise the experience stays the same.
 
Here’s a link to the OpenLineage issue outlining the timeline. Please feel free to contribute, provide feedback, and express any concerns you may have by commenting directly in this issue: https://github.com/OpenLineage/OpenLineage/issues/73