Graduation process from delta-incubator to delta-io

[dennyglee]

We propose additional guidance for incubation project graduation of:

• DCO
• More than one contributor for the project
• Ensure LF Trademark

Any other thoughts, feedback, or guidance here? cc @rtyler

[dennyglee]

Two projects to be considered for incubation include:

• delta-dotNet pending resolution of this thread
• delta-kernel-rs

cc @rtyler @tdas

[rtyler]

At a high level I think elevation into the "main" project should be done for projects with increased impact, visibility, and/or importance to the Delta Lake project as a whole. As I discussed with @dennyglee yesterday I think the bar should be relatively low but I will expand on some of the reasoning for others:

• At least 6 months in incubation, this gives the contributors time to sort out any license issues, access controls, and find a groove working together to produce releases
• Developer Certificate of Origin, we have already agreed to adopt this for the Delta Lake project with guidance from Linux Foundation AI & Data. We're a little behind on it, but the incubating project should either already be using it, or be comfortable adopting it in delta-io
• Multiple contributors, this is a good signal of a community with potential. Community is highly important since contributors can and should come and go as our interests shift over time. We should strive to keep delta-io projects vibrant and continually improving. I think this takes the ideal form of at least 2, but ideally more than 2 people with merge and release access for the incubating project.
• Trademark compliance, one of our big shared assets is the Delta Lake trademark held by the Linux Foundation. Before elevation into delta-io, I think incubating projects should be correctly using the trademark for development, documentation, and releases.

By these heuristics I believe that delta-dotnet (pending that thread @dennyglee mentions) and delta-kernel-rs are ready to "come up" to the delta-io organization so long as those projects want to and there is community support for it from the existing delta-io maintainers/contributors.

[dennyglee]

Thanks @rtyler - based on the feedback, we will work on updating the delta-io voting guidance to reflect per above. Thanks!

[felipepessoto]

Once a project is graduated would it move to https://github.com/delta-io/delta?

What are the benefits of moving multiple projects to delta-io/delta? I haven't seen much, but honestly, I'm only interested in delta-spark.
In my scenario the kernel and other projects only made things more complicated, with a complex build.sbt and slower IDE.

I'd like to hear from other devs what they think, any regrets about this migration?

Some context: #1824, https://github.com/delta-io/delta/pull/1837/files
This is lost: https://github.com/delta-io/connectors/issues/556

cc @tdas

[dennyglee]

Hey @felipepessoto - yes that is correct, once it is graduated it would be moved into delta-io per Incubation Project Graduation.

The benefits of moving projects from delta-incubator to delta-io is that there are many interesting Delta-based projects that are used by and developed by fellow users and contributors to the project. While I can empathize with slowing things down for kernel - the key thing we learned from creating the initial set of delta-spark, Delta Lake Trino integrations, delta-flink, and delta-rs was that in the end it would be faster for languages and frameworks to integrate if we abstracted away the logic of making sense of the metadata.