Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor/ordering #123

Merged
merged 5 commits into from
Feb 17, 2024
Merged

Conversation

itamar-marom
Copy link
Collaborator

@itamar-marom itamar-marom commented Dec 26, 2023

Description

Code refactoring to the ordering logic. Added also tests.


This PR has:

  • been tested on a real K8S cluster to ensure creation of a brand new Druid cluster works.
  • been tested for backward compatibility on a real K*S cluster by applying the changes introduced here on an existing Druid cluster. If there are any backward incompatible changes then they have been noted in the PR description.
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added documentation for new or modified features or behaviors.

Key changed/added files in this PR
  • controllers/druid/handler.go
  • controllers/druid/ordering.go
  • controllers/druid/ordering_test.go

@itamar-marom
Copy link
Collaborator Author

@cyril-corbon @AdheipSingh ping 😄

@AdheipSingh
Copy link
Contributor

@itamar-marom ill check this week. Thanks

@AdheipSingh
Copy link
Contributor

ill take this in the next release. Will test is locally once.
On high level LGTM !

Comment on lines +16 to +35
func getNodeSpecsByOrder(m *v1alpha1.Druid) []*ServiceGroup {

scaledServiceSpecsByNodeType := map[string][]*ServiceGroup{}
for _, t := range druidServicesOrder {
scaledServiceSpecsByNodeType[t] = []*ServiceGroup{}
}

for key, nodeSpec := range m.Spec.Nodes {
scaledServiceSpec := scaledServiceSpecsByNodeType[nodeSpec.NodeType]
scaledServiceSpecsByNodeType[nodeSpec.NodeType] = append(scaledServiceSpec, &ServiceGroup{key: key, spec: nodeSpec})
}

allScaledServiceSpecs := make([]*ServiceGroup, 0, len(m.Spec.Nodes))

for _, t := range druidServicesOrder {
allScaledServiceSpecs = append(allScaledServiceSpecs, scaledServiceSpecsByNodeType[t]...)
}

return allScaledServiceSpecs
}
Copy link
Contributor

@AdheipSingh AdheipSingh Feb 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a very good refactor. We can now abstract the order in the CR but it will not be not backward compatible.

@AdheipSingh
Copy link
Contributor

@itamar-marom if you can resolve this, we can merge it.

@itamar-marom
Copy link
Collaborator Author

I'll resolve this today/tomorrow and ping you

@itamar-marom
Copy link
Collaborator Author

@AdheipSingh ready

@AdheipSingh AdheipSingh merged commit 8b1f785 into datainfrahq:master Feb 17, 2024
1 check passed
@itamar-marom itamar-marom deleted the refactor/ordering branch February 17, 2024 17:25
TessaIO pushed a commit to TessaIO/druid-operator that referenced this pull request Jul 24, 2024
* (ordering): refactor code

* (ordering): refactor code

* (ordering): testing

* chore(branch): rebase branch with master

* fix(tests): validate nodes order by regex
AdheipSingh added a commit that referenced this pull request Jul 28, 2024
* Ingestion Controller (#53)

* ingestion spec acc to v3

* task creation

* supprt native batch

* fix router url

* revert license change

* revert go mod change

* fix main

* fix: made some changes as per review comments

* fix: removed unused package from ingestion reconciler

* rebase 1

* add example

* add review

* update dockerfile

---------

Co-authored-by: avtarOPS <[email protected]>

* Update Docs and Tutorials (#138)

* docs and tutorials

* Refactor/ordering (#123)

* (ordering): refactor code

* (ordering): refactor code

* (ordering): testing

* chore(branch): rebase branch with master

* fix(tests): validate nodes order by regex

* Bump controller-tools version (#140)

* Utilize the DruidIngestion controller in e2e tests (#146)

* adds needed volumes to eks deployment spec and improves getting started documentation by noting minio dependency (#149)

* Add support for annotations on Deployment/StatefulSet resources in DruidNodeSpec (#145)

* Add support for annotations on Deployment/StatefulSet resources

* Support setting ReplicationControllerAnnotations at the cluster-level

* rename replicationControllerAnnotations to workloadAnnotations

* suggestions from code review

* Add support for multi tier nodes with different PVC sizes (#106) (#152)

Co-authored-by: Farhad Farahi <[email protected]>

* fix: put Druid crds in the appropriate folder specified by Helm (#162)

Signed-off-by: ahmed.g <[email protected]>
Signed-off-by: TessaIO <[email protected]>

* Adds service account name to each druid node optionally (#164)

* Adds service account to each druid node optionally

* Use controller-gen v0.11.2

---------

Signed-off-by: ahmed.g <[email protected]>
Signed-off-by: TessaIO <[email protected]>
Co-authored-by: AdheipSingh <[email protected]>
Co-authored-by: avtarOPS <[email protected]>
Co-authored-by: Itamar Marom <[email protected]>
Co-authored-by: Jesper Larsson <[email protected]>
Co-authored-by: Sam Wheating <[email protected]>
Co-authored-by: Evan Jones <[email protected]>
Co-authored-by: Farhad Farahi <[email protected]>
Co-authored-by: Farhad Farahi <[email protected]>
Co-authored-by: Sadananda Aithal <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants