BE: Topics: Validate ISR/replication upon creation (#103)

[abhishekray323]

• Breaking change? (if so, please describe the impact and migration path for existing application instances)

What changes did you make? (Give an overview)
Resolves #103
Constraints related to configurations of kafka topic has been added. Before this change there were only default checks provided Kafka, which allowed configuration with inconsistent configs to get created. One such example is being able to set "min.insync.replicas" value greater than "replication factor", which doesn't makes sense. Following is a complete list of checks have been added:

• min.insync.replicas should be less than or equal to replication.factor
• compression level for a particular compression type should be set only when compression.type is set to corresponding type
• "min.cleanable.dirty.ratio", "min.compaction.lag.ms", "max.compaction.lag.ms" & "delete.retention.ms"should only be set if "cleanup.policy" is set to compact or compact,delete
• "local.retention.ms" & "local.retention.bytes" should only be set if "remote.storage.enable" is set to true.
• various "retention and deletion time" based configuration should follow this constraint: retention.ms >= local.retention.ms >= segment.ms
• various "retention and deletion memory" based configuration should follow this constraint: retention.bytes >= local.retention.bytes >= segment.bytes >= max.message.bytes

Is there anything you'd like reviewers to focus on?

• Please check does all the constraints helps user to avoid creating wrong configurations or not
• Please suggest more checks, if I have missed something.
• Please let me know if I need to add comments in code.

How Has This Been Tested? (put an "x" (case-sensitive!) next to an item)

• No need to
• Manually (please, describe, if necessary)
• Unit checks
• Integration checks
• Covered by existing automation

Checklist (put an "x" (case-sensitive!) next to all the items, otherwise the build will fail)

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation (e.g. ENVIRONMENT VARIABLES)
• My changes generate no new warnings (e.g. Sonar is happy)
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged

Check out Contributing and Code of Conduct

A picture of a cute animal (not mandatory but encouraged)

[github-actions]

Hi abhishekray323! 👋

Welcome, and thank you for opening your first PR in the repo!

Please wait for triaging by our maintainers.

Please take a look at our contributing guide.

[abhishekray323]

Hi ,
Should I fix the breaking workflows, then a somebody will review it? or should I wait for comments from reviewer and fix all the things together at once?

[Haarolean]

Hi , Should I fix the breaking workflows, then a somebody will review it? or should I wait for comments from reviewer and fix all the things together at once?

Please feel free to take a look at the failing tests & sonar checks, we'll take a look meanwhile

[abhishekray323]

Hi @Haarolean ,

Also, I'm trying to run Sonar analysis in my IntelliJ using SonarLint plugin, but it's running slow, do you have any suggestions on that how can I run sonar tests before raising a PR ?

[Haarolean]

Hi @Haarolean ,

Also, I'm trying to run Sonar analysis in my IntelliJ using SonarLint plugin, but it's running slow, do you have any suggestions on that how can I run sonar tests before raising a PR ?

@abhishekray323 not quite, unfortunately. Please fix the tests first and we'll run the cloud one. There might be no issues, just the failure related to a failing build.

[abhishekray323]

It seems like somehow this PR got closed. Raise a new PR: #1263

Hi @Haarolean ,
Also, I'm trying to run Sonar analysis in my IntelliJ using SonarLint plugin, but it's running slow, do you have any suggestions on that how can I run sonar tests before raising a PR ?

@abhishekray323 not quite, unfortunately. Please fix the tests first and we'll run the cloud one. There might be no issues, just the failure related to a failing build.

It was due to memory issue in my IDE, increased the heap memory, every thing worked fine. Thanks for responding .