Add topic properties to Migrator #3207
Conversation
| c.Key == "replication.factor" || | ||
| c.Key == "write.caching" || | ||
| c.Key == "redpanda.iceberg.mode" || | ||
| strings.HasPrefix(c.Key, "redpanda.remote.") { |
There was a problem hiding this comment.
Is this correct? Do we have to first query the destination server settings to ensure that we're not overwriting a custom setting?
| replication_factor_override: true | ||
| replication_factor: -1 |
There was a problem hiding this comment.
The template linter is a bit annoying, since it doesn't check if there are any unrecognised fields... Something to improve in Benthos.
|
|
||
| continue | ||
| } else { | ||
| mgr.Logger().Infof("Created topic %q", topic) |
There was a problem hiding this comment.
There's a debug message which gets printed if the topic already exists...
mihaitodor
force-pushed
the
mihaitodor-migrator-topic-properties
branch
2 times, most recently
from
b28c503 to
e3efdbd
Compare
| c.Key == "replication.factor" || | ||
| c.Key == "write.caching" || | ||
| c.Key == "redpanda.iceberg.mode" || | ||
| strings.HasPrefix(c.Key, "redpanda.remote.") { |
There was a problem hiding this comment.
I'm not sure we want to mirror redpanda.remote.* but just want to take defaults in the destination cluster. The thing is in Redpanda Cloud we never want this to be off, and the default is always on. Maybe we should just not mirror it...
There was a problem hiding this comment.
Sounds good, I removed redpanda.remote. from that list.
Also make sure ACLs are always migrated even if the destination topic already exists. Signed-off-by: Mihai Todor <[email protected]>
Signed-off-by: Mihai Todor <[email protected]>
mihaitodor
force-pushed
the
mihaitodor-migrator-topic-properties
branch
from
e3efdbd to
840ed26
Compare
| c.Key == "max.message.bytes" || | ||
| c.Key == "replication.factor" || | ||
| c.Key == "write.caching" || | ||
| c.Key == "redpanda.iceberg.mode" { |
There was a problem hiding this comment.
Maybe simpler as a switch or a slices.Contain?
There was a problem hiding this comment.
I went for a map since that's what Copilot suggested when I asked it to make the code look nicer 😅
Signed-off-by: Mihai Todor <[email protected]>
Signed-off-by: Mihai Todor <[email protected]>
Also make sure ACLs are always migrated even if the destination topic already exists.