Target replacement support for VCRs with multiple sub_volumes #1551
Conversation
Updated the volume layer code to support replacement of a target in a VCR that has multiple sub_volumes in it. Before this we only supported a single sub_volume. Most of the changes here are for tests. I've added a loop around vcr replacement tests to create multiple sub volumes and iterate through the test for different VCR configurations. In addition, it is now required that the generation number change for all sub-volumes when any target is changed.
| for (o_sv, new_sv) in | ||
| o_sub_volumes.iter().zip(n_sub_volumes.iter()) | ||
| { | ||
| let sv_res = Self::compare_vcr_for_replacement( |
There was a problem hiding this comment.
Any worries about unbounded recursion blowing up the stack here?
There was a problem hiding this comment.
Yes. But, I'm not sure what to do about it.
We currently don't have any limits in Nexus, so you could craft VCRs forever until something broke. I don't know what the limit is currently and what will break first.
Cleaned up some comments.
Created a function build_replacement_subvol that will create Vec<VolumeConstructionRequest> of Regions based on input provided. This should reduce a bunch of common code. Converted a bunch of other replacement subvolume creation to be done immutably.
There was a problem hiding this comment.
LGTM
The only follow-up suggestion (optional) is to consolidate "test on every ReadOnlyParentMode", e.g.
impl ReadOnlyParentMode
fn test_all<F: FnMut(Self)>(mut f: F) {
f(ReadOnlyParentMode::OnlyOriginal);
f(ReadOnlyParentMode::OnlyReplacement);
f(ReadOnlyParentMode::Neither);
f(ReadOnlyParentMode::Both);
}
}...which you could then use like
for sv in 1..4 {
for sv_changed in 0..sv {
for cid in 0..3 {
ReadOnlyParentMode::test_all(|mode|
test_volume_replace_inner(
sv,
sv_changed,
cid,
mode,
)
);
}
}
}
upstairs/src/volume.rs
Outdated
| block_size, | ||
| blocks_per_extent, | ||
| extent_count, | ||
| opts: opts.clone(), |
There was a problem hiding this comment.
The targets should be randomized here so that all sub volumes are not the same, using something like https://docs.rs/ipgen/1.0.2/ipgen/fn.ip.html
There was a problem hiding this comment.
I'm holding it wrong, somehow...
I get back:
Err(InvalidIpNetwork("[BUG] failed to parse the generated IP address a00:67 as IPv4"))
And, the source says:
// This error should never happen but I'm not a fan of panicking in libraries
.map_err(|_| Error::InvalidIpNetwork(format!("[BUG] failed to parse the generated IP address `{}` as IPv4", ipv6_addr.trim_start_matches(':'))))
There was a problem hiding this comment.
Ah, it is a bug in the latest release 1.0.2.
Fixed in main, but they have not rolled a new release out that contains the fix.
There was a problem hiding this comment.
I found another library that could generate random I.P. addresses for me and used that.
upstairs/src/volume.rs
Outdated
| block_size, | ||
| blocks_per_extent, | ||
| extent_count, | ||
| opts: replacement_opts.clone(), |
There was a problem hiding this comment.
This too will have multiple subvolume targets that are the same
| // A valid replacement VCR is provided with a larger generation | ||
| // number and only one target being different. | ||
| let vol_id = Uuid::new_v4(); | ||
| let opts = generic_crucible_opts(vol_id); |
There was a problem hiding this comment.
maybe generic_crucible_opts should also randomize the addresses?
Make each upstairs test use a randomly generated IP address for the targets in a downstairs.
|
This is ready to go back, but we are holding it for the moment because this may require some changes on the Omicron side. Also, this work should come behind oxidecomputer/omicron#6172 |
Updated the volume layer code to support replacement of a target in a VCR that has multiple sub_volumes in it.
Before this we only supported a single sub_volume.
Most of the changes here are for tests. I've added a loop around the VCR replacement tests to create multiple sub volumes and iterate through the test for different VCR configurations.
In addition, it is now required that the generation number change for all sub-volumes when any target is changed.
Some tests that were duplicates have been removed.