Adding basic version of DUMP and RESTORE commands

[s3w3nofficial]

Currently only work's on string type and does not support compression, but it's still a solid starting point.

Implemented

• redis length encoding
• redis compatible crc64 implementation
• DUMP command that works on strings without compression
• RESTORE command that works on strings without compression and additional modifiers

Not Implemented

• support for other types than strings
• support for lzf compression
• optional modifiers for RESTORE

Also about the copyright for the crc64, i only ported the pycrc generated version, but still i'm not sure if and which copyright should i add.

[s3w3nofficial]

@microsoft-github-policy-service agree

[badrishc]

Also about the copyright for the crc64, i only ported the pycrc generated version, but still i'm not sure if and which copyright should i add.

since you only ported the pycrc generated version, it seems we should be okay as it is ...

[vazois]

@s3w3nofficial, any updates on the open comments?

[s3w3nofficial]

@s3w3nofficial, any updates on the open comments?

Only open comment left is from @badrishc about the renaming which i already implemented, i guess i can close that one ?

[vazois]

LGTM!

[s3w3nofficial]

Every test is passing except the RestoreACLsAsync. It throws this exception: "NOAUTH Authentication required.".

I can't figure out why. Can you guys please take a look at it ?

[badrishc]

Every test is passing except the RestoreACLsAsync. It throws this exception: "NOAUTH Authentication required.".

I can't figure out why. Can you guys please take a look at it ?

@kevin-montrose, can you help how to fix the ACL unit test here? The error is:

[kevin-montrose]

Every test is passing except the RestoreACLsAsync. It throws this exception: "NOAUTH Authentication required.".
I can't figure out why. Can you guys please take a look at it ?

@kevin-montrose, can you help how to fix the ACL unit test here? The error is:

@badrishc @s3w3nofficial I suspect it's that try/catch not leaving the client in a nice state when an exception is encountered. The pattern elsewhere for ACLs with commands that aren't repeatable is to use a counter to differentiate each invocation.

[Test]
public async Task RestoreACLsAsync()
{
    var count = 0;

    await CheckCommandsAsync(
        "RESTORE",
        [DoRestoreAsync]
    );

    async Task DoRestoreAsync(GarnetClient client)
    {
        var payload = new byte[]
        {
            0x00, // value type 
            0x03, // length of payload
            0x76, 0x61, 0x6C,       // 'v', 'a', 'l'
            0x0B, 0x00, // RDB version
            0xDB, 0x82, 0x3C, 0x30, 0x38, 0x78, 0x5A, 0x99 // Crc64
        };

        count++;

        var val = await client.ExecuteForStringResultAsync(
            "$7\r\nRESTORE\r\n"u8.ToArray(),
            [
                Encoding.UTF8.GetBytes($"foo-{count}"), 
                "0"u8.ToArray(), 
                payload
            ]);

        ClassicAssert.AreEqual("OK", val);
    }
}

Works, and all ACL tests pass.

[kevin-montrose]

Lots of little notes for improvements, and a couple questions around correctness.

[s3w3nofficial]

Hey @kevin-montrose

Thanks for the help with test and for the comments. I will try to resolve these over the weekend.

Regarding the parsing, i'm working on a parser that support's the whole rdb format and i hope i will be able to implement that one in the future. For now i will look into optimizing the RespLengthEncodingUtils since the parser won't be ready any time soon.

[badrishc]

This PR looks ok, although a few spots for improvement remain:

• There should ideally be no need to copy over the entire value retrieved from the back-end to a temporary buffer. Instead, we would directly write the RDB-formatted output (with CRC) to the output buffer in the back-end logic, similar to the CopyRespTo calls.
• There needs to be a BDN in RawStringOperations to measure and improve the performance of DUMP and RESTORE.

However, these require deeper surgery than necessary, so we can go ahead and approve/merge this PR as is, and revisit later as per needs. Thank you.