Inconsistent FT.INFO results

[Czechh]

Description

When running ft.info command the results aren't parsed correctly. If you look into "numDocs":["bytes_collected",0] in the parsed response, something went wrong

Example of command result:

1) index_name
 2) "org-63daa72dfa5bebc3b6ffdccd-app-63f04a80d9aeb523717b0acd-index-644acc8f8a60a87f6e00f4b4"
 3) index_definition
 4) 1) key_type
    2) HASH
    3) prefixes
    4) 1) index:644acc8f8a60a87f6e00f4b4
    5) default_score
    6) "1"
 5) attributes
 6) 1) 1) identifier
       2) v
       3) attribute
       4) v
       5) type
       6) VECTOR
    2) 1) identifier
       2) id
       3) attribute
       4) id
       5) type
       6) TEXT
       7) WEIGHT
       8) "1"
    3) 1) identifier
       2) org
       3) attribute
       4) org
       5) type
       6) TEXT
       7) WEIGHT
       8) "1"
    4) 1) identifier
       2) createdAt
       3) attribute
       4) createdAt
       5) type
       6) NUMERIC
 7) index_options
 8) (empty array)
 9) gc_stats
10) 1) bytes_collected
    2) (integer) 0
11) cursor_stats
12) 1) global_idle
    2) (integer) 0
    3) global_total
    4) (integer) 0
    5) index_capacity
    6) (integer) 128
    7) index_total
    8) (integer) 0
13) dialect_stats
14) 1) dialect_1
    2) (integer) 0
    3) dialect_2
    4) (integer) 1
    5) dialect_3
    6) (integer) 0
15) num_docs
16) (integer) 31
17) max_doc_id
18) (integer) 31
19) num_terms
20) (integer) 37
21) num_records
22) (integer) 98
23) inverted_sz_mb
24) "0.00035858154296875"
25) total_inverted_index_blocks
26) (integer) 629596
27) vector_index_sz_mb
28) "6.2732696533203125"
29) offset_vectors_sz_mb
30) "5.245208740234375e-05"
31) doc_table_size_mb
32) "0.0036363601684570312"
33) sortable_values_size_mb
34) "0"
35) key_table_size_mb
36) "0.0013275146484375"
37) records_per_doc_avg
38) "3.1612904071807861"
39) bytes_per_record_avg
40) "3.8367347717285156"
41) offsets_per_term_avg
42) "0.56122446060180664"
43) offset_bits_per_record_avg
44) "8"
45) indexing
46) (integer) 0
47) percent_indexed
48) "1"
49) hash_indexing_failures
50) (integer) 1
51) number_of_uses
52) (integer) 32

Example from node client:

{"indexName":"org-63daa72dfa5bebc3b6ffdccd-app-63f04a80d9aeb523717b0acd-index-644acc8f8a60a87f6e00f4b4","indexOptions":["key_type","HASH","prefixes",["index:644acc8f8a60a87f6e00f4b4"],"default_score","1"],"indexDefinition":{"identifier,v,attribute,v,type,VECTOR":["identifier","id","attribute","id","type","TEXT","WEIGHT","1"],"identifier,org,attribute,org,type,TEXT,WEIGHT,1":["identifier","createdAt","attribute","createdAt","type","NUMERIC"]},"attributes":[],"numDocs":["bytes_collected",0],"maxDocId":["global_idle",0,"global_total",0,"index_capacity",128,"index_total",0],"numTerms":["dialect_1",0,"dialect_2",1,"dialect_3",0],"numRecords":31,"invertedSzMb":31,"vectorIndexSzMb":37,"totalInvertedIndexBlocks":98,"offsetVectorsSzMb":"0.00035858154296875","docTableSizeMb":629596,"sortableValuesSizeMb":"6.2732696533203125","keyTableSizeMb":"5.245208740234375e-05","recordsPerDocAvg":"0.0036363601684570312","bytesPerRecordAvg":"0","offsetsPerTermAvg":"0.0013275146484375","offsetBitsPerRecordAvg":"3.1612904071807861","hashIndexingFailures":"3.8367347717285156","indexing":"0.56122446060180664","percentIndexed":"8","gcStats":{},"cursorStats":{},"stopWords":1}

Comparing this response to a another it seems like the order of the keys is different. Not sure if this would count as a bug for the library or the server. -- notice index_options in another response (a local server) is in different place

1) index_name
 2) org-63daa72dfa5bebc3b6ffdccd-app-63f04a80d9aeb523717b0acd-index-644acc8f8a60a87f6e00f4b4
 3) index_options
 4) (empty array)
 5) index_definition
 6) 1) key_type
    2) HASH
    3) prefixes
    4) 1) index:644acc8f8a60a87f6e00f4b4
    5) default_score
    6) "1"
...

Node.js Version

18

Redis Server Version

6.2.10

Node Redis Version

4.6.4

Platform

Linux

Logs

No response

[leibale]

Seems like relying on the order wasn't a good idea.. (see https://github.com/redis/node-redis/blob/master/packages/search/lib/commands/INFO.ts)

in the meantime you can:

const reply = await client.sendCommand(['FT.INFO', 'idx']);
// will be the same as in `redis-cli`, an array of key-value pairs

[Czechh]

Thanks for the reply. I used the workaround to get the key I need.

And this might be too ugly?

function transformReply(rawReply: InfoRawReply): InfoReply {
    const reply: Partial<InfoReply> = {};

    for (let i = 0; i < rawReply.length; i += 2) {
        const key = rawReply[i];
        const value = rawReply[i + 1];

        switch (key) {
            case 'index_name':
                reply.indexName = value;
                break;
            case 'index_options':
                reply.indexOptions = value;
                break;
            case 'index_definition':
                reply.indexDefinition = transformTuplesReply(value);
                break;
            case 'attributes':
                reply.attributes = value.map(attribute => transformTuplesReply(attribute));
                break;
            case 'num_docs':
                reply.numDocs = value;
                break;
            case 'max_doc_id':
                reply.maxDocId = value;
                break;
            case 'num_terms':
                reply.numTerms = value;
                break;
            case 'num_records':
                reply.numRecords = value;
                break;
            case 'inverted_sz_mb':
                reply.invertedSzMb = value;
                break;
            case 'vector_index_sz_mb':
                reply.vectorIndexSzMb = value;
                break;
            case 'total_inverted_index_blocks':
                reply.totalInvertedIndexBlocks = value;
                break;
            case 'offset_vectors_sz_mb':
                reply.offsetVectorsSzMb = value;
                break;
            case 'doc_table_size_mb':
                reply.docTableSizeMb = value;
                break;
            case 'sortable_values_size_mb':
                reply.sortableValuesSizeMb = value;
                break;
            case 'key_table_size_mb':
                reply.keyTableSizeMb = value;
                break;
            case 'records_per_doc_avg':
                reply.recordsPerDocAvg = value;
                break;
            case 'bytes_per_record_avg':
                reply.bytesPerRecordAvg = value;
                break;
            case 'offsets_per_term_avg':
                reply.offsetsPerTermAvg = value;
                break;
            case 'offset_bits_per_record_avg':
                reply.offsetBitsPerRecordAvg = value;
                break;
            case 'hash_indexing_failures':
                reply.hashIndexingFailures = value;
                break;
            case 'indexing':
                reply.indexing = value;
                break;
            case 'percent_indexed':
                reply.percentIndexed = value;
                break;
            case 'gc_stats':
                reply.gcStats = {
                    bytesCollected: value[1],
                    totalMsRun: value[3],
                    totalCycles: value[5],
                    averageCycleTimeMs: value[7],
                    lastRunTimeMs: value[9],
                    gcNumericTreesMissed: value[11],
                    gcBlocksDenied: value[13],
                };
                break;
            case 'cursor_stats':
                reply.cursorStats = {
                    globalIdle: value[1],
                    globalTotal: value[3],
                    indexCapacity: value[5],
                    indexTotal: value[7],
                };
                break;
            case 'stopwords_list':
                reply.stopWords = value;
                break;
        }
    }

    return reply as InfoReply;
}

[leibale]

@Czechh we can do something like

const obj = {};
for (let i = 0; i < rawReply.length; i += 2) {
  obj[rawReply[i]] = rawReply[i + 1];
}

return {
  indexName: obj.index_name,
  // ...
};

which is a little bit "cleaner"/more readable, but will perform worse (I didn't actually benchmark it, but creating 2 objects can't be faster than creating just one...)

[mkadirtan]

Thanks for the reply. I used the workaround to get the key I need.

And this might be too ugly?

function transformReply(rawReply: InfoRawReply): InfoReply {
    const reply: Partial<InfoReply> = {};

    for (let i = 0; i < rawReply.length; i += 2) {
        const key = rawReply[i];
        const value = rawReply[i + 1];

        switch (key) {
            case 'index_name':
                reply.indexName = value;
                break;
            case 'index_options':
                reply.indexOptions = value;
                break;
            case 'index_definition':
                reply.indexDefinition = transformTuplesReply(value);
                break;
            case 'attributes':
                reply.attributes = value.map(attribute => transformTuplesReply(attribute));
                break;
            case 'num_docs':
                reply.numDocs = value;
                break;
            case 'max_doc_id':
                reply.maxDocId = value;
                break;
            case 'num_terms':
                reply.numTerms = value;
                break;
            case 'num_records':
                reply.numRecords = value;
                break;
            case 'inverted_sz_mb':
                reply.invertedSzMb = value;
                break;
            case 'vector_index_sz_mb':
                reply.vectorIndexSzMb = value;
                break;
            case 'total_inverted_index_blocks':
                reply.totalInvertedIndexBlocks = value;
                break;
            case 'offset_vectors_sz_mb':
                reply.offsetVectorsSzMb = value;
                break;
            case 'doc_table_size_mb':
                reply.docTableSizeMb = value;
                break;
            case 'sortable_values_size_mb':
                reply.sortableValuesSizeMb = value;
                break;
            case 'key_table_size_mb':
                reply.keyTableSizeMb = value;
                break;
            case 'records_per_doc_avg':
                reply.recordsPerDocAvg = value;
                break;
            case 'bytes_per_record_avg':
                reply.bytesPerRecordAvg = value;
                break;
            case 'offsets_per_term_avg':
                reply.offsetsPerTermAvg = value;
                break;
            case 'offset_bits_per_record_avg':
                reply.offsetBitsPerRecordAvg = value;
                break;
            case 'hash_indexing_failures':
                reply.hashIndexingFailures = value;
                break;
            case 'indexing':
                reply.indexing = value;
                break;
            case 'percent_indexed':
                reply.percentIndexed = value;
                break;
            case 'gc_stats':
                reply.gcStats = {
                    bytesCollected: value[1],
                    totalMsRun: value[3],
                    totalCycles: value[5],
                    averageCycleTimeMs: value[7],
                    lastRunTimeMs: value[9],
                    gcNumericTreesMissed: value[11],
                    gcBlocksDenied: value[13],
                };
                break;
            case 'cursor_stats':
                reply.cursorStats = {
                    globalIdle: value[1],
                    globalTotal: value[3],
                    indexCapacity: value[5],
                    indexTotal: value[7],
                };
                break;
            case 'stopwords_list':
                reply.stopWords = value;
                break;
        }
    }

    return reply as InfoReply;
}

I think this is fine ( although ugly ), but needs inner switches for sub objects. You are still relying on the order for cursorStats or gcStats.