You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/blog/jaas.md
+15-17Lines changed: 15 additions & 17 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,16 +1,16 @@
1
1
# JSON on Redis via RediSQL, SQL steroids for Redis
2
2
3
-
####RediSQL, Redis on SQL steroids.
3
+
### RediSQL, Redis on SQL steroids.
4
4
5
5
RediSQL is a redis module that embeds SQLite to provide full SQL capabilities to redis.
6
6
7
-
The fastest introduction to RediSQL is [our homepage](/)
7
+
The fastest introduction to RediSQL is [our homepage](https://github.com/RedBeardLab/zeeSQL-doc/tree/70a762a22db224251819ed5ce9119aa9d87adf58/README.md)
8
8
9
-
tl;dr; We build a **JSON as a Service** in less than 500 lines of javascript + RediSQL, you can check out the [whole source file here.][jaas]
9
+
tl;dr; We build a **JSON as a Service** in less than 500 lines of javascript + RediSQL, you can check out the [whole source file here.](https://github.com/RedBeardLab/JaaS/blob/master/index.js)
10
10
11
11
## JSON as a Service in 500 lines of code + RediSQL
12
12
13
-
While building web services is common to have the need to store some un-structured or semi-structured data (aka **JSON**) somewhere.
13
+
While building web services is common to have the need to store some un-structured or semi-structured data \(aka **JSON**\) somewhere.
14
14
15
15
Unfortunately, it is not always so easy.
16
16
@@ -20,7 +20,7 @@ If you are already using some kind of NoSQL database you may be a little luckier
20
20
21
21
## A faster solution
22
22
23
-
A faster solution could be to use RediSQL exploiting the [JSON1][JSON1] extension.
23
+
A faster solution could be to use RediSQL exploiting the [JSON1](https://www.sqlite.org/json1.html) extension.
24
24
25
25
SQLite provides several interesting extensions and one of our favourites is JSON1 that allow an efficient and fast manipulation of JSON data, all inside a full SQL engine.
26
26
@@ -34,7 +34,7 @@ We will create JSON object that will have names and then each object will live i
34
34
35
35
Then we will like to have a simple RediSQL interface like this one:
36
36
37
-
```
37
+
```text
38
38
127.0.0.1:6379> REDISQL.EXEC_STATEMENT DB create_namespace noises
39
39
1) DONE
40
40
2) (integer) 1
@@ -48,12 +48,12 @@ Then we will like to have a simple RediSQL interface like this one:
48
48
1) 1) "blablabla"
49
49
127.0.0.1:6379> REDISQL.EXEC_STATEMENT DB extract noises animals $.dog
50
50
1) 1) "woof"
51
-
127.0.0.1:6379>
51
+
127.0.0.1:6379>
52
52
```
53
53
54
54
Of course, we would also like to navigate complex JSONs and to add fields and values at will.
55
55
56
-
```
56
+
```text
57
57
127.0.0.1:6379> REDISQL.EXEC_STATEMENT DB create_namespace foo
58
58
1) DONE
59
59
2) (integer) 1
@@ -77,15 +77,15 @@ Of course, we would also like to navigate complex JSONs and to add fields and va
77
77
78
78
In this specific example we are only showing JSON, but keep in mind that the JSON field can be stored alongside the regular SQL fields as an extra field to hold any kind of unstructured data.
79
79
80
-
Also please note how these APIs are quite pleasant to work with, they seem almost native to redis and thanks to redis-module we have the possibility to simply create more powerful commands.
80
+
Also please note how these APIs are quite pleasant to work with, they seem almost native to redis and thanks to redis-module we have the possibility to simply create more powerful commands.
81
81
82
82
## Implementation
83
83
84
84
Now that we know what we are trying to achieve let's proceed to the implementation that you will see is quite simple.
85
85
86
86
Usually is a good idea to start from the data structure, and in our simple, but powerful, example, we need only a single table:
87
87
88
-
```SQL
88
+
```sql
89
89
CREATETABLEIF NOT EXISTS namespace (
90
90
namespace TEXTPRIMARY KEY
91
91
);
@@ -103,7 +103,7 @@ We could have done everything with just `json_data` and without `namespace`, but
103
103
104
104
Now that we have our data structure we can proceed with the procedures that I have show you above:
105
105
106
-
```SQL
106
+
```sql
107
107
-- create_namespace
108
108
INSERT INTO namespace VALUES(?1);
109
109
@@ -134,7 +134,7 @@ UPDATE json_data
134
134
135
135
In order to actually create those table here is an example on RediSQL that is more difficult to read but simpler to just copy and paste into the redis-cli.
136
136
137
-
```
137
+
```text
138
138
127.0.0.1:6379> REDISQL.CREATE_DB DB
139
139
OK
140
140
127.0.0.1:6379> REDISQL.EXEC DB "CREATE TABLE IF NOT EXISTS namespace (namespace TEXT PRIMARY KEY);"
@@ -155,9 +155,9 @@ OK
155
155
OK
156
156
```
157
157
158
-
Of course, there are a lot more commands in JSON1 API to use and explore, so I will simply [leave you the reference][JSON1].
158
+
Of course, there are a lot more commands in JSON1 API to use and explore, so I will simply [leave you the reference](https://www.sqlite.org/json1.html).
159
159
160
-
I also prepare a simple node application which exposes this exact same interface via REST API, it is a single, ~500 LOC, file that you can find [here][jaas]
160
+
I also prepare a simple node application which exposes this exact same interface via REST API, it is a single, ~500 LOC, file that you can find [here](https://github.com/RedBeardLab/JaaS/blob/master/index.js)
161
161
162
162
Feel free to use the node application as a blueprint for your next project.
163
163
@@ -173,7 +173,5 @@ Of course, if you have any question on RediSQL either open a public issue or wri
Copy file name to clipboardExpand all lines: docs/how-to/check-if-the-index-is-used.md
+17-31Lines changed: 17 additions & 31 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,6 @@
1
1
# How to check if an index is used in zeeSQL and SQLite
2
2
3
-
This tutorial will cover both zeeSQL and SQLite.
4
-
As you might know, zeeSQL is actually based on SQLite, so everything we say about zeeSQL can also be applied to raw SQLite.
3
+
This tutorial will cover both zeeSQL and SQLite. As you might know, zeeSQL is actually based on SQLite, so everything we say about zeeSQL can also be applied to raw SQLite.
5
4
6
5
## What are indexes
7
6
@@ -11,25 +10,21 @@ They don't store the main data, but only a subset of them, usually a few columns
11
10
12
11
The main point of indexes is that the data is always store sorted for the columns they are indexing.
13
12
14
-
For instance, if we create an index for the `score` of some users, in the index, the scores will be stored sorted, from the smallest to the largest (or vice-versa).
13
+
For instance, if we create an index for the `score` of some users, in the index, the scores will be stored sorted, from the smallest to the largest \(or vice-versa\).
15
14
16
-
Storing the field sorted, allows a fast comparison for those fields.
17
-
For instance, if we want to select all the users with a score greater than 20, we can look up in the index where the users with a score greater than 20 starts, and return all the users after that.
15
+
Storing the field sorted, allows a fast comparison for those fields. For instance, if we want to select all the users with a score greater than 20, we can look up in the index where the users with a score greater than 20 starts, and return all the users after that.
18
16
19
-
However, an index on the score, won't help if we are looking for all the users that played more than 15 games.
20
-
For that query, you will need a different index.
17
+
However, an index on the score, won't help if we are looking for all the users that played more than 15 games. For that query, you will need a different index.
21
18
22
19
As your use cases become complex, and the table larger, eventually a lot of indexes will be accumulated.
23
20
24
21
Then, for complex queries, it becomes very difficult to know what index, if any, is going to be used.
25
22
26
23
The SQL engine has its own algorithms and heuristic to figure out, what it thinks is the fastest way to execute a query.
27
24
28
-
Please note that while indexes help in retrieving information, they need to be maintained, which implies more load when the data are updated or added.
29
-
There is a tradeoff between insertion speed, (no index, fastest insertion) and query speed (the more indexes they are, the more query will run optimally).
25
+
Please note that while indexes help in retrieving information, they need to be maintained, which implies more load when the data are updated or added. There is a tradeoff between insertion speed, \(no index, fastest insertion\) and query speed \(the more indexes they are, the more query will run optimally\).
30
26
31
-
When no index can be used to speed up a query, SQLite will fallback in a full table scan, which means that it will look at every row in the table.
32
-
For big tables, this implies a higher latency and slower results.
27
+
When no index can be used to speed up a query, SQLite will fallback in a full table scan, which means that it will look at every row in the table. For big tables, this implies a higher latency and slower results.
33
28
34
29
## Am I using some index?
35
30
@@ -39,7 +34,7 @@ SQLite, and so zeeSQL, comes with a handy command to check what the SQL engine i
39
34
40
35
An example will help clarify, suppose we have two tables, `foo` and `bar`.
41
36
42
-
```
37
+
```text
43
38
127.0.0.1:6379> ZEESQL.CREATE_DB DB
44
39
1) 1) "OK"
45
40
127.0.0.1:6379> ZEESQL.EXEC DB COMMAND "create table foo(a int, b int, c int);"
@@ -52,7 +47,7 @@ An example will help clarify, suppose we have two tables, `foo` and `bar`.
52
47
53
48
At this point, we don't have any indexes, so any query will go through a full table scan.
54
49
55
-
```
50
+
```text
56
51
127.0.0.1:6379> ZEESQL.EXEC DB COMMAND "EXPLAIN QUERY PLAN select a from foo where a = 3 and b = 4 and c = 5;" NO_HEADER
57
52
1) 1) "RESULT"
58
53
2) 1) (integer) 2
@@ -65,7 +60,7 @@ In this example, were are scanning the whole `foo` table.
65
60
66
61
Adding a simple index to the `foo` table will help SQLite in searching more efficiently the table.
67
62
68
-
```
63
+
```text
69
64
127.0.0.1:6379> ZEESQL.EXEC DB COMMAND "CREATE INDEX foo_a on foo(a);" NO_HEADER
70
65
1) 1) "DONE"
71
66
2) 1) (integer) 0
@@ -79,7 +74,7 @@ Adding a simple index to the `foo` table will help SQLite in searching more effi
79
74
80
75
This works also when you are working with multiple tables.
81
76
82
-
```
77
+
```text
83
78
127.0.0.1:6379> ZEESQL.EXEC DB COMMAND "EXPLAIN QUERY PLAN select a from foo, bar where a = 3 and b = 4 and c = 5 and c = z;" NO_HEADER
84
79
1) 1) "RESULT"
85
80
2) 1) (integer) 4
@@ -96,7 +91,7 @@ On `foo`, we have the index we created previously that simplifies the search, in
96
91
97
92
As soon as we add an index:
98
93
99
-
```
94
+
```text
100
95
127.0.0.1:6379> ZEESQL.EXEC DB COMMAND "CREATE INDEX bar_z on bar(z);" NO_HEADER
101
96
1) 1) "DONE"
102
97
2) 1) (integer) 0
@@ -116,17 +111,15 @@ Now searches on both tables are using an index.
116
111
117
112
One detail, on `bar` we are using a `COVERING INDEX` instead of a standard index.
118
113
119
-
A standard index makes it faster to look up the id of the rows that we are interested in, but then we still need to fetch those rows from the database.
120
-
A covering index means that you are not going to fetch extra rows because all the data you care about are already stored in the index itself.
121
-
Note that in the query above we don't ask for any columns from the `bar` table in the result set, but we only compare that one column of `foo` is equal to one of `bar`.
114
+
A standard index makes it faster to look up the id of the rows that we are interested in, but then we still need to fetch those rows from the database. A covering index means that you are not going to fetch extra rows because all the data you care about are already stored in the index itself. Note that in the query above we don't ask for any columns from the `bar` table in the result set, but we only compare that one column of `foo` is equal to one of `bar`.
122
115
123
116
## Composite indexes
124
117
125
118
Indexes can be against a single column or against multiple columns.
126
119
127
120
The more the index is specific, the faster will be your query.
128
121
129
-
```
122
+
```text
130
123
127.0.0.1:6379> ZEESQL.EXEC DB COMMAND "CREATE INDEX foo_b_c on foo(b, c);" NO_HEADER
131
124
1) 1) "DONE"
132
125
2) 1) (integer) 0
@@ -142,7 +135,7 @@ In this case, instead of using the index `foo_a` the SQLite engine prefers the o
142
135
143
136
It is important to be careful between `AND` and `OR` conditions.
144
137
145
-
```
138
+
```text
146
139
127.0.0.1:6379> ZEESQL.EXEC DB COMMAND "EXPLAIN QUERY PLAN select a from foo where a = 3 OR b = 4 and c = 5;" NO_HEADER 1) 1) "RESULT"
147
140
2) 1) (integer) 4
148
141
2) (integer) 0
@@ -162,7 +155,7 @@ The query seems the same, but instead of an `AND` we have an `OR`, in this case,
162
155
163
156
If they were all `OR`, then a full table scan is almost guarantee, unless you don't have a separated index for each column in the table.
164
157
165
-
```
158
+
```text
166
159
127.0.0.1:6379> ZEESQL.EXEC DB COMMAND "EXPLAIN QUERY PLAN select a from foo where a = 3 OR b = 4 OR c = 5;" NO_HEADER
167
160
1) 1) "RESULT"
168
161
2) 1) (integer) 2
@@ -201,18 +194,11 @@ Indexes are fundamental for every database system, however, they might be confus
201
194
202
195
zeeSQL, and SQLite, provide a very simple way to check what indexes are going to be used for each of your queries, and when in doubt, it is a good idea to check it.
203
196
204
-
Do not add too many indexes, they slow down insertion.
205
-
Moreover, in small datasets, a full table scan can be fast enough.
197
+
Do not add too many indexes, they slow down insertion. Moreover, in small datasets, a full table scan can be fast enough.
206
198
207
199
Always verify your assumption about indexes.
208
200
209
201
## About zeeSQL
210
202
211
-
zeeSQL is a Redis Module that provides SQL capabilities to Redis.
212
-
It allows the creation and management of several SQL databases, each one independent from the other.
213
-
Moreover, zeeSQL provides out-of-the-box [secondary indexes](../secondary-indexes.md) capabilities, allowing fast and easy search by value in Redis.
214
-
215
-
216
-
[exec]: ../references.md#zeesql-exec
217
-
[query]: ../references.md#zeesql-query
203
+
zeeSQL is a Redis Module that provides SQL capabilities to Redis. It allows the creation and management of several SQL databases, each one independent from the other. Moreover, zeeSQL provides out-of-the-box [secondary indexes](../secondary-indexes.md) capabilities, allowing fast and easy search by value in Redis.
0 commit comments