Skip to content

Commit 65fdf8d

Browse files
SaltyAomSaltyAom
committed
🎉 feat: sanitize
1 parent 4c661c1 commit 65fdf8d

16 files changed

+712
-57
lines changed

‎CHANGELOG.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,10 @@
1+
# 0.1.0 - 5 Feb 2025
2+
Feature:
3+
- replace `arrayItems.join('",\"')` in favour of inline `joinStringArray` to improve performance
4+
- add `sanitize` option for handling unsafe character
5+
- new behavior is `sanitize`, previously is equivalent to `manual`
6+
- support inline a literal value
7+
18
# 0.0.2 - 4 Feb 2025
29
Feature:
310
- support integer, bigint, date, datetime

‎README.md

Lines changed: 63 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,16 +5,16 @@ Accelerate JSON stringification by providing OpenAPI/TypeBox model.
55
By providing model ahead of time, the library will generate a function that will serialize the object into a JSON string.
66

77
```
8-
$ npx tsx benchmarks/medium.ts
8+
$ npx tsx benchmarks/medium-manual.ts
99
1010
clk: ~3.02 GHz
1111
cpu: Apple M1 Max
1212
runtime: node 22.6.0 (arm64-darwin)
1313
1414
summary
1515
JSON Accelerator
16-
1.8x faster than JSON Stingify
17-
2.15x faster than Fast Json Stringify
16+
2.12x faster than JSON Stingify
17+
2.66x faster than Fast Json Stringify
1818
```
1919

2020
## Installation
@@ -52,7 +52,7 @@ console.log(encode(value)) // {"id":0,"name":"saltyaom"}
5252

5353
## Caveat
5454

55-
This library **WILL NOT** check for the validity of the schema, it is expected that the schema is **always** correct.
55+
This library **WILL NOT** check for the type validity of the schema, it is expected that the schema is **always** correct.
5656

5757
This can be achieved by checking the input validity with TypeBox before passing it to the accelerator.
5858

@@ -78,3 +78,62 @@ if (guard.Check(value)) encode(value)
7878
```
7979

8080
If the shape is incorrect, the output will try to corece the value into an expected model but if failed the error will be thrown.
81+
82+
## Options
83+
84+
This section is used to configure the behavior of the encoder.
85+
86+
`Options` can be passed as a second argument to `createAccelerator`.
87+
88+
```ts
89+
createAccelerator(shape, {
90+
unsafe: 'throw'
91+
})
92+
```
93+
94+
## Unsafe
95+
96+
If unsafe character is found, how should the encoder handle it?
97+
98+
This value only applied to string field.
99+
100+
- `'auto'`: Sanitize the string and continue encoding
101+
- `'manual'`: Ignore the unsafe character, this implied that end user should specify fields that should be sanitized manually
102+
- `'throw'`: Throw an error
103+
104+
The default behavior is `auto`.
105+
106+
### format sanitize
107+
108+
Since this library is designed for a controlled environment (eg. Restful API), most fields are controlled by end user which doesn't include unsafe characters for JSON encoding.
109+
110+
We can improve performance by specifying a `sanitize: 'manual'` and provide a field that should be sanitized manually.
111+
112+
We can add `sanitize: true` to a schema which is an uncontrolled field submit by user that might contains unsafe characters.
113+
114+
When a field is marked as `sanitize`, the encoder will sanitize the string and continue encoding regardless of the `unsafe` configuration.
115+
116+
```ts
117+
import { Type as t } from '@sinclair/typebox'
118+
import { createAccelerator } from 'json-accelerator'
119+
120+
const shape = t.Object({
121+
name: t.String(),
122+
id: t.Number(),
123+
unknown: t.String({ sanitize: true })
124+
})
125+
126+
const value = {
127+
id: 0,
128+
name: 'saltyaom',
129+
unknown: `hello\nworld`
130+
} satisfies typeof shape.static
131+
132+
const encode = createAccelerator(shape, {
133+
sanitize: 'manual'
134+
})
135+
136+
console.log(encode(value)) // {"id":0,"name":"saltyaom","unknown":"hello\\nworld"}
137+
```
138+
139+
This allows us to speed up a hardcode

‎benchmarks/large-manual.ts

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
/**
2+
* *-manual.ts is where end user specifiy which fields should be sanitized manually
3+
**/
4+
5+
import { t } from 'elysia'
6+
import { benchmark } from './utils'
7+
8+
benchmark(
9+
t.Array(
10+
t.Object({
11+
id: t.Number(),
12+
name: t.String(),
13+
bio: t.String({
14+
sanitize: true
15+
}),
16+
user: t.Object({
17+
name: t.String(),
18+
password: t.String(),
19+
email: t.Optional(t.String({ format: 'email' })),
20+
age: t.Optional(t.Number()),
21+
avatar: t.Optional(t.String({ format: 'uri' })),
22+
cover: t.Optional(t.String({ format: 'uri' }))
23+
}),
24+
playing: t.Optional(t.String()),
25+
wishlist: t.Optional(t.Array(t.Number())),
26+
games: t.Array(
27+
t.Object({
28+
id: t.Number(),
29+
name: t.String(),
30+
hoursPlay: t.Optional(t.Number({ default: 0 })),
31+
tags: t.Array(
32+
t.Object({
33+
name: t.String(),
34+
count: t.Number()
35+
})
36+
)
37+
})
38+
),
39+
metadata: t.Intersect([
40+
t.Object({
41+
alias: t.String()
42+
}),
43+
t.Object({
44+
country: t.Nullable(t.String()),
45+
region: t.Optional(t.String())
46+
})
47+
]),
48+
social: t.Optional(
49+
t.Object({
50+
facebook: t.Optional(t.String()),
51+
twitter: t.Optional(t.String()),
52+
youtube: t.Optional(t.String())
53+
})
54+
)
55+
})
56+
),
57+
[
58+
{
59+
id: 1,
60+
name: 'SaltyAom',
61+
bio: 'I like train\n',
62+
user: {
63+
name: 'SaltyAom',
64+
password: '123456',
65+
avatar: 'https://avatars.githubusercontent.com/u/35027979?v=4',
66+
cover: 'https://saltyaom.com/cosplay/pekomama.webp'
67+
},
68+
playing: 'Strinova',
69+
wishlist: [4_154_456, 2_345_345],
70+
games: [
71+
{
72+
id: 4_154_456,
73+
name: 'MiSide',
74+
hoursPlay: 17,
75+
tags: [
76+
{ name: 'Psychological Horror', count: 236_432 },
77+
{ name: 'Cute', count: 495_439 },
78+
{ name: 'Dating Sim', count: 395_532 }
79+
]
80+
},
81+
{
82+
id: 4_356_345,
83+
name: 'Strinova',
84+
hoursPlay: 365,
85+
tags: [
86+
{ name: 'Free to Play', count: 205_593 },
87+
{ name: 'Anime', count: 504_304 },
88+
{ name: 'Third-Person Shooter', count: 395_532 }
89+
]
90+
},
91+
{
92+
id: 2_345_345,
93+
name: "Tom Clancy's Rainbow Six Siege",
94+
hoursPlay: 287,
95+
tags: [
96+
{ name: 'FPS', count: 855_324 },
97+
{ name: 'Multiplayer', count: 456_567 },
98+
{ name: 'Tactical', count: 544_467 }
99+
]
100+
}
101+
],
102+
metadata: {
103+
alias: 'SaltyAom',
104+
country: 'Thailand',
105+
region: 'Asia'
106+
},
107+
social: {
108+
twitter: 'SaltyAom'
109+
}
110+
},
111+
{
112+
id: 2,
113+
name: 'VLost',
114+
bio: 'ไม่พี่คืองี้\n',
115+
user: {
116+
name: 'nattapon_kub',
117+
password: '123456'
118+
},
119+
games: [
120+
{
121+
id: 4_154_456,
122+
name: 'MiSide',
123+
hoursPlay: 17,
124+
tags: [
125+
{ name: 'Psychological Horror', count: 236_432 },
126+
{ name: 'Cute', count: 495_439 },
127+
{ name: 'Dating Sim', count: 395_532 }
128+
]
129+
},
130+
{
131+
id: 4_356_345,
132+
name: 'Strinova',
133+
hoursPlay: 365,
134+
tags: [
135+
{ name: 'Free to Play', count: 205_593 },
136+
{ name: 'Anime', count: 504_304 },
137+
{ name: 'Third-Person Shooter', count: 395_532 }
138+
]
139+
}
140+
],
141+
metadata: {
142+
alias: 'vlost',
143+
country: 'Thailand'
144+
}
145+
},
146+
{
147+
id: 2,
148+
name: 'eika',
149+
bio: 'こんにちわ！',
150+
user: {
151+
name: 'ei_ka',
152+
password: '123456'
153+
},
154+
games: [
155+
{
156+
id: 4_356_345,
157+
name: 'Strinova',
158+
hoursPlay: 365,
159+
tags: [
160+
{ name: 'Free to Play', count: 205_593 },
161+
{ name: 'Anime', count: 504_304 },
162+
{ name: 'Third-Person Shooter', count: 395_532 }
163+
]
164+
}
165+
],
166+
metadata: {
167+
alias: 'eika',
168+
country: 'Japan'
169+
}
170+
}
171+
],
172+
{
173+
sanitize: 'manual'
174+
}
175+
)

‎benchmarks/medium-manual.ts

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
/**
2+
* *-manual.ts is where end user specifiy which fields should be sanitized manually
3+
**/
4+
5+
import { t } from 'elysia'
6+
import { benchmark } from './utils'
7+
8+
benchmark(
9+
t.Object({
10+
id: t.Number(),
11+
name: t.Literal('SaltyAom'),
12+
bio: t.String({
13+
sanitize: true
14+
}),
15+
user: t.Object({
16+
name: t.String(),
17+
password: t.String()
18+
}),
19+
playing: t.Optional(t.String()),
20+
games: t.Array(
21+
t.Object({
22+
name: t.String(),
23+
hoursPlay: t.Number({ default: 0 }),
24+
tags: t.Array(t.String())
25+
})
26+
),
27+
metadata: t.Intersect([
28+
t.Object({
29+
alias: t.String()
30+
}),
31+
t.Object({
32+
country: t.Nullable(t.String())
33+
})
34+
]),
35+
social: t.Optional(
36+
t.Object({
37+
facebook: t.Optional(t.String()),
38+
twitter: t.Optional(t.String()),
39+
youtube: t.Optional(t.String())
40+
})
41+
)
42+
}),
43+
{
44+
id: 1,
45+
name: 'SaltyAom',
46+
bio: 'I like train\n',
47+
user: {
48+
name: 'SaltyAom',
49+
password: '123456'
50+
},
51+
games: [
52+
{
53+
name: 'MiSide',
54+
hoursPlay: 17,
55+
tags: ['Psychological Horror', 'Cute', 'Dating Sim']
56+
},
57+
{
58+
name: 'Strinova',
59+
hoursPlay: 365,
60+
tags: ['Free to Play', 'Anime', 'Third-Person Shooter']
61+
},
62+
{
63+
name: "Tom Clancy's Rainbow Six Siege",
64+
hoursPlay: 287,
65+
tags: ['FPS', 'Multiplayer', 'Tactical']
66+
}
67+
],
68+
metadata: {
69+
alias: 'SaltyAom',
70+
country: 'Thailand'
71+
},
72+
social: {
73+
twitter: 'SaltyAom'
74+
}
75+
},
76+
{
77+
sanitize: 'manual'
78+
}
79+
)

‎benchmarks/medium.ts

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,9 @@ benchmark(
55
t.Object({
66
id: t.Number(),
77
name: t.Literal('SaltyAom'),
8-
bio: t.String(),
8+
bio: t.String({
9+
sanitize: true
10+
}),
911
user: t.Object({
1012
name: t.String(),
1113
password: t.String()
@@ -37,7 +39,7 @@ benchmark(
3739
{
3840
id: 1,
3941
name: 'SaltyAom',
40-
bio: 'I like train',
42+
bio: 'I like train\nhere',
4143
user: {
4244
name: 'SaltyAom',
4345
password: '123456'

‎benchmarks/quote.ts

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
import { t } from 'elysia'
2+
import { benchmark } from './utils'
3+
4+
benchmark(
5+
t.Object({
6+
id: t.String({
7+
format: 'input'
8+
}),
9+
name: t.String()
10+
}),
11+
{
12+
id: '\n',
13+
name: 'SaltyAom'
14+
}
15+
)

0 commit comments

Comments
 (0)