Feat add json plugin and support low, medium, high reasoning efforts for thinking models #163

codelion · 2025-02-05T13:58:57Z

Add support to generate JSON from any model using OpenAI compatible pydantic types or jsonschema.
Add support for reasoning_effort, you can use it with deepseek r1 models.
Simplify local inference and support flash-atten 2 the local inference now works with very little degradation in performance when compared to vllm for requests that are not batched.

disable optimizations in inference

adjust minimum thinking tokens

fix the scirpt

add math500 eval

fixes

…tillm into feat-add-json-plugin

collect right tokesn

fix the thinking budgets

codelion added 19 commits January 30, 2025 13:21

add initial implementaion

2b811ba

Update inference.py

b40e011

disable optimizations in inference

add think deeper plugin

75e386c

Update thinkdeeper_plugin.py

dd3661a

Update thinkdeeper_plugin.py

3935597

Update thinkdeeper_plugin.py

57f1cf3

Update thinkdeeper_plugin.py

c23c294

Update thinkdeeper_plugin.py

fde8ea3

Update thinkdeeper_plugin.py

bbe663b

Update thinkdeeper_plugin.py

298a02c

Update thinkdeeper_plugin.py

4f09a37

move code around

982cd77

fix config

d5e7f6c

return response

c630dcc

singleton cache

8223736

Update thinkdeeper.py

ffc1ade

singleon cache

8c3466d

add tip

74b3bc2

add num_traces

5bf0c0e

codelion marked this pull request as draft February 5, 2025 13:59

codelion added 10 commits February 7, 2025 00:11

fix bugs

4bc73b5

fix tokens

c5c8d83

Update thinkdeeper.py

d56a6b6

adjust minimum thinking tokens

Update eval_aime_benchmark.py

6c06d47

fix the scirpt

removing num traces

3f4eb0d

Update eval_math500_benchmark.py

403416d

add math500 eval

Update eval_math500_benchmark.py

0479f85

Update eval_math500_benchmark.py

c2523ee

Update eval_math500_benchmark.py

c5a8ba7

Update eval_math500_benchmark.py

63f1a2a

fixes

codelion added 26 commits February 14, 2025 10:54

Update eval_math500_benchmark.py

befbfc3

Merge branch 'feat-add-json-plugin' of https://github.com/codelion/op…

ee744a4

…tillm into feat-add-json-plugin

Update eval_math500_benchmark.py

7330140

Update eval_math500_benchmark.py

7f957bc

Update eval_math500_benchmark.py

91ec172

Update eval_math500_benchmark.py

527df53

Update eval_math500_benchmark.py

edc67ad

Update eval_math500_benchmark.py

4068976

Update eval_math500_benchmark.py

839e5d6

Create gen_optillmbench.py

dca4b6f

Update gen_optillmbench.py

98fd9e4

add eval script

f89f89e

removed TIP added max_thoughts token

4b85f01

support reasoning_effort parameter for reaosning models

aad3062

Update inference.py

1bd01f4

Update thinkdeeper.py

4f9bdc6

fixes

3159768

Update inference.py

6524e18

Update eval_aime_benchmark.py

5e78652

Update eval_aime_benchmark.py

62e4948

collect right tokesn

Update eval_aime_benchmark.py

120ad87

Update eval_aime_benchmark.py

50ac90e

init

718b678

Update thinkdeeper.py

3a7d0f0

fix scripts

7755946

Update inference.py

2e380b6

fix the thinking budgets

codelion changed the title ~~Feat add json plugin~~ Feat add json plugin and support low, medium, high reasoning efforts for thinking models Feb 27, 2025

codelion marked this pull request as ready for review February 27, 2025 00:46

codelion merged commit 7be1f50 into main Feb 27, 2025
1 check passed

codelion deleted the feat-add-json-plugin branch February 27, 2025 00:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat add json plugin and support low, medium, high reasoning efforts for thinking models #163

Feat add json plugin and support low, medium, high reasoning efforts for thinking models #163

codelion commented Feb 5, 2025 •

edited

Loading

Feat add json plugin and support low, medium, high reasoning efforts for thinking models #163

Feat add json plugin and support low, medium, high reasoning efforts for thinking models #163

Conversation

codelion commented Feb 5, 2025 • edited Loading

codelion commented Feb 5, 2025 •

edited

Loading