Skip to content

Commit

Permalink
v3.0.11: fix version upgrade bug
Browse files Browse the repository at this point in the history
- use newer version of semver -- old version was sorting lexically
  instead of numerically

Refactor version comparison, check errors, maintain logic

summary of diff --git  a/v3/core/api.go b/v3/core/api.go

- Extract version comparison operation into a separate line to obtain result and error
- Check for errors after comparing versions with semver.Cmp
- Keep existing logic to append paths based on version comparison results

Enhance commit to allow custom git diff arguments

summary of diff --git  a/v3/core/cli.go b/v3/core/cli.go

- Add `Diffargs` field to `cmdCommit` struct to allow passing of custom arguments to `git diff`
- Modify `commitMessage` function to accept variable arguments for `git diff`
- Set default `git diff` arguments to `--staged` if no arguments provided by user
- Update `Cli` function to handle `commit` command with customizable `git diff` arguments
- Ensure `commit` and `commit <diffargs>` commands fall through to the same case, enabling argument passing

Update Tassert error message to include stdout and stderr

summary of diff --git  a/v3/core/cli_test.go b/v3/core/cli_test.go

- Update the error message in `Tassert` to include `stdout` and `stderr` information for better debugging insight

Ensure CLI output matches expectations with stdout.String()

summary of diff --git ")

- Assert that the CLI returns the expected output based on stdout.String()

Update version in grokker.go from 3.0.10 to 3.0.11

summary of diff --git  a/v3/core/grokker.go b/v3/core/grokker.go

- Update version from 3.0.10 to 3.0.11 in grokker.go

Refactor semver checks and error handling in migrate.go

summary of diff --git  a/v3/core/migrate.go b/v3/core/migrate.go

- Extract comparison of `dbver` and `codever` into a separate variable `cmp` to avoid calling `semver.Cmp()` twice
- Check for errors immediately after calling `semver.Cmp()`
- Update the check for database version newer than code to use the `cmp` variable
- Expand the call to `semver.Upgrade()` to include an error check and adapt it to an updated function signature that returns four values instead of three

Prefix ./ to grok in TestMigration_2_1_2; add TestMigration_3_0_10 test.

summary of diff --git  a/v3/core/migration_test.go b/v3/core/migration_test.go

- Prefix "./" to the "grok" command in the calls within `TestMigration_2_1_2` test function to ensure correct executable path
- Introduce a new test function `TestMigration_3_0_10` to verify integer ordering works as expected across upgrades
- In `TestMigration_3_0_10`, add steps to build "grok" for versions "v3.0.9" and "v3.0.10" and execute "ls" command to trigger version upgrade process

Update go-openai, relocate stevegt/semver with new version

summary of diff --git  a/v3/go.mod b/v3/go.mod

- Update github.com/sashabaranov/go-openai from v1.9.0 to v1.19.1
- Remove github.com/stevegt/semver version 92220054a49f from the first require block
- Add github.com/stevegt/semver version 5913d1a31c26 to the second require block

Update stevegt/semver to new version, replace hashes

summary of diff --git  a/v3/go.sum b/v3/go.sum

- Update github.com/stevegt/semver to version v0.0.0-20240217000820-5913d1a31c26
- Replace previous hashes for github.com/stevegt/semver with new ones reflecting the update
  • Loading branch information
stevegt committed Feb 17, 2024
1 parent 4a3cef1 commit 85c407b
Show file tree
Hide file tree
Showing 8 changed files with 42 additions and 16 deletions.
4 changes: 3 additions & 1 deletion v3/core/api.go
Original file line number Diff line number Diff line change
Expand Up @@ -359,7 +359,9 @@ func (g *GrokkerInternal) ListDocuments() (paths []string) {
v100, err := semver.Parse([]byte("1.0.0"))
current, err := semver.Parse([]byte(g.Version))
Ck(err)
if semver.Cmp(current, v100) > 0 {
cmp, err := semver.Cmp(current, v100)
Ck(err)
if cmp > 0 {
path = doc.RelPath
}
paths = append(paths, path)
Expand Down
18 changes: 13 additions & 5 deletions v3/core/cli.go
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,9 @@ type cmdChat struct {
NoAddToDb bool `short:"D" help:"Do not add the chat history file to the knowledge base."`
}

type cmdCommit struct{}
type cmdCommit struct {
Diffargs []string `arg:"" optional:"" type:"string" help:"Arguments to pass to git diff. If not provided, defaults to '--staged'."`
}

type cmdCtx struct {
Tokenlimit int `arg:"" type:"int" help:"Maximum number of tokens to include in the context."`
Expand Down Expand Up @@ -572,8 +574,13 @@ func Cli(args []string, config *CliConfig) (rc int, err error) {
Ck(err)
Pl(res)
case "commit":
fallthrough
case "commit <diffargs>":
// generate a git commit message
summary, err := commitMessage(grok)
if len(cli.Commit.Diffargs) < 1 {
cli.Commit.Diffargs = []string{"--staged"}
}
summary, err := commitMessage(grok, cli.Commit.Diffargs...)
Ck(err)
Pl(summary)
case "models":
Expand Down Expand Up @@ -672,11 +679,12 @@ func msg(g *GrokkerInternal, sysmsg string, input string) (res string, err error
}

// generate a git commit message
func commitMessage(grok *GrokkerInternal) (summary string, err error) {
func commitMessage(grok *GrokkerInternal, args ...string) (summary string, err error) {
defer Return(&err)

// run `git diff --staged`
cmd := exec.Command("git", "diff", "--staged")
// run `git diff @args
args = append([]string{"diff"}, args...)
cmd := exec.Command("git", args...)
cmd.Stderr = os.Stderr
out, err := cmd.Output()
Ck(err)
Expand Down
2 changes: 1 addition & 1 deletion v3/core/cli_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -406,7 +406,7 @@ func TestCli(t *testing.T) {
run(t, "git", "add", ".")
// git diff
stdout, stderr, err = grok(emptyStdin, "commit")
Tassert(t, err == nil, "CLI returned unexpected error: %v", err)
Tassert(t, err == nil, "CLI returned unexpected error: %v\nstdout: %v\nstderr: %v", err, stdout.String(), stderr.String())
// check that the stdout buffer contains the expected output
match = strings.Contains(stdout.String(), "diff --git")
Tassert(t, match, "CLI did not return expected output: %s", stdout.String())
Expand Down
2 changes: 1 addition & 1 deletion v3/core/grokker.go
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ import (
const (
// See the "Semantic Versioning" section of the README for
// information on API and db stability and versioning.
version = "3.0.10"
version = "3.0.11"
)

type GrokkerInternal struct {
Expand Down
10 changes: 7 additions & 3 deletions v3/core/migrate.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,13 +47,16 @@ func (g *GrokkerInternal) migrate() (migrated bool, was, now string, err error)
Ck(err)
codever, err = semver.Parse([]byte(version))
Ck(err)
if semver.Cmp(dbver, codever) == 0 {
var cmp int
cmp, err = semver.Cmp(dbver, codever)
Ck(err)
if cmp == 0 {
// no migration necessary
break
}

// see if db is newer version than code
if semver.Cmp(dbver, codever) > 0 {
if cmp > 0 {
// db is newer than code
err = fmt.Errorf("grokker db is version %s, but you're running version %s -- upgrade grokker", g.Version, version)
return
Expand All @@ -62,7 +65,8 @@ func (g *GrokkerInternal) migrate() (migrated bool, was, now string, err error)
Fpf(os.Stderr, "migrating from %s to %s\n", g.Version, version)

// if we get here, then dbver < codever
_, minor, patch := semver.Upgrade(dbver, codever)
var minor, patch bool
_, minor, patch, _, err = semver.Upgrade(dbver, codever)
Assert(patch, "patch should be true: %s -> %s", dbver, codever)

// figure out what kind of migration we need to do
Expand Down
16 changes: 14 additions & 2 deletions v3/core/migration_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -163,13 +163,25 @@ func TestMigration_2_1_2(t *testing.T) {
// create a file with 1 chunk of 20000 bytes
// (about 11300 tokens each chunk depending on hash content)
mkDataFile("testfile-1-20000.txt", 1, 20000)
run(t, "grok", "add", "testfile-1-20000.txt")
run(t, "./grok", "add", "testfile-1-20000.txt")

// test with 3 chunks much larger than GPT-4 token size
// create a file with 3 chunks of 300000 bytes
// (about 167600 tokens each chunk depending on hash content)
mkDataFile("testfile-3-300000.txt", 3, 300000)
run(t, "grok", "add", "testfile-3-300000.txt")
run(t, "./grok", "add", "testfile-3-300000.txt")
}

// test migration from v3.0.9 to 3.0.10 so we know integer ordering
// works as expected
func TestMigration_3_0_10(t *testing.T) {
mkGrok(t, "v3.0.9", "v3/cmd/grok")
// run ls to get upgraded to 3.0.9
run(t, "./grok", "ls")
// now build 3.0.10
mkGrok(t, "v3.0.10", "v3/cmd/grok")
// run ls to get upgraded to 3.0.10
run(t, "./grok", "ls")
}

func TestMigrationHead(t *testing.T) {
Expand Down
2 changes: 1 addition & 1 deletion v3/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,13 @@ require (
// github.com/sashabaranov/go-openai v1.9.0
github.com/sashabaranov/go-openai v1.19.1
github.com/stevegt/goadapt v0.7.0
github.com/stevegt/semver v0.0.0-20230512043732-92220054a49f
github.com/tiktoken-go/tokenizer v0.1.0
)

require (
github.com/sergi/go-diff v1.3.1
github.com/stevegt/envi v0.2.0
github.com/stevegt/semver v0.0.0-20240217000820-5913d1a31c26
)

require (
Expand Down
4 changes: 2 additions & 2 deletions v3/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ github.com/stevegt/envi v0.2.0/go.mod h1:Z8w7bE5V9Ce3H02CWNTEYkW3zWE8ulgfROtEt6y
github.com/stevegt/goadapt v0.0.13/go.mod h1:BWNnTsXdIxaseRo0W/MoVgDeLNf+6L4S4fPhyAsBTi0=
github.com/stevegt/goadapt v0.7.0 h1:brUmaaA4mr3hqQfglDAQh7/MVSWak52mEAOzfbSoMDg=
github.com/stevegt/goadapt v0.7.0/go.mod h1:vquRbAl0Ek4iJHCvFUEDxziTsETR2HOT7r64NolhDKs=
github.com/stevegt/semver v0.0.0-20230512043732-92220054a49f h1:erQJkdWx1bhOImDDPiVoNy+qP8sBoOJ8EsJ7gUiy8S8=
github.com/stevegt/semver v0.0.0-20230512043732-92220054a49f/go.mod h1:OgjZERzdomhsWRdqEbqfC+2DI1DpdmZwc1CF7vhqAZ8=
github.com/stevegt/semver v0.0.0-20240217000820-5913d1a31c26 h1:z1tzm2Q22jtCN87NlApplnTx/EPQQ4zyZaDgNvw3wg8=
github.com/stevegt/semver v0.0.0-20240217000820-5913d1a31c26/go.mod h1:Jm8NvUiaWMGzaCtI0Ja7Vy7/7ZESxEMWYGswGyqW1+A=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
Expand Down

0 comments on commit 85c407b

Please sign in to comment.