Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Experimental use of LLMs for completing issues #87

Open
adamamer20 opened this issue Aug 29, 2024 · 2 comments
Open

Experimental use of LLMs for completing issues #87

adamamer20 opened this issue Aug 29, 2024 · 2 comments
Labels
ci Changes to CI configuration files and scripts.

Comments

@adamamer20
Copy link
Collaborator

adamamer20 commented Aug 29, 2024

I believe with a good CI suite in place and a good template for issues (eg. current behavior, expected behavior...) we could use a local LLM like DeepSeek-Coder to solve the easiest issues and simply review the changes. This would aid development greatly because it would allow to focus only on the most difficult issues and less on actual mantainance.
Look at this repo: https://github.com/princeton-nlp/SWE-agent?tab=readme-ov-file

@adamamer20 adamamer20 added the ci Changes to CI configuration files and scripts. label Aug 29, 2024
@rht
Copy link
Contributor

rht commented Aug 29, 2024

According to OpenAI's modification of SWE-bench, Aider performs better than SWE-Agent, while Agentless is the best performing one.

@adamamer20
Copy link
Collaborator Author

adamamer20 commented Aug 29, 2024

According to OpenAI's modification of SWE-bench, Aider performs better than SWE-Agent, while Agentless is the best performing one.

Thanks @rht! Always on top :). I took a look at Agentless and right now the documentation is fairly lacking (it shows how to reproduce the SWE-Bench result but that's it). We can try using Aider, it seems well documented and easy to setup!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci Changes to CI configuration files and scripts.
Projects
None yet
Development

No branches or pull requests

2 participants