Wishlist.

[arthurwolf]

(Love the project, awesome work).

I see Devin/this project doing demos like "do this simple video game from scratch", or "do a to-do list web app", starting from zero and using whatever libraries/technologies it prefers.

Those are neat, but I don't think they are the most useful, at least for some people.

Here's what I would want/need these projects to do, in case that would be useful to know, in the form of the prompts as I'd ask them of the agents:

1. (Coming into my existing project/database), tell it: « in src/lib/Editor.ts, there's a function to resize images according to the specs it finds in the database, I'd like that function (and the other sub-functions it calls) to be put into a new library named Resizer.ts in src/lib/utils/, and change Editor.ts to use that library »

2. « Once that's done, find other places in the code where we resize images, and for each, change them to use the new Resizer library. »

3. « In src/test, using the same format and testing library as I am already using in my tools, write a series of tests for the Resizer library. You can find "sample" images to resize inside of data/samples/resized-images/ »

4. « My project is made of two parts, the scripts (run with Node) in src/, and the ui (written in Vuejs, run in the browser) in ui/src/. Each part has its own "Page" class/abstraction with different functions/properties. I'd like you to join both of these into a single file/class that both the scripts and the UI can call/use. This is tricky because a few of the libraries the "scripts"-side class uses will not work in the browser. You must find a solution that allows us to have only one class everything imports, but to not have errors in the browser despite this. Propose and explain in detail a possible solution to this problem for me to review, then if I like the solution I'll ask you to implement it. »

If your project was able to do this, it would completely change my life.

Just a wishlist :)

Awesome work by the way.

[dorbanianas]

Liked your suggestions, thank you!

[foragerr]

That is certainly the eventual goal @arthurwolf
But agent capability isn't quite there yet. And I don't mean for OpenDevin, I mean even for the most cutting edge capability today.

swe-bench is a benchmark created by researchers to evaluate how well "AI software engineers" can perform against real world code fix scenarios, and even Devin which made a giant splash recently only scored 13% on it. Which makes these sort of efforts very interesting research areas, but not useful in the real world yet.

But progress is rapid! A NUS team just announced a 22% achievement on a slightly modified benchmark (called swe-bench-lite). This was just 2 days ago!

This PR is looking to implement a better agent, to achieve a swe-bench score close to Devin's.

OpenDevin is looking to incorporate all the latest agent advances as quickly as possible. Here's to hoping your ask becomes reality sooner rather than later!

Oh and I must add: The project welcomes contributions and PRs!

[rbren]

@arthurwolf we have many of the same goals!

Since this issue isn't particularly actionable, I'm going to close it. But thanks for starting some discussion! Feel free to join the Slack or Discord to discuss further.

Or if you have a specific feature request or enhancement to propose (e.g. this issue on using stack graphs to analyze existing codebases) feel free to open a new issue!

[arthurwolf]

@foragerr I completely get we're not there yet. I hope as we get better models (they seem to be coming quickly/often) it'll help you guys make it further. Was mostly trying to "express user needs" to make sure this sort of stuff isn't forgotten, but you all are also devs so I'm sure you also have similar wishes.

Good to know on the contributions, I'll probably only start using it once it's a bit more capable than it is now (it's still early), but as soon as it can help for what I do I'll be all-in, and likely to contribute.

Thanks again.