[Feature]: Aider-inspired linting functionality

[neubig]

What problem or use case are you trying to solve?

Aider has functionality for linting code in multiple languages that could be helpful for OpenDevin.

Describe the UX of the solution you'd like

It would be good if OpenDevin agents (optionally?) linted the code they generated so that they could create code that passes basic syntax and style checks.

Do you have thoughts on the technical implementation?

Currently OpenDevin supports some rudimentary linting.

In contrast, Aider's linting has a number of good points that OpenDevin does not yet support:

• Multi-language support with treesitter
• The conversion of the output into a prompt seems more sophisticated

We may want to take some inspiration from their work, either by importing the library wholesale and replacing our current agentskills implementation with theirs, or more piecemeal.

Additional context

Aider article on swe-bench lite.
Aider super-issue #120

[tobitege]

Fyi, started working on a PR to make use of Aider's Linter class within agentskills edit_file.
Unit tests already done.

[jeremi]

What about also supporting pre-commit? That would allow linting to be more customizable and add an abstraction, so this project does not need to support all the linting systems.

[tobitege]

What about also supporting pre-commit? That would allow linting to be more customizable and add an abstraction, so this project does not need to support all the linting systems.

Pre-commit already has linting enabled. Here we are talking about linting of LLM-generated code.

[jeremi]

Yes, the generated code could be linted with pre-commit before it being run.
pre-commit run --files $(git ls-files -m)
In our case, we run many different linters, some specific to our project. The advantage is that they already have all the code to install the needed dependencies for almost all linters out there.

[tobitege]

That's an interesting approach for sure. Does it install dependencies by file extention, for example?
Just wondering whether it auto-magically works with .cs or .lua files instead of .py

[jeremi]

We can make it do it if we configure all the languages. However, by configuring all the languages, I guess the issue is the time it will take to set up the virtual environment with all the linters.
I see different options:

• Project create their own config, specifying the one they want.
• Based on the language used in the project we generate the adapted config (for example, just supporting Python & JS if the project just use this).

For example, ours is quite advanced:
https://github.com/OpenSPP/openspp-modules/blob/17.0/.pre-commit-config.yaml

but some can be simpler:
https://github.com/pre-commit/pre-commit/blob/main/.pre-commit-config.yaml

I like that it standardizes the way to run them, and most are already integrated with it:
https://pre-commit.com/hooks.html

[neubig]

@jeremi I love this idea! Overall one big philosophy of OpenDevin is that we want to re-use the tools that already exist for software development, and pre-commit is exactly that sort of tool.

[JeffKatzy]

I'm working on integrating the aider linter, and with Python code everything works fine. But was also trying to get fixes with non-Python code. For example, with ruby code, the linter detects the error, and creates an ipythoncellobservation indicating there is a syntax error, but the agent does not actually provide the correct fix -- it just repeats the code with poor syntax. Any ideas?

I tried some prompt engineering with the observation, but it doesn't matter.

All of this is good

**IPythonRunCellAction**
THOUGHT: Now, I will write the Ruby code with the syntax error into the file `run.rb`:
CODE:
EDITED_CODE="""def print_hello_world()\n puts 'Hello World'\n"""
edit_file('run.rb', start=1, end=1, content=EDITED_CODE)
21:29:06 - opendevin:INFO: browser_env.py:95 - Browser env started.
21:29:06 - OBSERVATION
**IPythonRunCellObservation**
Now linting file run.rb
[Your proposed edit has introduced new syntax error(s) in the Ruby programming language. Please understand the errors and retry your edit command.]
ERRORS:
/workspace/run.rb:0
[This is how your edit would have looked if applied]
-------------------------------------------------
1|def print_hello_world()
2| puts 'Hello World'
-------------------------------------------------

This code in line 1 has a syntax error: 1|def print_hello_world()
Your changes have NOT been applied. Please fix your edit command and try again.
You either need to 1) Specify the correct start/end line arguments or 2) Correct your edit code.
DO NOT re-run the same failed edit command. Running it again will lead to the same error.

But this is bad. In ruby, the parentheses should be removed. So the updated code should be def print_hello_world. But it just outputs the original code.

**IPythonRunCellAction**
THOUGHT: It seems there was a syntax error in the provided Ruby code. Let me correct the syntax error and update the file `run.rb` with the corrected code:
CODE:
EDITED_CODE="""def print_hello_world()\n puts 'Hello World'\nend"""
edit_file('run.rb', start=1, end=1, content=EDITED_CODE)

In python, with similar level of hints in the observation, it just makes the fix.

Any ideas on how to correct this? Is multi-language support needed?

[tobitege]

cc @neubig please also see #2489 which @JeffKatzy has been working on for a while.
Wanted to get your review first before merging. Might need a little work to catch up with the refactors now, but other than that, was good to go.

[neubig]

Fixed by #2489 !