cli/dev/website/_posts/2024-05-22-linting.md

title, excerpt, highlight_image, nav_exclude

title	excerpt	highlight_image	nav_exclude
Linting code for LLMs with tree-sitter	Dev now lints code after every LLM edit and automatically fixes errors, using tree-sitter and AST-aware code context.	/assets/linting.jpg	true

{% if page.date %}

{{ page.date | date: "%B %d, %Y" }}

{% endif %}

Linting code for LLMs with tree-sitter

Dev now lints your code after every LLM edit, and offers to automatically fix any linting errors. You can also use dev's lint-and-fix functionality on your source files any time you like, to speedily resolve issues with code written by humans.

Dev shows linting errors to the LLM in a novel format, using tree-sitter to help display relevant code context for each error. This increases the ability of the LLM to understand the problem and make the correct changes to resolve it.

Dev ships with basic linters built with tree-sitter that support most popular programming languages. These built in linters will detect syntax errors and other fatal problems with the code.

You can also configure dev to use your preferred linters. This allows dev to check for a larger class of problems, keep the code style aligned with the rest of your team, etc.

Linting and fixing your code

Dev now lints each source file after it applies the edits suggested by an LLM. If problems are found, dev will ask if you'd like it to attempt to fix the errors. If so, dev will send the LLM a report of the lint errors and request changes to fix them. This process may iterate a few times as the LLM works to fully resolve all the issues.

You can also lint and fix files any time, on demand from within the dev chat or via the command line:

• The in-chat /lint command will lint and fix all the files which have been added to the chat by default. Or you can name any files in your git repo as arguments.
• From the command line, you can run dev --lint to lint and fix all the dirty files in the repo. Or you can specify specific filenames on the command line.

An LLM-friendly lint report

Most linting tools produce terse and cryptic output, which is one reason many engineers appreciate IDEs that highlight linting errors. LLM's don't have the luxury of using an IDE, so dev sends the linting errors in an LLM friendly format.

Here's an example of raw output of the flake8 python linter:

app.py:23:36: F821 undefined name 'num'
app.py:41:16: F541 f-string is missing placeholders

This sort of output depends on the user to reference line numbers to find and fix each reported error. LLMs are quite bad at working with source code line numbers, often making off-by-one errors and other mistakes even when provided with a fully numbered code listing.

Dev augments the raw linter by displaying and highlighting the lines that have errors within their containing functions, methods, classes. To do this, dev uses tree-sitter to obtain the code's AST and analyzes it in light of the linting errors. LLMs are more effective at editing code that's provided with context like this.

app.py:23:36: F821 undefined name 'num'
app.py:41:16: F541 f-string is missing placeholders

app.py:
...⋮...
  6│class LongNum:
  7│    def __init__(self, num):
  8│        """
  9│        Initialize the number.
 10│        """
...⋮...
 19│    def __str__(self):
 20│        """
 21│        Render the number as a string.
 22│        """
 23█        return str(num)
 24│
 25│
 26│@app.route('/subtract/<int:x>/<int:y>')
...⋮...
 38│@app.route('/divide/<int:x>/<int:y>')
 39│def divide(x, y):
 40│    if y == 0:
 41█        return f"Error: Cannot divide by zero"
 42│    else:
 43│        result = x / y
 44│        return str(result)
 45│
...⋮...

Basic linters for most popular languages

Dev comes batteries-included with built in linters for most popular programming languages. This provides wide support for linting without requiring users to manually install a linter and configure it to work with dev.

Dev's built in language-agnostic linter uses tree-sitter to parse the AST of each file. When tree-sitter encounters a syntax error or other fatal issue parsing a source file, it inserts an AST node with type ERROR. Dev simply uses these ERROR nodes to identify all the lines with syntax or other types of fatal error, and displays them in the LLM friendly format described above.

Configuring your preferred linters

You can optionally configure dev to use your preferred linters with the --lint-cmd switch.

# To lint javascript with jslint
dev --lint-cmd javascript:jslint

# To lint python with flake8 using some specific args:
dev --lint-cmd "python:flake8 --select=E9,F821,F823..."

You can provide multiple --lint-cmd switches to set linters for various languages. You can also durably set linters in your .dev.conf.yml file.