Do you have an idea for what kind of hooks you would like? Off the top of my head I'm thinking:

1. Session start
2. Linting a file start
3. Linting a file end
4. Session end

Any others you can think of?

When a file CRUD takes place: would be good too. If file A depends on file B and file B changes, file A will likely need to be re-linted.

Yeah that's something that's missing - currently eslint assumes single-file linting, so there's no way to inform eslint of the dependencies between files.

This causes two immediate issues:

1. eslint's cache flag is broken if you use type-aware rules or many of eslint-plugin-import's rules
2. IDEs do not know that they should recalculate lints for an open file.

An LSP is a topic that has been brought up a few times in the past, but since other people rushed to create their own we never looked very closely at it.

I'll admit I'm a bit ignorant as to how LSPs work -- do all editors use the same format? Are there good example LSPs we can look at for guidance? (I"m guessing TypeScript is a bit heavy and maybe it would be easier to look at another example?)

Yes, all editors use the same format. LSP server is a standalone tool that reads/writes JSONs according to the spec.

I think the starting point is this LSP guide. It even mentions ESLint LSP as an example :). So @dbaeumer can tell more about what it takes to create and maintain the server.

You don't need to maintain the LSP client and VS Code ESLint extension, you are only interested in the server-related docs from that page.

Technically, you don't have to use the TypeScript API that VS Code suggests. For example, it's possible to write a server in Java or Kotlin using Eclipse LSP4J library. You don't even have to use any existing library and can write LSP server from scratch in any programming language (like Rust and Dart support is written in Rust and Dart respectively). Though I think that using TypeScript API is the easiest path for ESLint.

At the moment, each editor is maintaining its own wrapper over ESLint engine that converts ESLint logic into some IDE-compatible long-running language server. At JetBrains, we are using some custom non-standard way to communicate with our custom long-running wrapper process, but we'd be happy to switch to a standard LSP protocol.

Yup, Alexander is correct.
LSP = "language server protocol". It's a spec which defines the API for language server implementations.
Though note that "language server" is a bit of a misnomer, a better name would be "IDE interaction protocol" or something. As an example - an ESLint LSP wouldn't deal with a language, instead it would define the set of interactions that ESLint enables within an IDE.

The idea is that you can define an LSP implementation and have any IDE instantiate that, talk to it using LSP JSON messages, and have the same set of behaviours everywhere.

Eg an ESLint LSP implementation would ask the IDE to tell it when a file is changed, and in response would inform the IDE of the lint results for the file (which the IDE can then choose how to display to the user).

Awesome, thanks so much for all of the details @alexander-doroshko and @bradzacher. We can definitely look into creating an LSP server.

One thing to add: VS Code maintains a library to write LSP servers in JS / TS and run them in the browser in a worker or in Node. The library is vscode-languageserver. Source code is here: https://github.com/microsoft/vscode-languageserver-node.git in the server directory.

For context: right now fixes are actually multi-pass

Of course - which is entirely the problem.

In the CLI usecase this causes a problem for memory management.
If we're trying to figure out when we're done linting "project A" - we need to be sure that ESLint will not ask us to parse file X again.

However based on the current information that ESLint provides - all that we know is that ESLint wants us to parse file X. We don't know if parse 1 of file X is the first and last, or the first of 11 parses of the file.
This leads to a problem where we can never know for certain whether or not we're done with a file or not.

If we had some form of "lint phase" indicator for a file then we could know for sure when we're officially done with a file. Taken in culmination - when all files in project A are in the "lint complete" stage, then we can ditch project A from memory - helping prevent OOMs.

In terms of the persistent usecase - the lint phase would be less helpful overall, but could potentially unlock optimisations in future.

As a quick example - if we know that ESLint is going to ask us to parse / calculate types for file X up to 10 times in a row, then we may be able to optimise the incremental updates we're making to the types as the file contents are changed by the fixes.
We know that fixer changes will always apply just to the file being linted, and we also know that there's no way ESLint will ask us to lint a different file whilst it's fixing the current file.
With these two bits of information we could optimise the type updates so that we only do them "shallowly" during the fix passes. Put another way - instead of trying to update the types for file X and the entire tree of dependents on file X, we could cull that tree and just deal with file X. Oncethe file is "lint complete" then we know ESLint is about to lint the next file, so we would then do a complete typecheck pass on the final fixed text and update the entire tree of dependents with it.

Such an optimisation could theoretically improve autofix-on-save speed in codebases that use type-aware linting.

I'm assuming that A and B in your examples are source files in your project?

Yup, that's what I meant. Specifically thinking in terms of files

In the editor case, it seems like this would be a perfect example for an LSP to handle rather than a plugin?

Another usecase separate to IDEs is ESLint caching. Right now ESLint caching assumes all lint rules are single-file only and thus that changes to file A will never impact the lint results of file B.

If there was a way to tell ESLint "hey we know that file A depends on file B", then ESLint can do its own change-analysis to determine side-effects for things like invalidating caching correctly, or to inform IDE consumers that changes to file A will require re-linting of file B.

In terms of "should the LSP handle this" - it's difficult because you need to define what it means for files A and B to depend on one another.
On the surface it might seem simple for the pure JS usecase "does A import B?". But even then it can get complicated because you could have a case like this in B: export { val } from 'C'. Now A directly depends on B and transitively depends on C - meaning changes to C might also trigger re-lints on A.

But for TS it's much the same for values and types, but there are some extra cases to be aware of like global types declared in file D are used in file A, thus file A depends directly on file D.

For JS-like languages like angular or vue with their own templating languages, the template "html" files they use (which can be linted in ESLint today with @angular-eslint or vue-eslint-parser) will depend on the backing JS files that define their models.

For JS/TS there's also considerations about import mapping in tools (eg tsconfig paths or webpack loaders or webpack resolve aliases or rollup aliases, etc, etc).

All of this is a lot of information to encode into an LSP that is there to run ESLint on files. Some of it may even be impossible to properly encode without the user specifying their config in some way!

Instead it would probably make more sense for plugins to define this information and inform ESLint of these dependencies. For example:

• eslint-plugin-import has its own "plugin" system called "import resolvers" which allow the user to teach eslint-plugin-import how to resolve imports to files on disk so it can do its dependency analysis (eg eslint-import-resolver-typescript can be used to handle tsconfig paths and ensure the plugin looks for .ts files, not .js files when the extension isn't specified).
• When doing type-aware linting. @typescript-eslint knows all of the dependencies for any given file being linted because we've already computed the types for the entire project.
• Angular/vue eslint plugins have intricate knowledge of how to relate a template file to its model file.

If not, can you explain how you envision this communication?

I think there's probably a few ways you could go about this, but the simplest of ways would be to make this a first-class concept of ESLint "plugins". I.e. when ESLint asks a plugin to parse a file, the plugin parses it and also includes a list of the file's known dependencies.

That way ESLint has that information to act on for its caching, and it could pass it on to LSP consumers so that they can correctly re-lint files in response to changes.

Worth noting that for pure JS files doing purely single-file linting, this sort of system would be useless because two files cannot influence one-another. This sort of system is only interesting when there is cross-file linting in some way (type-aware linting, export shape analysis, etc).

If we're trying to figure out when we're done linting "project A" - we need to be sure that ESLint will not ask us to parse file X again.

Ah! 💡 I get it now. We assumed re-parsing the file was a zero-cost activity, and it is for Espree and other single-file parsers, but not for TypeScript. So it would be useful to be notified each step of the way. This is very helpful, thank you.

when ESLint asks a plugin to parse a file, the plugin parses it and also includes a list of the file's known dependencies.

This is something I've been thinking about as well, but I don't see a way of doing this without making import resolvers part of the JS language core for ESLint. Because dependencies can be packages and not just files, we'd need some way to resolve them into files to be actually useful.

without making import resolvers part of the JS language core for ESLint

I don't think you specifically need import resolvers - probably just doing "dependency resolvers" would be fine.

For example a plugin could export an API like

function resolveDependencies(
  ast: ESTree.Program,
  filePath: string,
): string[];

I.e. ESLint can ask for the list of absolute paths to files that are dependencies of this file. This would be a completely language or import resolution strategy agnostic thing - how the plugin arrives at the list of files is up to it.

• ESLint core could ship with a default no-op resolver - because ESLint core's rules all assume files are safe to parse in complete isolation.
• eslint-plugin-import would return the set of files resolved by its import resolver tooling.
• @typescript-eslint would resolve the set of files resolved by TS.
• @angular-eslint and Vue would return the model/controller file(s) for the template file.

ESLint can use these lists of files to build a dependency graph for the entire project which will let it correctly handle its cache invalidation, as well as triggering IDE re-linting.

Just spitballing - there may be better ways to do this!

Fair point. Putting the responsibility on the plugin makes sense -- it still seems like we'd want to be able to specify dependency resolvers as part of config, which would mean having it be a feature of the JavaScript language instead of asking every plugin to implement their own versions.

Definitely worth thinking more about this.

One issue that exists for a stateful parser in the IDE world is the dirty state of a file.

IDEs lint a files raw, unsaved text as the user types. This text is "dirty" until such time that the user saves the file and commits the text to disk.

Currently though stateful tooling doesn't know if it's parsing dirty state from an IDE or clean state from the disk because ESLint just says "parse this text".

The tricky thing is that you have to be careful about updating the internal state based on dirty text. If the user chooses not to save and instead closes the file without saving, then we will not be told about it by ESLint or the IDE - so we can't correctly purge the dirty state.

So if ESLint is looking to build tooling specifically for IDEs to integrate with - being able to have the IDE tell eslint "this file was opened", "this file was saved", "this file was closed" would be really vital in terms of knowing how to manage state.

So if ESLint is looking to build tooling specifically for IDEs to integrate with - being able to have the IDE tell eslint "this file was opened", "this file was saved", "this file was closed" would be really vital in terms of knowing how to manage state.

Excellent point. @dbaeumer does such a communication mechanism exist in VS Code?

@nzakas yes, such a communication mechanism does exist in VS Code and the LSP (https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_synchronization)

Would you be able to share more details about the problems you have around this? I'm wondering if the new config system makes your life easier or worse? This is particularly useful to understand because a lot of the internal file resolution in the old config system has been removed in favor of a JS-only config file, and I'm hoping that makes it easier for editors to integrate with?

Yes, it has mostly to do with how ESLint reads and finds config files. Here is an example to check if the config files can be parse correctly https://github.com/microsoft/vscode-eslint/blob/53cbf1b315ef3ae898d4ca6f1313272c05746a5c/server/src/eslintServer.ts#L510 which I do by validating an empty content with a fake file location.

Same trick is used here: https://github.com/microsoft/vscode-eslint/blob/53cbf1b315ef3ae898d4ca6f1313272c05746a5c/server/src/eslint.ts#L1065

I think the flat config files will ease that, but the validating of the config file is still something I would need to fake since I haven't found API to do this.

Thanks for those details! When you say that you're validating the config file, does that mean you're just validating its existence? Or are you actually wanting to make sure it doesn't cause an errors as its loaded?

For flat config, we could potentially expose a config file loader API. I need to think a bit more about that.

And do you have any examples of when plugins didn't work in VS Code due to the long-running session? If you have any issues you could point me to, that would be helpful.