Upcoming removal of built-in libraries

[carsakiller]

Hello 👋

With the VS Code addon manager releasing soon, we have decided to remove the built-in libraries that ship with the language server in a future release. This will likely take place in approximately 6 months.

This means the ${3rd} shortcut for workspace.library will no longer work.

Note
Older versions of the server that still contain the built-in libraries will continue to function as-is.

Why?

The addon manager for VS Code is being implemented soon, meaning everyone using the VS Code extension will still be able to install definition files super easily.

Removing the built-in libraries helps prevent duplicates for those using VS Code and can help cut down on bloat for everyone. It is unlikely that everyone is interested in all the built-in libraries, so many of them end up not being used.

What is the New Method?

Libraries are now addons. The name “libraries” is a little misleading, as they are actually just definitions (and maybe a plugin).

Those using VS Code will soon be able to use the addon manager to install addons that contain all the definition files that were included in the language server, and more!

Those using other clients can still download all the libraries that were included in the server from the LuaCATS organization. They can then be dropped into a folder and linked to with workspace.userThirdParty for functionality similar to how the built-ins have previously worked, or manually linked using workspace.library.

Contribute

Collaborating on definition files is now better than ever.

Do you have definition files for a library? Now is the time to contribute them! If you would like it to show up in the addon manager, you can add it to LLS-Addons.

Do you have definition files that you can no longer maintain? If you transfer ownership to me (@carsakiller), I can transfer the project to LuaCATS, where the community can work together to keep the definitions up-to-date.

Future Changes

I am currently investigating how we can make using and sharing definitions files even easier. Ideally, the language server will be able to install and manage addons defined in a setting by simply providing the git clone URLs. This would hopefully be cross-platform and super easy for all users.

Thank you for reading, and please feel free to leave feedback below.

[ShamblesSM]

I'm curious, how big are the built-in libraries, exactly? This would make the download lighter, which is a nice plus.

P.S: There's this repo for The Binding of Isaac Lua modding: https://github.com/filloax/isaac-lua-api-vscode, you should probably get in touch with the author to have it be ported.

[carsakiller]

I believe the currently included ones are around 3 MB (source), so they account for about 10% of the total size of the language server. It is still a big improvement for organization and collaboration, though 🙂

There isn't much “porting” to do. The addons are nearly identical to the environment emulation libraries that have existed for a while.

[ShamblesSM]

Yeah, I'm thinking that if it'd be in a repo, there's no need to update your extension anymore, and they need to update the plugin instead.

[goldenstein64]

My addon at goldenstein64/pl-definitions depends on ${3rd}/lfs for the pl.path module to stay faithful to its original implementation. If built-in libraries are getting removed soon, is there at least a way to reference those other libraries or mark them as dependencies?

So far, it looks like I have three options:

1. Ask for a change to info.json or config.json to support referencing dependencies
2. Copy lfs definitions into the library folder
3. Tell users to install the LuaFileSystem addon

I'm okay with any of these decisions, but I would prefer (1) the most.

[carsakiller]

Hello 👋

I plan to add to config.json so that addons can depend on each other. This should be completed before the libraries are removed. Currently, I am experimenting and trying to move the addon manager to the language server, so it can be cross-editor. Until these are completed, I recommend just adding to your description that another addon is required 🙂.

[esatterwhite]

move the addon manager to the language server, so it can be cross-editor

as someone that does a lot of lua development and doe not vscode, I see this as a must have.

honestly, if a little bit of bloat means I get the same experience out of the language server on any platform and not have to dredge the documentation of 10 repos to figure out how to set all this up or have to maintain everyone's editor specific config - huge win.

at the day job we have several hundred libraries and dozens of developers all using different editors and local setups. As a rule we do not commit editor specific settings to repos and lean on agnostic configuration so everyone is on a level playing ground and not try to force people into using an editor.

[sewbacca]

Could the LuaCATS Addon Manager itself support a ${3rd} variable? In my mind this would have 2 benefits:

• The settings.json stays distributable. As long as there are user paths used, noone else can use your settings.json as is.
• Additionally with a variable, LuaCATS could autodetect what modules fo download.

[carsakiller]

I plan to add a ${addon} variable that points to the addon installation path... wherever that may be. The settings file not being distributable has always annoyed me.

[fesily]

I need it!

[forestbelton]

Hey @carsakiller, I wanted to check in and see support for an ${addon}-like variable was still in the works. This would be a huge help for simplifying the onboarding process for our team!

I looked around within the codebase to see where this change could be introduced but didn't come up with much. If you could point me in the right direction here, I'd be happy to add it in and open a PR.

Regardless, thanks for all of your hard work here!

[carsakiller]

@forestbelton, I had forgotten about this 😵‍💫. I took a poke around and appear to have come up with a potential solution.

[suy]

I'm still very confused as to how is this supposed to be used, because the ${addons} placeholder added in #2866 is specific to VS Code, and I don't see why ${3rd} is deprecated but apparently userThirdParty is not?

For example, both of this configurations seem to work:

-- approach 1. This works.
-- * Adds rocks and busted to workspace.library, using the "3rd" shortcut,
-- * userThirdParty set to the path to addons
require('lspconfig').lua_ls.setup {
    settings = {
        Lua = {
            workspace = {
                library = {
                    vim.fn.expand('~/local/rocks'),
                    "${3rd}/busted/library",
                },
                checkThirdParty = 'ApplyInMemory',
                userThirdParty = { -- This where I copied the LuaCATS/busted repo
                    vim.fn.expand("~/personal/configs/lua-language-server-addons/")
                },
            }
        }
    }
}

-- approach 2. This works
-- * Adds rocks and busted/library directly to workspace.library, without shortcut
-- * userThirdParty is not used at all
require('lspconfig').lua_ls.setup {
    settings = {
        Lua = {
            workspace = {
                library = {
                    vim.fn.expand('~/local/rocks'),
                    vim.fn.expand('~/personal/configs/lua-language-server-addons/busted/library'),
                },
            }
        }
    }
}

Could you hint about how one is advised to configure this, @carsakiller?

Thank you in advance!

[suy]

An update. I've been trying still to make it work, and I think I've been able to. First, I narrowed it down to something that wasn't working, to see if I could fix it. I tried this:

-- approach 3. This does NOT work
-- * Adds userThirdParty directory, but doesn't refer to it in library.
-- (this is the snippet from https://github.com/LelouchHe/xmake-luals-addon/pull/1/files)
require('lspconfig').lua_ls.setup {
    settings = {
        Lua = {
            workspace = {
                userThirdParty = {
                   vim.fn.expand('~/personal/configs/lua-language-server-addons/')
                }
            }
        }
    }
}

-- approach 4. This does NOT work.... with busted, but works with xmake!!
-- * Adds userThirdParty directory, but doesn't refer to it in library.
-- * Adds checkThirdParty, which is in other working examples
require('lspconfig').lua_ls.setup {
    settings = {
        Lua = {
            workspace = {
                checkThirdParty = 'ApplyInMemory',
                userThirdParty = {
                   vim.fn.expand('~/personal/configs/lua-language-server-addons/')
                }
            }
        }
    }
}

The 3rd approach was not working for me, but I tried the 4th, and by chance I realized that it was working for an xmake.lua file, which was detecting fine the xmake specific global functions. But it was failing for a plain Lua project that used busted, so I dug up, and realized that the xmake addon/LuaCATS has a rule to enable itself on xmake.lua files, but the busted one has nothing... So I made a change, and with that, it works, at last.

It would be nice to know still about the ${addon} placeholder being "cross editor", because it doesn't seem so to me. In fact, the busted addon sets luassert in workspace.library using ${3rd}/luassert/library. If this goes away, I don't see how this addon will achieve this feature.

Thanks.

[carsakiller]

Lua.workspace.library is an array of paths to library files.

Lua.workspace.userThirdParty is an array of paths where addons can be found. If the addon has words or files that define when it should be activated, then the language server can do it automatically for you.

${3rd} refers to the libraries that are built into the Language server, that we plan to remove in the future.

And ${addons} is used by the addon manager, but the location will probably move somewhere cross-platform in the future.

[Thaodan]

Does this mean extensions can still be dropped into the installation directory? What is the replaced path for the 3rd directory?