Browser-only static extension support #14776

robertjndw · 2025-01-27T12:45:08Z

What it does

This PR introduces support for static extension/plugin declaration in the Browser-Only mode. The required plugins can either be download and added manually to the respective directory or automatically through the theiaPlugins property. I've added a script which automatically unpacks the .vsix files in the theiaPluginsDir property and bundles them with the browser-only variant of Theia.
Currently, the metadata have to be specified by hand afterwards and will be injected at startup. An example of this can be found the in the respective directory.
Details on how to specify and craft those metadata are described in the file ADOPTING.md

How to test

As an example, I have chosen the Typescript Language tools as they run smoothly and represent a real world use case:

Head to the browser-only example and run npm run download:plugins
Change the corresponding paths in example-static-plugin-metadata.ts to the respective path on your system. Currently, it assumes /home/user/theia as path, just replace this part with your local theia location
Run the npm run build command to build and package the plugins with the browser-only version
Start the application with npm run start
Head over to localhost:3000
Verify that the plugins are installed (under the extensions tab, they should be listed as builtin)
Create a Typescript file and while coding the autocompletion and syntax highlighting should work

Follow-ups

Unfortunately, there are some limitation which can/should be tackled in future work

The metadata configuration is currently done by hand. There are some methods in the node directory which would come in handy in automatic this. These should be outsourced into a independent package for handling Extension package.json files in both the node and browser-only variant
There is no dynamic installation of plugins. They have to be configured and bundled with Theia. In future work, there might be an option to install plugins at runtime from the OpenVSX registry.
Some VScode Web Extensions do not work. It seems that it has to do with the way they are bundled. An error message like module is undefined occurs when the plugin is executed and it happens in statements like module.export = i. This is strange as module is typically a Node.js thing. However, the general installation and bundling works and the error occurs directly in the plugin's bundled code when executed. To be investigated further ...

Breaking changes

This PR introduces breaking changes and requires careful review. If yes, the breaking changes section in the changelog has been updated.

Attribution

Review checklist

As an author, I have thoroughly tested my changes and carefully followed the review guidelines

Reminder for reviewers

As a reviewer, I agree to behave in accordance with the review guidelines

…one'

…r-only implementation

… method

…e naming of package.json scripts

…r-only-extension-support

sdirix

Thanks for the contribution!

Sadly I was not able to get the TS plugin working on my machine. It seems that some plugin storage directory was not creatable in opfs.

failed to load plugins: Unknown (FileSystemError): Error while accessing resource file:///globalStorage
    at createFileSystemProviderError (http://localhost:3000/bundle.js:381479:27)
    at OPFSFileSystemProvider.stat (http://localhost:3000/packages_filesystem_lib_browser-only_browser-only-filesystem-frontend-module_js.js:235:61)
    at async FileService.mkdirp (http://localhost:3000/bundle.js:378429:30)
    at async FileService.createFolder (http://localhost:3000/bundle.js:378413:9)
    at async HostedPluginSupport.getHostGlobalStoragePath (http://localhost:3000/packages_plugin-ext_lib_hosted_browser_hosted-plugin_js.js:732:13)
    at async Promise.all (index 2)
    at async HostedPluginSupport.startPlugins (http://localhost:3000/packages_plugin-ext_lib_hosted_browser_hosted-plugin_js.js:1401:71)
    at async HostedPluginSupport.doLoad (http://localhost:3000/packages_plugin-ext_lib_hosted_browser_hosted-plugin_js.js:1265:9)
    at async ProgressService.withProgress (http://localhost:3000/bundle.js:357799:20)
    at async http://localhost:3000/packages_plugin-ext_lib_hosted_browser_hosted-plugin_js.js:1214:17

sdirix · 2025-01-29T07:12:02Z

ADOPTING.md

+There are two ways to retrieve extensions:
+1. VSIX packages: Copy the `.vsix` files to the directory specified in the `package.json` file by the `theiaPluginsDir` property. (e.g. `"theiaPluginsDir": "plugins"`)
+2. Open VSX Link: Specify a list of `id:link` mappings in the `package.json` file by the `theiaPlugins` property (e.g. `"theiaPlugins": { "vscodevim.vim": "https://open-vsx.org/api/vscodevim/vim/1.29.0/file/vscodevim.vim-1.29.0.vsix" }`). Extensions can be found on the [Open VSX Registry](https://open-vsx.org/) by searching for the extension and copying the link linked to the Download button.
+When using the command `npm run download:plugins`, Theia will download the `.vsix` files from the specified links and install them in the directory specified by the `theiaPluginsDir` property.


Suggested change

When using the command `npm run download:plugins`, Theia will download the `.vsix` files from the specified links and install them in the directory specified by the `theiaPluginsDir` property.

When using the Theia CLI command `theia download:plugins`, the Theia CLI will download the `.vsix` files from the specified links and install them in the directory specified by the `theiaPluginsDir` property.

sdirix · 2025-01-29T07:12:19Z

ADOPTING.md

+2. Open VSX Link: Specify a list of `id:link` mappings in the `package.json` file by the `theiaPlugins` property (e.g. `"theiaPlugins": { "vscodevim.vim": "https://open-vsx.org/api/vscodevim/vim/1.29.0/file/vscodevim.vim-1.29.0.vsix" }`). Extensions can be found on the [Open VSX Registry](https://open-vsx.org/) by searching for the extension and copying the link linked to the Download button.
+When using the command `npm run download:plugins`, Theia will download the `.vsix` files from the specified links and install them in the directory specified by the `theiaPluginsDir` property.
+
+After the extensions are downloaded, they need be packaged into the Theia Browser-Only application. To do this, run the command `npm run package:plugins` which in turn will run the [script](examples/browser-only/prepare-plugins.js) to unpack the `.vsix` files and copy the contents to the `lib/frontend/hostedPlugin` directory. 


Suggested change

After the extensions are downloaded, they need be packaged into the Theia Browser-Only application. To do this, run the command `npm run package:plugins` which in turn will run the [script](examples/browser-only/prepare-plugins.js) to unpack the `.vsix` files and copy the contents to the `lib/frontend/hostedPlugin` directory.

After the extensions are downloaded, they need to be packaged into the Theia Browser-Only application. To do this, run the command `npm run package:plugins` which in turn will run the [script](examples/browser-only/prepare-plugins.js) to unpack the `.vsix` files and copy the contents to the `lib/frontend/hostedPlugin` directory.

sdirix · 2025-01-29T07:15:07Z

ADOPTING.md

+2. Open VSX Link: Specify a list of `id:link` mappings in the `package.json` file by the `theiaPlugins` property (e.g. `"theiaPlugins": { "vscodevim.vim": "https://open-vsx.org/api/vscodevim/vim/1.29.0/file/vscodevim.vim-1.29.0.vsix" }`). Extensions can be found on the [Open VSX Registry](https://open-vsx.org/) by searching for the extension and copying the link linked to the Download button.
+When using the command `npm run download:plugins`, Theia will download the `.vsix` files from the specified links and install them in the directory specified by the `theiaPluginsDir` property.
+
+After the extensions are downloaded, they need be packaged into the Theia Browser-Only application. To do this, run the command `npm run package:plugins` which in turn will run the [script](examples/browser-only/prepare-plugins.js) to unpack the `.vsix` files and copy the contents to the `lib/frontend/hostedPlugin` directory. 


The ADOPTING.md is meant for downstream consumers. They don't have access to the npm run package:plugins script in their code base.

Suggestion: Describe first what needs to be done, i.e. unpacking vsix files to which specific directory, and then link to the prepare-plugins.js script as an example on how this can be done automatically.

sdirix · 2025-01-29T07:16:13Z

ADOPTING.md

+metadata: {
+    'host': 'main', // Should always be set to 'main'
+    'model': {
+        'packagePath': '/home/user/theia/examples/browser-only/plugins/theia.helloworld-web-sample/extension', // Deprecated


What do you mean with "Deprecated"?

sdirix · 2025-01-29T07:18:31Z

ADOPTING.md

+        'stopMethod': 'deactivate', // the method to call when the extension is deactivated; typically 'deactivate' for VS Code extensions and 'stop' for Theia plugins
+        'frontendModuleName': 'theia_helloworld_web_sample', // the id specified above but with underscores instead of dots and dashes similar to iconUrl, readmeUrl, and licenseUrl
+        'frontendInitPath': 'plugin-vscode-init-fe.js', // the path to the frontend initialization script; only required for VS Code extensions
+        'backendInitPath': '/Users/user/theia/examples/browser/lib/backend/plugin-vscode-init' // the path to the backend initialization script; for Theia plugins, this path ends with 'backend-init-theia'


Is this path needed? As we don't have a backend. If yes then we must make sure that we are self-contained and don't rely on browser build artifacts.

sdirix · 2025-01-29T07:23:08Z

examples/browser-only/prepare-plugins.js

@@ -0,0 +1,42 @@
+const { promisify } = require('util');


please add a copyright

sdirix · 2025-01-29T07:24:13Z

packages/terminal/src/browser-only/terminal-frontend-only-module.ts

+import '../../src/browser/style/terminal.css';
+import 'xterm/css/xterm.css';


Suggested change

import '../../src/browser/style/terminal.css';

import 'xterm/css/xterm.css';

sdirix · 2025-01-29T08:26:00Z

packages/terminal/src/browser-only/terminal-frontend-only-module.ts

@@ -0,0 +1,27 @@
+// *****************************************************************************
+// Copyright (C) 2017 TypeFox and others.


Should be your copyright?

sdirix · 2025-01-29T08:26:29Z

packages/terminal/src/browser-only/terminal-frontend-only-contribution.ts

@@ -0,0 +1,65 @@
+// *****************************************************************************
+// Copyright (C) 2017 TypeFox and others.


your copyright?

sdirix · 2025-01-29T08:38:42Z

examples/browser-only/prepare-plugins.js

+        const targetDir = path.join('lib', 'frontend', 'hostedPlugin', pluginName);
+
+        // Ensure the target directory exists.
+        await fs.mkdir(targetDir, { recursive: true });


If the target directory already exists we should skip copying, or at least only copy missing files.

robertjndw and others added 30 commits September 28, 2024 22:22

Initial setup and experiments with OPFS

70e7153

Add missing methods and event emitter

a28a085

Remove references to browserfs and replace it with OPFS

6578c25

Improve coding style and remove logging

e928eec

Merge branch 'eclipse-theia:master' into feat/switch-to-opfs

28d70de

Update CHANGELOG.md

05e26ad

Add yarn.lock file and AsyncIterable to base tsconfig

4fa4158

Implement feedback from PR and fix directory renaming bug

263815e

Adapt eslintrc config

e24978c

Fix linting issues

8fa511b

Add empty files for the plugin-ext for browser-only

39ab4f2

Add initial implementation; still WIP

969c31b

Add browser-only terminal frontend implementation

459dbce

Remove unused pluginDirectory property from PluginLocalOptions

3a693bb

Add example mock-plugin-metadata file

d0e4de7

Add script to move plugins for static browser-only implementation

c4620b0

Update getDefaultShell method to return an empty string instead of 'n…

82d34b0

…one'

Add mock plugin metadata and update prepare-plugins script for browse…

3bf06fe

…r-only implementation

Fix lifecycle methods and activation events

42a50f4

Filter deployed plugins by specified plugin IDs in getDeployedPlugins…

fbd9da8

… method

Rename mock plugin metadata to static plugin metadata

a1bc7e8

Refactor plugin ID handling to use PluginIdentifiers for consistency

82baacd

Add documentation for Browser-Only static extension support and updat…

8fea9e5

…e naming of package.json scripts

Add TypeScript plugins and enhance ADOPTING.md

f1ea8c0

Merge branch 'master' of github.com:robertjndw/theia into feat/browse…

4853c5c

…r-only-extension-support

Update commands in ADOPTING.md and package.json to use npm

a46f214

Code cleanup

26837bf

Fix linting error

3394ee3

More linting fixes

6cacac4

Ignore metadata file for linting

42be38e

sdirix self-requested a review January 29, 2025 07:08

sdirix requested changes Jan 29, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Browser-only static extension support #14776

Browser-only static extension support #14776

robertjndw commented Jan 27, 2025 •

edited

Loading

sdirix left a comment

sdirix Jan 29, 2025

sdirix Jan 29, 2025

sdirix Jan 29, 2025

sdirix Jan 29, 2025

sdirix Jan 29, 2025 •

edited

Loading

sdirix Jan 29, 2025

sdirix Jan 29, 2025

sdirix Jan 29, 2025

sdirix Jan 29, 2025

sdirix Jan 29, 2025

	When using the command `npm run download:plugins`, Theia will download the `.vsix` files from the specified links and install them in the directory specified by the `theiaPluginsDir` property.
	When using the Theia CLI command `theia download:plugins`, the Theia CLI will download the `.vsix` files from the specified links and install them in the directory specified by the `theiaPluginsDir` property.

	After the extensions are downloaded, they need be packaged into the Theia Browser-Only application. To do this, run the command `npm run package:plugins` which in turn will run the [script](examples/browser-only/prepare-plugins.js) to unpack the `.vsix` files and copy the contents to the `lib/frontend/hostedPlugin` directory.
	After the extensions are downloaded, they need to be packaged into the Theia Browser-Only application. To do this, run the command `npm run package:plugins` which in turn will run the [script](examples/browser-only/prepare-plugins.js) to unpack the `.vsix` files and copy the contents to the `lib/frontend/hostedPlugin` directory.

		import '../../src/browser/style/terminal.css';
		import 'xterm/css/xterm.css';

		@@ -0,0 +1,27 @@
		// *****************************************************************************
		// Copyright (C) 2017 TypeFox and others.

		@@ -0,0 +1,65 @@
		// *****************************************************************************
		// Copyright (C) 2017 TypeFox and others.

Browser-only static extension support #14776

Are you sure you want to change the base?

Browser-only static extension support #14776

Conversation

robertjndw commented Jan 27, 2025 • edited Loading

What it does

How to test

Follow-ups

Breaking changes

Attribution

Review checklist

Reminder for reviewers

sdirix left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sdirix Jan 29, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

robertjndw commented Jan 27, 2025 •

edited

Loading

sdirix Jan 29, 2025 •

edited

Loading