Shared-Reality-Lab · jaydeepsingh25 · Sep 14, 2025 · Sep 8, 2025 · Sep 14, 2025
diff --git a/FEATURES.md b/FEATURES.md
@@ -0,0 +1,101 @@
+# IMAGE Browser Extension Features
+
+The IMAGE browser extension provides accessible interpretations of graphics, charts, and maps for users with visual impairments. The goal is to provide people who are blind or have low vision with a new and useful experience of internet graphics that goes beyond automatically generating alt tags.
+
+## Core Features
+
+### Graphics Interpretation
+
+- **Rich Interpretations**: Provides detailed, context-aware interpretations of graphics that go far beyond basic alt text
+- **Support for Both Web and Local Graphics**:
+  - **Web Graphics**: Right-click on any graphic on the internet to access IMAGE processing options via the context menu
+  - **Local Graphics**: Use the IMAGE Launchpad (Alt+I) and select "Use IMAGE on a graphic on my computer" to interpret graphics saved locally on your machine
+- **Multiple Rendering Types**: Receive different types of interpretations tailored to the graphic content:
+  - Detailed text descriptions
+  - Audio renderings with spatial cues
+  - Tactile SVG graphics for haptic devices
+  - Specialized renderings based on graphic type and content
+
+### Maps Support
+
+- **Interactive Map Processing**: Specialized handling for maps found on web pages
+- **Spatial Information**: Convey spatial relationships and geographic information in accessible formats
+- **Context-Aware Descriptions**: Descriptions that focus on relevant geographic features and relationships
+
+### Charts Support
+
+- **Data Visualization Interpretation**: Make charts and graphs accessible through multiple modalities
+- **Trend Identification**: Highlight important trends, patterns, and outliers in data visualizations
+- **Highcharts Integration**: Special support for Highcharts-based visualizations
+- **Data Sonification**: Convert chart data to audio patterns where appropriate
+
+### Tactile Graphics
+
+- **Tactile Authoring Tool**: Load graphics into a Tactile Authoring Tool for creating tactile representations
+- **Monarch Integration**: Send graphics to Monarch haptic devices for tactile feedback
+- **SVG Rendering**: Generate SVG representations of graphics optimized for tactile output
+- **Tactile Simplification**: Automatically simplify complex graphics for effective tactile exploration
+
+## Keyboard Shortcuts
+
+The IMAGE browser extension provides several keyboard shortcuts for quick access to its features:
+
+| Shortcut | Function | Description |
+|----------|----------|-------------|
+| `Alt+I` | Launch IMAGE Launchpad | Opens the IMAGE Launchpad interface with various options for processing graphics |
+| `Ctrl+B` | Process Focused Image | When an image is focused (selected), this shortcut processes it with IMAGE |
+
+## Accessibility Features
+
+- **Screen Reader Integration**: Designed to work seamlessly with screen readers
+- **Invisible Buttons**: Toggle-able invisible buttons that are only visible to screen readers
+- **Keyboard Navigation**: Full keyboard accessibility for all features
+- **Tabindex Management**: Automatically adds tabindex to images without it for improved keyboard navigation
+- **Progressive Enhancement**: Works alongside existing accessibility features rather than replacing them
+
+## Customization Options
+
+### Interpretation Preferences
+
+- **Audio Renderings**: Toggle audio interpretations on/off
+- **Text Renderings**: Toggle text descriptions on/off
+- **Rendering Preferences**: Customize which types of renderings are prioritized
+
+### Language Settings
+
+- **Multiple Languages**: Support for English and French
+- **Automatic Language Detection**: Option to use the browser's language
+
+### Server Configuration
+
+- **McGill Server**: Use the default McGill University IMAGE server
+- **Custom Server**: Specify a custom server URL for processing
+
+### Haptic Device Settings
+
+- **Monarch Configuration**: Settings for Monarch haptic devices:
+  - Title
+  - Channel ID
+  - Secret key
+  - Encryption key
+
+## Developer Features
+
+- **Developer Mode**: Toggle additional developer options
+- **Preprocessing Only**: Request only the preprocessing step
+- **Request Only**: Download the raw request JSON
+- **Debug Information**: Access to additional debugging information
+
+## User Experience
+
+- **Progress Indication**: Visual feedback during graphic processing
+- **Error Handling**: Comprehensive error messages and recovery options
+- **First Launch Experience**: Guided introduction for new users
+- **Feedback System**: Built-in feedback mechanism for reporting issues or suggestions
+
+## Technical Capabilities
+
+- **Image Compression**: Automatic compression of large graphics (>4MB)
+- **Context Extraction**: Analysis of surrounding content to improve interpretation
+- **Local File Support**: Process graphics from local files (file:// URLs)
+- **Data URL Support**: Process graphics encoded as data URLs
diff --git a/README.md b/README.md
@@ -1,70 +1,150 @@
 # IMAGE-browser 
-IMAGE browser extensions &amp; client-side code
-
-To install the latest development version of the extension you'll need to follow [these instructions](https://github.com/Shared-Reality-Lab/IMAGE-browser/wiki/Installing-Development-Extensions), using this [packaged extension file](https://nightly.link/Shared-Reality-Lab/IMAGE-browser/workflows/typescript-check/main/extension.zip).
+IMAGE browser extensions & client-side code
 
 ## IMAGE project information
 Please see https://image.a11y.mcgill.ca for general information about the project.
 
 If you wish to contribute to the project, the following wiki page is a good starting point, including for those on the IMAGE project team:
 https://github.com/Shared-Reality-Lab/IMAGE-server/wiki
 
-## Set Up for Chrome Extension
+## Extension Features
 
-Clone this repository. Note that the schemas are a submodule, so you need to either get them in the initial clone, e.g.,
-```
+For a comprehensive list of features and capabilities provided by the IMAGE browser extension, please see the [FEATURES.md](FEATURES.md) file. This document details all the functionality available in the extension, including:
+
+- Graphics, charts, and maps interpretation
+- Keyboard shortcuts
+- Accessibility features
+- Customization options
+- Developer features
+- And more
+
+## Development Setup
+
+### Prerequisites
+- [Node.js](https://nodejs.org/) (latest LTS version recommended)
+- [npm](https://www.npmjs.com/) (v7 or later)
+- Git
+
+### Clone the Repository
+Clone this repository with submodules (for schemas):
+```bash
 git clone --recurse-submodules [email protected]:Shared-Reality-Lab/IMAGE-browser.git
 ```
 
-or else get them after you've done the initial clone (while in the root of the cloned repo on your local machine):
-```
+If you've already cloned the repository without submodules, you can initialize and update them:
+```bash
 git submodule init
 git submodule update
 ```
 
-Install the dependencies using NPM.
-Ensure you have npm v7 or later installed, ideally by using [nvm](https://github.com/nvm-sh/nvm)!
+### Install Dependencies
+Navigate to the project directory and install dependencies:
 ```bash
-npm i
+cd IMAGE-browser
+npm install
 ```
-This includes the [web-ext](https://github.com/mozilla/web-ext) tool for running the tool in Firefox and Chrome/Chromium and building for firefox.
-As the project is written in Typescript, we do use Parcel to resolve packages and convert to JavaScript first.
-An automatically updating parcel build can be triggered with
+
+## Building the Extension
+
+### Development/Test Build
+To create a development build with debugging enabled:
 ```bash
-npm start
-```
-from the command line.
-To use web-ext to start, from the root of the project run
-```
-npx web-ext run -s build
+npm run pack:test
 ```
-to open it in Firefox.
+This will:
+- Set the NODE_ENV to "development"
+- Use the environment variables from `.env.development`
+- Output the build to the `build-chrome` directory
+- Add "(test)" suffix to the extension name
 
-A version ready to be bundled for distribution as a browser extension can be created using the
-```
-npm run pack
-```
-script.
-A version that directly creates an *unsigned* Firefox package can be called with
-```
-npm run build:ff
+### Production Build
+To create a production-ready build:
+```bash
+npm run pack:prod
 ```
+This will:
+- Set the NODE_ENV to "production"
+- Use the environment variables from `.env.production`
+- Output the build to the `build-chrome` directory
+- Use the standard extension name without any suffix
 
-A signed package can be made with Chromium by running
-```
-npm run build:chromium
-```
-where the environment variable `PATH_TO_CHROME_KEY` points to the extension's key.
+#### Submitting Extension Build to Chrome Web Store
+When preparing to submit the extension to the Chrome Web Store, follow these steps:
 
-It should also be possible to just load a zipped version of the `build` directory generated by `npm run pack` in any browser supporting WebExtensions, but tools might do other things too.
+1. Create a production build using the instructions above.
+
+2. Update the extension version in the generated `build-chrome/manifest.json` file to match the corresponding git tag of the codebase. You can view available tags [here](https://github.com/Shared-Reality-Lab/IMAGE-browser/tags).
+
+3. Create a ZIP file of the `build-chrome` directory.
+
+4. Submit the ZIP file to the Chrome Web Store Developer Dashboard following Google's submission guidelines.
+
+This versioning approach ensures consistency between the published extension and the corresponding source code in the repository.
+
+## Build System
+
+### Webpack Configuration
+The IMAGE browser extension uses webpack as its bundling tool. The webpack configuration is defined in `webpack.config.ts` and includes:
+
+- **Entry Points**: Multiple entry points for various parts of the extension (background scripts, content scripts, UI pages, etc.)
+- **Output Configuration**: Compiled files are output to the `build-chrome` directory
+- **TypeScript Processing**: TypeScript files are processed using ts-loader
+- **CSS Processing**: CSS files are processed using style-loader and css-loader
+- **File Copying**: HTML, PNG, JSON, and MP3 files are copied from the src directory to the build directory using CopyWebpackPlugin
+- **Environment Variables**: Environment variables from `.env.development` or `.env.production` are injected into the build using webpack.DefinePlugin
+
+### Environment Variables
+The build process uses different environment variables based on the build type:
+
+- **Development Build**: Uses variables from `.env.development`, which includes `SUFFIX_TEXT = " (test)"` to append "(test)" to the extension name
+- **Production Build**: Uses variables from `.env.production`
+
+### Manifest Modification
+During the production build process, the `manifest.json` file is modified to set the extension name to `__MSG_extensionName__`, which allows for localization of the extension name.
+
+## Installing and Using the Extension
+
+### Production Build
+The official production build of the IMAGE extension is available on the Chrome Web Store:
+- **URL**: [https://chromewebstore.google.com/detail/image-extension/iimmeciildnckfmnpbhglofiahllkhop](https://chromewebstore.google.com/detail/image-extension/iimmeciildnckfmnpbhglofiahllkhop)
+- **Description**: This is the stable, production-ready version of the extension recommended for most users.
+- **Installation**: Click the "Add to Chrome" button on the Chrome Web Store page.
+
+### Test Build
+The test build is a pre-release version with newer features that are still being tested:
+- **URL**: [https://chromewebstore.google.com/detail/image-extension-test/emmcfbcpilagikejimnbilgoihgjfdln](https://chromewebstore.google.com/detail/image-extension-test/emmcfbcpilagikejimnbilgoihgjfdln)
+- **Description**: This version includes features that are being tested before inclusion in the production build.
+- **Installation**: Click the "Add to Chrome" button on the Chrome Web Store page.
+
+### Nightly Build
+The nightly build contains the latest changes from the main branch:
+- **URL**: [https://nightly.link/Shared-Reality-Lab/IMAGE-browser/workflows/typescript-check/main/extension.zip](https://nightly.link/Shared-Reality-Lab/IMAGE-browser/workflows/typescript-check/main/extension.zip)
+- **Description**: This build is automatically generated from the latest code in the main branch and may contain experimental features or bugs.
+- **Installation**: 
+  1. Download the ZIP file from the URL above
+  2. Extract the ZIP file to a folder on your computer
+  3. Follow the "Local Build" installation instructions below
+
+
+**Warning**: Installing different versions of the extension (production and test) in a browser at the same time can cause unexpected issues, and the extension may not function properly. It is recommended to only have one version of the extension installed at any given time.
+
+### Local Build
+To use a locally built version of the extension:
+
+#### Chrome/Chromium
+1. Build the extension using either `npm run pack:test` or `npm run pack:prod` as described in the "Building the Extension" section
+2. Open Chrome/Chromium
+3. Navigate to `chrome://extensions/`
+4. Enable "Developer mode" (toggle in the top-right corner)
+5. Click "Load unpacked"
+6. Select the `build-chrome` directory from your project
 
 ## EARLY BETA: Safari Extension (desktop and iOS)
 The Chrome extension can be converted into an extension that will run in desktop Safari, or (with a separate conversion) in Safari on iOS.
 It is not currently possible to generate a signed extension that is installable on iOS, or deployable to the App Store, due to restrictions on using background workers on iPhones.
 However, it is possible to run and test the iOS extension by installing it directly onto an iPhone via Xcode.
 Details are available in [Safari.md](Safari.md).
 
-
 ## License
 
 IMAGE project components (e.g., IMAGE browser extension and IMAGE Services), henceforth "Our Software" are licensed under GNU GPL3 (https://www.gnu.org/licenses/gpl-3.0.en.html) or AGPLv3 terms (https://www.gnu.org/licenses/agpl-3.0.txt) or later, as indicated in specific repositories or files on the project github located at https://github.com/Shared-Reality-Lab.