Skip to content

Commit 13d51df

Browse files
authored
Update instructions in README (#110)
1 parent ac8123a commit 13d51df

File tree

2 files changed

+14
-12
lines changed

2 files changed

+14
-12
lines changed

README.md

Lines changed: 13 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -45,14 +45,14 @@ Besides supporting WebGPU, this project also provides the harness for other kind
4545
```shell
4646
gem install jekyll-remote-theme
4747
```
48-
6. Install [Chrome Canary](https://www.google.com/chrome/canary/). It is a developer version of Chrome that enables the use of WebGPU.
48+
6. Install [Chrome](https://www.google.com/chrome/) with version at least 113. WebGPU has shipped to Chrome in version 113.
4949

5050
We can verify the success installation by trying out `emcc`, `jekyll` and `wasm-pack` in terminal respectively.
5151

5252
3. Import, optimize and build the LLM model:
5353
* Get Model Weight
5454

55-
Currently we support LLaMA and Vicuna.
55+
Currently we support LLaMA and Vicuna and RedPajama. To get the Vicuna model weights, follow the instructions below:
5656

5757
1. Get the original LLaMA weights in the huggingface format by following the instructions [here](https://huggingface.co/docs/transformers/main/model_doc/llama).
5858
2. Use instructions [here](https://github.com/lm-sys/FastChat#vicuna-weights) to get vicuna weights
@@ -62,17 +62,19 @@ Besides supporting WebGPU, this project also provides the harness for other kind
6262
ln -s your_model_path mlc-llm/dist/models/model_name
6363
6464
# For example:
65-
# ln -s path/to/vicuna-7b-v1 mlc-llm/dist/models/vicuna-7b-v1
65+
# ln -s path/to/vicuna-v1-7b mlc-llm/dist/models/vicuna-v1-7b
6666
```
6767

6868
If you want to use your own mlc-llm branch, set `MLC_LLM_HOME` to that path and link weights under `$MLC_LLM_HOME/dist/models/model_name`
69-
* Optimize and build model to webgpu backend and export the executable to disk in the WebAssembly file format.
69+
70+
You can download the RedPajama weights from the HuggingFace repo [here](https://huggingface.co/togethercomputer/RedPajama-INCITE-Chat-3B-v1).
71+
72+
* Optimize and build the models to WebGPU backend and export the executable to disk in the WebAssembly file format.
7073
```shell
71-
./build.sh --quantization q4f32_0
74+
./build.sh --model=vicuna-v1-7b --quantization q4f32_0
75+
./build.sh --model=RedPajama-INCITE-Chat-3B-v1 --quantization q4f32_0
7276
```
73-
By default `build.sh` takes `vicuna-7b-v1` as model name
74-
75-
Note: build.py can be run on MacOS with 32GB memory and other OS with at least 50GB CPU memory. We are currently optimizing the memory usage to enable more people to try out locally.
77+
Note: build.py for Vicuna-v1-7B requires 16GB of memory for Mac, and about 30GB CPU memory for other OS. We are continuously optimizing for reducing build memory requirement to enable more people to try out locally.
7678

7779
4. Deploy the model on web with WebGPU runtime
7880

@@ -86,11 +88,11 @@ Besides supporting WebGPU, this project also provides the harness for other kind
8688
./scripts/local_deploy_site.sh
8789
```
8890

89-
With the site set up, you can go to `localhost:8888/web-llm/` in Chrome Canary to try out the demo on your local machine. Remember: you will need 6.4G GPU memory to run the demo. Don’t forget to use
91+
With the site set up, you can go to `localhost:8888/web-llm/` in Chrome to try out the demo on your local machine. You will need around 6GB GPU memory to run the Vicuna model, or 3GB GPU memory to run the RedPajama model. You can use
9092
```shell
91-
/Applications/Google\ Chrome\ Canary.app/Contents/MacOS/Google\ Chrome\ Canary --enable-dawn-features=disable_robustness
93+
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --enable-dawn-features=disable_robustness
9294
```
93-
to launch Chrome Canary to turn off the robustness check from Chrome.
95+
to launch Chrome from the command line to turn off the robustness check from Chrome and enable better performance.
9496

9597

9698
## How

0 commit comments

Comments
 (0)