You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+13-11Lines changed: 13 additions & 11 deletions
Original file line number
Diff line number
Diff line change
@@ -45,14 +45,14 @@ Besides supporting WebGPU, this project also provides the harness for other kind
45
45
```shell
46
46
gem install jekyll-remote-theme
47
47
```
48
-
6. Install [Chrome Canary](https://www.google.com/chrome/canary/). It is a developer version of Chrome that enables the use of WebGPU.
48
+
6. Install [Chrome](https://www.google.com/chrome/) with version at least 113. WebGPU has shipped to Chrome in version 113.
49
49
50
50
We can verify the success installation by trying out `emcc`, `jekyll` and `wasm-pack`in terminal respectively.
51
51
52
52
3. Import, optimize and build the LLM model:
53
53
* Get Model Weight
54
54
55
-
Currently we support LLaMA and Vicuna.
55
+
Currently we support LLaMA and Vicuna and RedPajama. To get the Vicuna model weights, follow the instructions below:
56
56
57
57
1. Get the original LLaMA weights in the huggingface format by following the instructions [here](https://huggingface.co/docs/transformers/main/model_doc/llama).
58
58
2. Use instructions [here](https://github.com/lm-sys/FastChat#vicuna-weights) to get vicuna weights
@@ -62,17 +62,19 @@ Besides supporting WebGPU, this project also provides the harness for other kind
By default `build.sh` takes `vicuna-7b-v1` as model name
74
-
75
-
Note: build.py can be run on MacOS with 32GB memory and other OS with at least 50GB CPU memory. We are currently optimizing the memory usage to enable more people to try out locally.
77
+
Note: build.py for Vicuna-v1-7B requires 16GB of memory for Mac, and about 30GB CPU memory for other OS. We are continuously optimizing for reducing build memory requirement to enable more people to try out locally.
76
78
77
79
4. Deploy the model on web with WebGPU runtime
78
80
@@ -86,11 +88,11 @@ Besides supporting WebGPU, this project also provides the harness for other kind
86
88
./scripts/local_deploy_site.sh
87
89
```
88
90
89
-
With the site set up, you can go to `localhost:8888/web-llm/`in Chrome Canary to try out the demo on your local machine. Remember: you will need 6.4G GPU memory to run the demo. Don’t forget to use
91
+
With the site set up, you can go to `localhost:8888/web-llm/`in Chrome to try out the demo on your local machine. You will need around 6GB GPU memory to run the Vicuna model, or 3GB GPU memory to run the RedPajama model. You can use
0 commit comments