-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathindex.html
More file actions
executable file
·501 lines (498 loc) · 33.8 KB
/
index.html
File metadata and controls
executable file
·501 lines (498 loc) · 33.8 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="Cerebras Research">
<meta name="author" content="Cerebras Research">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<link href="img/favicon.ico" rel="icon" type="image/x-icon" />
<link rel="stylesheet" href="css/bootstrap.min.css">
<!-- <link rel="stylesheet" href="css/font-awesome.min.css"> -->
<link rel="stylesheet" href="css/style.css">
<title>Cerebras Research</title>
</head>
<body class="custom-body">
<div class="section" id="about-me">
<div class='section-container'>
<!-- <div class="portrait-container"> -->
<!-- <img class="portrait" src="img/logo_black.png"> -->
<!-- <div class="title">Cerebras Research</div> -->
<!-- </div> -->
<div class="portrait-container">
<!-- <img class="portrait" src="img/logo_black.png"> -->
<div class="title">Cerebras Research</div>
</div>
<!-- <div class='bio'> -->
<!-- <p>At Cerebras Research we research XXX.</p> -->
<!-- </div> -->
</div>
</div>
<!-- Research -->
<div class="section" id="projects">
<div class='section-container'>
<div class="section-title projects-section-title" id="projects">Publications</div>
<div class="projects">
<div class="project">
<div class="project-image">
<img src="img/completep.png">
</div>
<div class="project-text">
<div class="project-title">Don't be lazy: CompleteP enables compute-efficient deep transformers</div>
<div class="project-venue">arXiv, 2025</div>
<div class="project-authors">Nolan Dey*, Bin Claire Zhang*, Lorenzo Noci, Mufan Li, Blake Bordelon, Shane Bergsma, Cengiz Pehlevan, Boris Hanin, Joel Hestness</div>
<div class="project-links"><a href='https://arxiv.org/abs/2505.01618' target="_blank">[arXiv]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/d2z.png">
</div>
<div class="project-text">
<div class="project-title">Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs</div>
<div class="project-venue">ICLR, 2025</div>
<div class="project-authors">Shane Bergsma, Nolan Dey, Gurpreet Gosal, Gavia Gray, Daria Soboleva, Joel Hestness</div>
<div class="project-links"><a href='https://openreview.net/forum?id=hrOlBgHsMI' target="_blank">[OpenReview]</a> <a href='https://www.arxiv.org/abs/2502.15938' target="_blank">[arXiv]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/mup_guide.jpg">
<!-- <img src="img/mup.png"> -->
</div>
<div class="project-text">
<div class="project-title">The practitioner's guide to the maximal update parameterization</div>
<div class="project-venue">Blog & open-source code, 2024</div>
<div class="project-authors">Nolan Dey, Quentin Anthony, Joel Hestness</div>
<div class="project-links"><a href='https://cerebras.ai/blog/the-practitioners-guide-to-the-maximal-update-parameterization' target="_blank">[Cerebras Blog]</a> <a href='https://blog.eleuther.ai/mutransfer/' target="_blank">[Eleuther AI Blog]</a> <a href='https://github.com/EleutherAI/nanoGPT-mup' target="_blank">[nanoGPT-mup Code]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/empirical_upper_bounds.png">
</div>
<div class="project-text">
<div class="project-title">Empirical Upper Bounds for Unstructured Sparsity in Compute-Efficient Language Modeling</div>
<div class="project-venue">Machine Learning and Compression NeurIPS Workshop, 2024</div>
<div class="project-authors">Esha Singh, Shane Bergsma, Nolan Dey, Joel Hestness, Gavia Gray</div>
<div class="project-links"><a href="https://openreview.net/forum?id=qOnKSqiGtR">[OpenReview]</a> <a href='https://neurips.cc/virtual/2024/98189' target="_blank">[Poster]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/normalization_layer_gns.png">
</div>
<div class="project-text">
<div class="project-title">Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers</div>
<div class="project-venue">NeurIPS, 2024</div>
<div class="project-authors">Gavia Gray, Aman Tiwari, Shane Bergsma, Joel Hestness</div>
<div class="project-links"><a href='https://arxiv.org/abs/2411.00999' target="_blank">[arXiv]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/supar.png">
</div>
<div class="project-text">
<div class="project-title">Sparse maximal update parameterization: A holistic approach to sparse training dynamics</div>
<div class="project-venue">NeurIPS, 2024</div>
<div class="project-authors">Nolan Dey, Shane Bergsma, Joel Hestness</div>
<div class="project-links"><a href='https://arxiv.org/abs/2405.15743' target="_blank">[arXiv]</a> <a href='https://github.com/EleutherAI/nanoGPT-mup/tree/supar' target="_blank">[Code]</a> <a href='https://neurips.cc/virtual/2024/poster/95363' target="_blank">[NeurIPS Poster]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/self_data_distillation.png">
</div>
<div class="project-text">
<div class="project-title">Self-Data Distillation for Recovering Quality in Pruned Large Language Models</div>
<div class="project-venue">arXiv, 2024</div>
<div class="project-authors">Vithursan Thangarasa, Ganesh Venkatesh, Mike Lasby, Nish Sinnadurai, Sean Lie</div>
<div class="project-links"><a href='https://arxiv.org/abs/2410.09982' target="_blank">[arXiv]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/todo.png">
</div>
<div class="project-text">
<div class="project-title">Bilingual Adaptation of Monolingual Foundation Models</div>
<div class="project-venue">FM-Wild ICML Workshop, 2024</div>
<div class="project-authors">Gurpreet Gosal, Yishi Xu, Gokulakrishnan Ramakrishnan, Rituraj Joshi, Avraham Sheinin, Zhiming Chen, Biswajit Mishra, Sunil Kumar Sahu, Neha Sengupta, Natalia Vassilieva, Joel Hestness, Samujjwal Ghosh, Bokang Jia, Onkar Arun Pandit, Satheesh Katipomu, Samta Kamboj, Rahul Pal, Parvez Mullah, Soundar Balaji Doraiswamy, Karim Chami, Preslav Nakov</div>
<div class="project-links"><a href='https://openreview.net/forum?id=XfA4HYYGLz' target="_blank">[OpenReview]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/mediswift.png">
</div>
<div class="project-text">
<div class="project-title">MediSwift: Efficient Sparse Pre-trained Biomedical Language Models</div>
<div class="project-venue">ACL, 2024</div>
<div class="project-authors">Vithursan Thangarasa, Mahmoud Salem, Shreyas Saxena, Kevin Leong, Joel Hestness, Sean Lie</div>
<div class="project-links"><a href='https://arxiv.org/abs/2403.00952' target="_blank">[arXiv]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/breaking_md_timescale.png">
</div>
<div class="project-text">
<div class="project-title">Breaking the Molecular Dynamics Timescale Barrier Using a Wafer-Scale System</div>
<div class="project-venue">SC, 2024</div>
<div class="project-authors">Kylee Santos, Stan Moore, Tomas Oppelstrup, Amirali Sharifian, Ilya Sharapov, Aidan Thompson, Delyan Z Kalchev, Danny Perez, Robert Schreiber, Scott Pakin, Edgar A Leon, James H Laros III, Michael James, Sivasankaran Rajamanickam</div>
<div class="project-links"><a href='https://arxiv.org/abs/2405.07898' target="_blank">[arXiv]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/high_sparsity_llama.png">
</div>
<div class="project-text">
<div class="project-title">Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment</div>
<div class="project-venue">arXiv, 2023</div>
<div class="project-authors">Abhinav Agarwalla, Abhay Gupta, Alexandre Marques, Shubhra Pandit, Michael Goin, Eldar Kurtic, Kevin Leong, Tuan Nguyen, Mahmoud Salem, Dan Alistarh, Sean Lie, Mark Kurtz</div>
<div class="project-links"><a href='https://arxiv.org/abs/2405.03594' target="_blank">[arXiv]</a> <a href="https://www.cerebras.ai/blog/introducing-sparse-llama-70-smaller-3x-faster-full-accuracy">[Blog]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/per_example_gns.png">
</div>
<div class="project-text">
<div class="project-title">Efficient and Approximate Per-Example Gradient Norms for Gradient Noise Scale</div>
<div class="project-venue">WANT NeurIPS Workshop, 2023</div>
<div class="project-authors">Gavia Gray, Anshul Samar, Joel Hestness</div>
<div class="project-links"><a href='https://openreview.net/forum?id=xINTMAvPQA' target="_blank">[OpenReview]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/todo.png">
</div>
<div class="project-text">
<div class="project-title">Efficient Algorithms for Monte Carlo Particle Transport on AI Accelerator Hardware</div>
<div class="project-venue">Computer Physics Communications, 2024</div>
<div class="project-authors">John Tramm, Bryce Allen, Kazutomo Yoshii, Andrew Siegel, Leighton Wilson</div>
<div class="project-links"><a href='https://arxiv.org/abs/2311.01739' target="_blank">[arXiv]</a> <a href='https://www.sciencedirect.com/science/article/abs/pii/S0010465523004174' target="_blank">[ScienceDirect]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/alibi_pi.png">
</div>
<div class="project-text">
<div class="project-title">Position Interpolation Improves ALiBi Extrapolation</div>
<div class="project-venue">Technical Report, 2023</div>
<div class="project-authors">Faisal Al-Khateeb, Nolan Dey, Daria Soboleva, Joel Hestness</div>
<div class="project-links"><a href='https://arxiv.org/abs/2310.13017' target="_blank">[arXiv]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/scaling_memory_wall.png">
</div>
<div class="project-text">
<div class="project-title">Scaling the “Memory Wall” for Multi-Dimensional Seismic Processing with Algebraic Compression on Cerebras CS-2 Systems</div>
<div class="project-venue">SC, 2023</div>
<div class="project-authors">Hatem Ltaief, Yuxi Hong, Leighton Wilson, Mathias Jacquelin, Matteo Ravasi, David Elliot Keyes</div>
<div class="project-links"><a href='https://dl.acm.org/doi/10.1145/3581784.3627042' target="_blank">[Paper]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/btlm.jpg">
</div>
<div class="project-text">
<div class="project-title">BTLM-3B-8K: 7B Performance in a 3 Billion Parameter Model</div>
<div class="project-venue">Efficient Natural Language and Speech Processing NeurIPS Workshop, 2023</div>
<div class="project-authors">Nolan Dey*, Daria Soboleva*, Faisal Al-Khateeb, Ribhu Pathria, Hemant Khachane, Shaheer Muhammad, Zhiming (Charles) Chen, Bowen Yang, Siyun Li, Abhay Gupta, Shreyas Saxena, Robert Myers, Jacob Robert Steeves, Marvin Tom, Joel Hestness</div>
<div class="project-links"><a href='https://arxiv.org/abs/2309.11568' target="_blank">[arXiv]</a> <a href='https://neurips2023-enlsp.github.io/papers/paper_45.pdf' target="_blank">[Workshop Paper]</a> <a href='https://www.cerebras.net/blog/btlm-3b-8k-7b-performance-in-a-3-billion-parameter-model/' target="_blank">[Blog]</a> <a href='https://huggingface.co/cerebras/btlm-3b-8k-base' target="_blank">[Hugging Face]</a> <a href="img/hf_btlm_230818_leaderboard_all.jpg" target="_blank" type="application/pdf">[1.08M downloads and 10th most popular text generation model in first month]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/slimpj.jpg">
</div>
<div class="project-text">
<div class="project-title">SlimPajama: A 627B token cleaned and deduplicated version of RedPajama</div>
<div class="project-venue">Open-Source Dataset Release, 2023</div>
<div class="project-authors">Daria Soboleva*, Faisal Al-Khateeb*, Robert Myers, Jacob Robert Steeves, Joel Hestness, Nolan Dey</div>
<div class="project-links"><a href='https://www.cerebras.net/blog/slimpajama-a-627b-token-cleaned-and-deduplicated-version-of-redpajama' target="_blank">[Blog]</a> <a href='https://huggingface.co/datasets/cerebras/SlimPajama-627B' target="_blank">[Hugging Face]</a> <a href='https://github.com/Cerebras/modelzoo/tree/main/modelzoo/transformers/data_processing/slimpajama' target="_blank">[Data processing code]</a> <a href="img/hf_slimpj_230824.jpg" target="_blank" type="application/pdf">[693k downloads and 3rd most popular text dataset in first month]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/jais.png">
</div>
<div class="project-text">
<div class="project-title">Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models</div>
<div class="project-venue">arXiv, 2023</div>
<div class="project-authors">Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Satheesh Katipomu, Haonan Li, Fajri Koto, William Marshall, Gurpreet Gosal, Cynthia Liu, Zhiming Chen, Osama Mohammed Afzal, Samta Kamboj, Onkar Pandit, Rahul Pal, Lalit Pradhan, Zain Muhammad Mujahid, Massa Baali, Xudong Han, Sondos Mahmoud Bsharat, Alham Fikri Aji, Zhiqiang Shen, Zhengzhong Liu, Natalia Vassilieva, Joel Hestness, Andy Hock, Andrew Feldman, Jonathan Lee, Andrew Jackson, Hector Xuguang Ren, Preslav Nakov, Timothy Baldwin, Eric Xing</div>
<div class="project-links"><a href='https://arxiv.org/abs/2308.16149' target="_blank">[arXiv]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/cerebras_arch_deep_dive.png">
</div>
<div class="project-text">
<div class="project-title">Cerebras Architecture Deep Dive: First Look Inside the Hardware/Software Co-Design for Deep Learning</div>
<div class="project-venue">Hot Chips Theme Article, 2023</div>
<div class="project-authors">Sean Lie</div>
<div class="project-links"><a href='https://8968533.fs1.hubspotusercontent-na1.net/hubfs/8968533/IEEE%20Micro%202023-03%20Hot%20Chips%2034%20Cerebras%20Architecture%20Deep%20Dive.pdf' target="_blank">[Article]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/cbgpt.png">
</div>
<div class="project-text">
<div class="project-title">Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster</div>
<div class="project-venue">Technical Report, 2023</div>
<div class="project-authors">Nolan Dey, Gurpreet Gosal, Zhiming (Charles) Chen, Hemant Khachane, William Marshall, Ribhu Pathria, Marvin Tom, Joel Hestness</div>
<div class="project-links"><a href='https://arxiv.org/abs/2304.03208' target="_blank">[arXiv]</a> <a href='https://huggingface.co/cerebras' target="_blank">[Hugging Face]</a> <a href='https://www.cerebras.net/blog/cerebras-gpt-a-family-of-open-compute-efficient-large-language-models/' target="_blank">[Blog]</a> <a href='https://www.youtube.com/watch?v=QmmNgiFuIog&t=1s' target="_blank">[Podcast]</a> </div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/sift.png">
</div>
<div class="project-text">
<div class="project-title">Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency</div>
<div class="project-venue">ICML, 2024</div>
<div class="project-authors">Vithursan Thangarasa, Shreyas Saxena, Abhay Gupta, Sean Lie</div>
<div class="project-links"><a href='https://arxiv.org/abs/2303.11525' target="_blank">[arXiv]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/spdf.png">
</div>
<div class="project-text">
<div class="project-title">SPDF: Sparse Pre-training and Dense Fine-tuning for Large Language Models</div>
<div class="project-venue">UAI, 2023</div>
<div class="project-authors">Vithursan Thangarasa, Abhay Gupta, William Marshall, Tianda Li, Kevin Leong, Dennis DeCoste, Sean Lie, Shreyas Saxena</div>
<div class="project-links"><a href='https://arxiv.org/abs/2303.10464' target="_blank">[arXiv]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/wafer_fft.png">
</div>
<div class="project-text">
<div class="project-title">Wafer-Scale Fast Fourier Transforms</div>
<div class="project-venue">ICS, 2023</div>
<div class="project-authors">Marcelo Orenes-Vera, Ilya Sharapov, Robert Schreiber, Mathias Jacquelin, Philippe Vandermersch, Sharan Chetlur</div>
<div class="project-links"><a href='https://arxiv.org/abs/2209.15040' target="_blank">[arXiv]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/todo.png">
</div>
<div class="project-text">
<div class="project-title">GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics</div>
<div class="project-venue">The International Journal of High Performance Computing Applications, 2022</div>
<div class="project-authors">Maxim Zvyagin, Alexander Brace, Kyle Hippe, Yuntian Deng, Bin Zhang, Cindy Orozco Bohorquez, Austin Clyde, Bharat Kale, Danilo Perez-Rivera, Heng Ma, Carla M. Mann, Michael Irvin, J. Gregory Pauloski, Logan Ward, Valerie Hayot-Sasson, Murali Emani, Sam Foreman, Zhen Xie, Diangen Lin, Maulik Shukla, Weili Nie, Josh Romero, Christian Dallago, Arash Vahdat, Chaowei Xiao, Thomas Gibbs, Ian Foster, View ORCID ProfileJames J. Davis, Michael E. Papka, Thomas Brettin, Rick Stevens, Anima Anandkumar, Venkatram Vishwanath, Arvind Ramanathan</div>
<div class="project-links"><a href='https://www.biorxiv.org/content/10.1101/2022.10.10.511571v2' target="_blank">[bioRXiv]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/disruptive_changes.png">
</div>
<div class="project-text">
<div class="project-title">Disruptive Changes in Field Equation Modeling: A Simple Interface for Wafer Scale Engines</div>
<div class="project-venue">arXiv, 2022</div>
<div class="project-authors">Mino Woo, Terry Jordan, Robert Schreiber, Ilya Sharapov, Shaheer Muhammad, Abhishek Koneru, Michael James, Dirk Van Essendelft</div>
<div class="project-links"><a href='https://arxiv.org/abs/2209.13768' target="_blank">[arXiv]</a> <a href='https://www.cerebras.ai/press-release/cerebras-systems-and-national-energy-technology-laboratory-set-new-milestones-for-high-performance-energy-efficient-field-equation-modeling-using-simple-python-interface' target="_blank">[Press release]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/tensorflow_as_a_dsl.png">
</div>
<div class="project-text">
<div class="project-title">TensorFlow as a DSL for stencil-based computation on the Cerebras Wafer Scale Engine</div>
<div class="project-venue">arXiv, 2022</div>
<div class="project-authors">Nick Brown, Brandon Echols, Justs Zarins, Tobias Grosser</div>
<div class="project-links"><a href='https://arxiv.org/abs/2210.04795' target="_blank">[arXiv]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/revbifpn.png">
</div>
<div class="project-text">
<div class="project-title">RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network</div>
<div class="project-venue">MLSys, 2023</div>
<div class="project-authors">Vitaliy Chiley, Vithursan Thangarasa, Abhay Gupta, Anshul Samar, Joel Hestness, Dennis DeCoste</div>
<div class="project-links"><a href='https://arxiv.org/abs/2206.14098' target="_blank">[arXiv]</a> <a href='https://mlsys.org/media/mlsys-2023/Slides/2474.pdf' target="_blank">[Slides]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/massively_scalable_stencil.png">
</div>
<div class="project-text">
<div class="project-title">Massively scalable stencil algorithm</div>
<div class="project-venue">arXiv, 2022</div>
<div class="project-authors">Mathias Jacquelin, Mauricio Araya-Polo, Jie Meng</div>
<div class="project-links"><a href='https://arxiv.org/abs/2204.03775' target="_blank">[arXiv]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/epigenomic_lm.png">
</div>
<div class="project-text">
<div class="project-title">Epigenomic language models powered by Cerebras</div>
<div class="project-venue">ICLR MLDD Workshop, 2023</div>
<div class="project-authors">Meredith V. Trotter, Cuong Q. Nguyen, Stephen Young, Rob T. Woodruff, Kim M. Branson</div>
<div class="project-links"><a href='https://arxiv.org/abs/2112.07571' target="_blank">[arXiv]</a> <a href='https://openreview.net/forum?id=MaRBIXXk0M' target="_blank">[OpenReview]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/braggnn.png">
</div>
<div class="project-text">
<div class="project-title">BraggNN: Fast X-ray Bragg Peak Analysis Using Deep Learning</div>
<div class="project-venue">IUCrJ, 2022</div>
<div class="project-authors">Zhengchun Liu, Hemant Sharma, Jun-Sang Park, Peter Kenesei, Antonino Miceli, Jonathan Almer, Rajkumar Kettimuthu, Ian Foster</div>
<div class="project-links"><a href='https://arxiv.org/abs/2008.08198' target="_blank">[arXiv]</a> <a href='https://journals.iucr.org/m/issues/2022/01/00/fs5198/' target="_blank">[IUCrJ]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/intelligent_resolution.png">
</div>
<div class="project-text">
<div class="project-title">Intelligent Resolution: Integrating Cryo-EM with AI-driven Multi-resolution Simulations to Observe the SARS-CoV-2 Replication-Transcription Machinery in Action</div>
<div class="project-venue">SC, 2021</div>
<div class="project-authors">Anda Trifan, Defne Gorgun, Zongyi Li, Alexander Brace, Maxim Zvyagin, Heng Ma, Austin Clyde, David Clark, Michael Salim, David J. Hardy, Tom Burnley, Lei Huang, John McCalpin, Murali Emani, Hyenseung Yoo, Junqi Yin, Aristeidis Tsaris, Vishal Subbiah, Tanveer Raza, Jessica Liu, Noah Trebesch, Geoffrey Wells, Venkatesh Mysore, Thomas Gibbs, James Phillips, S. Chakra Chennubhotla, Ian Foster, Rick Stevens, Anima Anandkumar, Venkatram Vishwanath, John E. Stone, View ORCID ProfileEmad Tajkhorshid, Sarah A. Harris, Arvind Ramanathan</div>
<div class="project-links"><a href='https://www.biorxiv.org/content/10.1101/2021.10.09.463779v1' target="_blank">[bioRXiv]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/path_to_wafer_scale.gif.webp">
</div>
<div class="project-text">
<div class="project-title">The Path to Successful Wafer-Scale Integration: The Cerebras Story</div>
<div class="project-venue">IEEE Micro, 2021</div>
<div class="project-authors">Gary Lauterbach</div>
<div class="project-links"><a href='https://www.computer.org/csdl/magazine/mi/2021/06/09623424/1yJTq0E9m1O' target="_blank">[Paper]</a> <a href='https://8968533.fs1.hubspotusercontent-na1.net/hubfs/8968533/IEEE%20Micro%202021-11%20Path%20to%20Wafer-Scale%20Integration.pdf' target="_blank">[THEME ARTICLE: MICROPROCESSOR AT 50]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/stream_ai_md.png">
</div>
<div class="project-text">
<div class="project-title">Stream-AI-MD: streaming AI-driven adaptive molecular simulations for heterogeneous computing platforms</div>
<div class="project-venue">PASC, 2021</div>
<div class="project-authors">Alexander Brace, Michael Salim, Vishal Subbiah, Heng Ma, Murali Emani, Anda Trifa, Austin R. Clyde, Corey Adams, Thomas Uram, Hyunseung Yoo, Andew Hock, Jessica Liu, Venkatram Vishwanath, Arvind Ramanathan</div>
<div class="project-links"><a href='https://dl.acm.org/doi/10.1145/3468267.3470578' target="_blank">[ACM]</a> <a href='https://openreview.net/forum?id=FYjf0Nkoy0' target="_blank">[OpenReview]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/mem_efficient_unet.png">
</div>
<div class="project-text">
<div class="project-title">Memory Efficient 3D U-Net with Reversible Mobile Inverted Bottlenecks for Brain Tumor Segmentation</div>
<div class="project-venue">Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, 2021</div>
<div class="project-authors">Mihir Pendse, Vithursan Thangarasa, Vitaliy Chiley, Ryan Holmdahl, Joel Hestness, Dennis DeCoste</div>
<div class="project-links"><a href='https://arxiv.org/abs/2104.09648' target="_blank">[arXiv]</a> <a href='https://www.springerprofessional.de/en/memory-efficient-3d-u-net-with-reversible-mobile-inverted-bottle/19007114' target="_blank">[Springer]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/pipelined_backprop.png">
</div>
<div class="project-text">
<div class="project-title">Pipelined Backpropagation at Scale: Training Large Models without Batches</div>
<div class="project-venue">MLSys, 2021</div>
<div class="project-authors">Atli Kosson, Vitaliy Chiley, Abhinav Venigalla, Joel Hestness, Urs Köster</div>
<div class="project-links"><a href='https://arxiv.org/abs/2003.11666' target="_blank">[arXiv]</a> <a href='https://mlsys.org/media/mlsys-2021/Slides/1534.pdf' target="_blank">[Slides]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/todo.png">
</div>
<div class="project-text">
<div class="project-title">System Integration of Neocortex, a Unique, Scalable AI Platform</div>
<div class="project-venue">PEARC, 2021</div>
<div class="project-authors">Paola A. Buitrago, Julian A. Uran, Nicholas A. Nystrom</div>
<div class="project-links"><a href='https://dl.acm.org/doi/abs/10.1145/3437359.3465604' target="_blank">[Paper]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/todo.png">
</div>
<div class="project-text">
<div class="project-title">Fast Stencil-Code Computation on a Wafer-Scale Processor</div>
<div class="project-venue">SC, 2020</div>
<div class="project-authors">Kamil Rocki, Dirk Van Essendelft, Ilya Sharapov, Robert Schreiber, Michael Morrison, Vladimir Kibardin, Andrey Portnoy, Jean Francois Dietiker, Madhava Syamlal, Michael James</div>
<div class="project-links"><a href='https://arxiv.org/abs/2010.03660' target="_blank">[arXiv]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/todo.png">
</div>
<div class="project-text">
<div class="project-title">The curious case of developmental BERTology: On sparsity, transfer learning, generalization and the brain</div>
<div class="project-venue">arXiv, 2020</div>
<div class="project-authors">Xin Wang</div>
<div class="project-links"><a href='https://arxiv.org/abs/2007.03774' target="_blank">[arXiv]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/todo.png">
</div>
<div class="project-text">
<div class="project-title">Generating SIMD Instructions for Cerebras CS-1 using Polyhedral Compilation Techniques</div>
<div class="project-venue">IMPACT, 2020</div>
<div class="project-authors">Sven Verdoolaege, Rob Schreiber, Manjunath Kudlur, Harinath Kamepalli</div>
<div class="project-links"><a href='https://cdn.sanity.io/files/e4qjo92p/production/e304b6a4a69fa63fc9cb006ab5836fc17d0de528.pdf' target="_blank">[Paper]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/template_isl.png">
</div>
<div class="project-text">
<div class="project-title">A Templated C++ Interface for isl</div>
<div class="project-venue">IMPACT, 2021</div>
<div class="project-authors">Sven Verdoolaege, Oleksandr Zinenko, Manjunath Kudlur, Ron Estrin, Tianjiao Sun, Harinath Kamepalli</div>
<div class="project-links"><a href='https://8968533.fs1.hubspotusercontent-na1.net/hubfs/8968533/Whitepapers/A%20Templated%20C++%20Interface%20for%20isl.pdf' target="_blank">[Paper]</a></div>
</div>
</div>
<div class="project">
<div class="project-image">
<img src="img/online_norm.png">
</div>
<div class="project-text">
<div class="project-title">Online Normalization for Training Neural Networks</div>
<div class="project-venue">NeurIPS, 2019</div>
<div class="project-authors">Vitaliy Chiley, Ilya Sharapov, Atli Kosson, Urs Koster, Ryan Reece, Sofia Samaniego de la Fuente, Vishal Subbiah, Michael James</div>
<div class="project-links"><a href='https://papers.nips.cc/paper/2019/hash/cb3ce9b06932da6faaa7fc70d5b5d2f4-Abstract.html' target="_blank">[NeurIPS]</a> <a href="https://arxiv.org/abs/1905.05894">[arXiv]</a></div>
</div>
</div>
</div>
</div>
</div>
</div>
<script src="js/jquery-2.2.4.min.js"></script>
<script src="js/jquery.scrollTo.min.js"></script>
<script src="js/scrollTo.js"></script>
<script src="js/bootstrap.min.js"></script>
</body>
</html>