BERT MULTI GPU

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

REQUIREMENT

python 3

tensorflow 1.12.0

0, edit the input and output file name in create_pretraining_data.py and run_pretraining_gpu.py

1, run create_pretraining_data.py

2, run run_pretraining_gpu.py

Edit n_gpus in run_pretraining_gpu.py

In sample_text.txt, sentence is end by \n, paragraph is splitted by empty line.

Name	Name	Last commit message	Last commit date
Latest commit guotong1988 Update run_pretraining_gpu.py Mar 18, 2019 675951a · Mar 18, 2019 History 17 Commits
tmp_data	tmp_data	1	Dec 25, 2018
.gitignore	.gitignore	1	Dec 25, 2018
CONTRIBUTING.md	CONTRIBUTING.md	1	Dec 25, 2018
LICENSE	LICENSE	1	Dec 25, 2018
README.md	README.md	Update README.md	Feb 25, 2019
__init__.py	__init__.py	1	Dec 25, 2018
bert_config.json	bert_config.json	1	Dec 25, 2018
create_pretraining_data.py	create_pretraining_data.py	Update create_pretraining_data.py	Feb 21, 2019
extract_features.py	extract_features.py	1	Dec 25, 2018
modeling.py	modeling.py	1	Dec 25, 2018
modeling_test.py	modeling_test.py	1	Dec 25, 2018
multilingual.md	multilingual.md	1	Dec 25, 2018
optimization.py	optimization.py	1	Dec 25, 2018
optimization_gpu.py	optimization_gpu.py	1	Dec 25, 2018
optimization_test.py	optimization_test.py	1	Dec 25, 2018
requirements.txt	requirements.txt	1	Dec 25, 2018
run_classifier.py	run_classifier.py	1	Dec 25, 2018
run_pretraining.py	run_pretraining.py	1	Dec 25, 2018
run_pretraining_gpu.py	run_pretraining_gpu.py	Update run_pretraining_gpu.py	Mar 18, 2019
run_squad.py	run_squad.py	1	Dec 25, 2018
sample_text.txt	sample_text.txt	1	Dec 25, 2018
tokenization.py	tokenization.py	1	Dec 25, 2018
tokenization_test.py	tokenization_test.py	1	Dec 25, 2018
vocab.py	vocab.py	1	Dec 25, 2018
vocab.txt	vocab.txt	1	Dec 25, 2018