Skip to content

README.md

Latest commit

 

History

History
24 lines (22 loc) · 1.15 KB

README.md

File metadata and controls

24 lines (22 loc) · 1.15 KB

AutoTrain

Features

Trains automaticly (look in the code)

Important twerks for tensorflow in order to save all checkpoints

1 keep all checkpoints

  • Found here
  • Add keep_checkpoint_max=0 like config = tf.estimator.RunConfig(model_dir=FLAGS.model_dir, keep_checkpoint_max=500) in model_main.py.
  • Add max_to_keep= 10000 (or large number) to model_lib.py like that in saver:
saver = tf.train.Saver(
	variables_to_restore,
	keep_checkpoint_every_n_hours=keep_checkpoint_every_n_hours,
	max_to_keep=500)# <= added max_to_keep argument here
	
	
saver = tf.train.Saver(
	sharded=True,
	keep_checkpoint_every_n_hours=keep_checkpoint_every_n_hours,
	save_relative_paths=True,
	max_to_keep=500)# <= added max_to_keep argument here

2 keep checkpoint every n steps

  • link from here
  • Add save_checkpoints_steps=1000 (or any other number) to config = tf.estimator.RunConfig(model_dir=FLAGS.model_dir, session_config=session_config, save_checkpoints_steps=1000, keep_checkpoint_max=None) in model_main.py