Skip to content
Snippets Groups Projects
Commit a0ad3bd2 authored by Michael Langguth's avatar Michael Langguth
Browse files

Try caching dataset.

parent c98551cd
Branches
No related tags found
No related merge requests found
Pipeline #97240 failed
......@@ -181,6 +181,8 @@ class ERA5Dataset(object):
random.shuffle(filenames)
dataset = tf.data.TFRecordDataset(filenames, buffer_size = 8* 1024 * 1024)
# cache dataset
dataset = dataset.cache()
# dataset = dataset.filter(self.filter)
if shuffle:
dataset = dataset.apply(tf.contrib.data.shuffle_and_repeat(buffer_size =1024, count = self.num_epochs))
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment