Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
esde
machine-learning
AMBS
Commits
2dc95872
Commit
2dc95872
authored
Jun 30, 2020
by
scarlet stadtler
🎧
Browse files
Changed destination directory and make use of the 2 tier pystager.
parent
5bb7c002
Pipeline
#40089
passed with stages
in 1 minute and 53 seconds
Changes
1
Pipelines
2
Hide whitespace changes
Inline
Side-by-side
video_prediction_savp/HPC_scripts/DataExtraction.sh
View file @
2dc95872
...
...
@@ -23,8 +23,8 @@ module load h5py/2.9.0-Python-3.6.8
module load mpi4py/3.0.1-Python-3.6.8
module load netcdf4-python/1.5.0.1-Python-3.6.8
srun python ../../workflow_parallel_frame_prediction/DataExtraction/mpi_stager_v2.py
--source_dir
/p/fastdata/slmet/slmet111/met_data/ecmwf/era5/nc/2014/
--destination_dir
${
SAVE_DIR
}
/extractedData/2014
#
srun python ../../workflow_parallel_frame_prediction/DataExtraction/mpi_stager_v2.py --source_dir /p/fastdata/slmet/slmet111/met_data/ecmwf/era5/nc/2014/ --destination_dir ${SAVE_DIR}/extractedData/2014
# 2tier pystager
#
srun python ../../workflow_parallel_frame_prediction/DataExtraction/main_single_master.py --source_dir /p/fastdata/slmet/slmet111/met_data/ecmwf/era5/nc/201
7
/ --destination_dir
/p/scratch/deepacf/${USE
R}/extractedData/201
7
srun python ../../workflow_parallel_frame_prediction/DataExtraction/main_single_master.py
--source_dir
/p/fastdata/slmet/slmet111/met_data/ecmwf/era5/nc/201
3
/
--destination_dir
${
SAVE_DI
R
}
/extractedData/201
3
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment