INFO [2300506] ______________________________________________________________________ _____ ____ __ ____ __ _ _____ _ | ____/ ___|| \/ \ \ / /_ _| |_ _|__ ___ | | | _| \___ \| |\/| |\ \ / / _` | | | |/ _ \ / _ \| | | |___ ___) | | | | \ V / (_| | | | | (_) | (_) | | |_____|____/|_| |_| \_/ \__,_|_| |_|\___/ \___/|_| ______________________________________________________________________ ESMValTool - Earth System Model Evaluation Tool. http://www.esmvaltool.org CORE DEVELOPMENT TEAM AND CONTACTS: Birgit Hassler (Co-PI; DLR, Germany - birgit.hassler@dlr.de) Alistair Sellar (Co-PI; Met Office, UK - alistair.sellar@metoffice.gov.uk) Bouwe Andela (Netherlands eScience Center, The Netherlands - b.andela@esciencecenter.nl) Lee de Mora (PML, UK - ledm@pml.ac.uk) Niels Drost (Netherlands eScience Center, The Netherlands - n.drost@esciencecenter.nl) Veronika Eyring (DLR, Germany - veronika.eyring@dlr.de) Bettina Gier (UBremen, Germany - gier@uni-bremen.de) Remi Kazeroni (DLR, Germany - remi.kazeroni@dlr.de) Nikolay Koldunov (AWI, Germany - nikolay.koldunov@awi.de) Axel Lauer (DLR, Germany - axel.lauer@dlr.de) Saskia Loosveldt-Tomas (BSC, Spain - saskia.loosveldt@bsc.es) Ruth Lorenz (ETH Zurich, Switzerland - ruth.lorenz@env.ethz.ch) Benjamin Mueller (LMU, Germany - b.mueller@iggf.geo.uni-muenchen.de) Valeriu Predoi (URead, UK - valeriu.predoi@ncas.ac.uk) Mattia Righi (DLR, Germany - mattia.righi@dlr.de) Manuel Schlund (DLR, Germany - manuel.schlund@dlr.de) Breixo Solino Fernandez (DLR, Germany - breixo.solinofernandez@dlr.de) Javier Vegas-Regidor (BSC, Spain - javier.vegas@bsc.es) Klaus Zimmermann (SMHI, Sweden - klaus.zimmermann@smhi.se) For further help, please read the documentation at http://docs.esmvaltool.org. Have fun! INFO [2300506] Package versions INFO [2300506] ---------------- INFO [2300506] ESMValCore: 2.11.1 INFO [2300506] ESMValTool: 2.11.0 INFO [2300506] ---------------- INFO [2300506] Using config file /nird/home/tomast/.esmvaltool/config-user.yml INFO [2300506] Writing program log files to: /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/run/main_log.txt /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/run/main_log_debug.txt /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/run/cmor_log.txt INFO [2300506] Using default ESGF configuration, configuration file /nird/home/tomast/.esmvaltool/esgf-pyclient.yml not present. INFO [2300506] Starting the Earth System Model Evaluation Tool at time: 2025-01-31 16:50:42 UTC INFO [2300506] ---------------------------------------------------------------------- INFO [2300506] RECIPE = /projects/NS2980K/home/tomast/ESMValTool/recipes-2.11.0/standard_recipes/atmosphere/recipe_validation.yml INFO [2300506] RUNDIR = /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/run INFO [2300506] WORKDIR = /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/work INFO [2300506] PREPROCDIR = /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/preproc INFO [2300506] PLOTDIR = /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/plots INFO [2300506] ---------------------------------------------------------------------- INFO [2300506] Running tasks using at most 16 processes INFO [2300506] If your system hangs during execution, it may not have enough memory for keeping this number of tasks in memory. INFO [2300506] If you experience memory problems, try reducing 'max_parallel_tasks' in your user configuration file. WARNING [2300506] Using the Dask basic scheduler. This may lead to slow computations and out-of-memory errors. Note that the basic scheduler may still be the best choice for preprocessor functions that are not lazy. In that case, you can safely ignore this warning. See https://docs.esmvaltool.org/projects/ESMValCore/en/latest/quickstart/configure.html#dask-distributed-configuration for more information. INFO [2300506] Creating tasks from recipe INFO [2300506] Creating tasks for diagnostic validation_with_CERES-EBAF INFO [2300506] Creating diagnostic task validation_with_CERES-EBAF/lat_lon INFO [2300506] Creating diagnostic task validation_with_CERES-EBAF/zonal_mean INFO [2300506] Creating diagnostic task validation_with_CERES-EBAF/meridional_mean INFO [2300506] Creating preprocessor task validation_with_CERES-EBAF/rtnt INFO [2300506] Creating preprocessor 'pp_rad_derive_var' task for variable 'rtnt' INFO [2300506] Found input files for Dataset: rtnt, Amon, CMIP5, MPI-ESM-LR, historical, r1i1p1, v20120315 INFO [2300506] Found input files for Dataset: rtnt, Amon, CMIP5, NorESM1-M, historical, r1i1p1, v20120412 INFO [2300506] Found input files for Dataset: rtnt, Amon, obs4MIPs, CERES-EBAF, v20160610 INFO [2300506] PreprocessingTask validation_with_CERES-EBAF/rtnt created. INFO [2300506] Creating preprocessor task validation_with_CERES-EBAF/rsut INFO [2300506] Creating preprocessor 'pp_rad' task for variable 'rsut' INFO [2300506] Found input files for Dataset: rsut, Amon, CMIP5, MPI-ESM-LR, historical, r1i1p1, v20120315 INFO [2300506] Found input files for Dataset: rsut, Amon, CMIP5, NorESM1-M, historical, r1i1p1, v20120412 INFO [2300506] Found input files for Dataset: rsut, Amon, obs4MIPs, CERES-EBAF, v20160610 INFO [2300506] PreprocessingTask validation_with_CERES-EBAF/rsut created. INFO [2300506] These tasks will be executed: validation_with_CERES-EBAF/lat_lon, validation_with_CERES-EBAF/meridional_mean, validation_with_CERES-EBAF/zonal_mean, validation_with_CERES-EBAF/rsut, validation_with_CERES-EBAF/rtnt INFO [2300506] Wrote recipe with version numbers and wildcards to: file:///projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/run/recipe_validation_filled.yml INFO [2300506] Will download 44.09 MB Will download the following files: 44.09 MB ESGFFile:obs4MIPs/CERES-EBAF/v20160610/rsdt_CERES-EBAF_L3B_Ed2-8_200003-201404.nc on hosts ['dpesgf03.nccs.nasa.gov'] Downloading 44.09 MB.. INFO [2300506] Downloaded /scratch/tomast/ESGF/obs4MIPs/CERES-EBAF/v20160610/rsdt_CERES-EBAF_L3B_Ed2-8_200003-201404.nc (44.09 MB) in 2.35 seconds (18.77 MB/s) from dpesgf03.nccs.nasa.gov INFO [2300506] Downloaded 44.09 MB in 2.37 seconds (18.57 MB/s) INFO [2300506] Successfully downloaded all requested files. INFO [2300506] Using the Dask basic scheduler. INFO [2300506] Running 5 tasks using 5 processes INFO [2300664] Starting task validation_with_CERES-EBAF/rsut in process [2300664] INFO [2300663] Starting task validation_with_CERES-EBAF/rtnt in process [2300663] INFO [2300506] Progress: 2 tasks running, 3 tasks waiting for ancestors, 0/5 done INFO [2300664] Successfully completed task validation_with_CERES-EBAF/rsut (priority 4) in 0:00:04.921280 INFO [2300506] Progress: 1 tasks running, 3 tasks waiting for ancestors, 1/5 done INFO [2300663] Successfully completed task validation_with_CERES-EBAF/rtnt (priority 3) in 0:00:12.637487 INFO [2300506] Progress: 0 tasks running, 3 tasks waiting for ancestors, 2/5 done INFO [2300665] Starting task validation_with_CERES-EBAF/lat_lon in process [2300665] INFO [2300666] Starting task validation_with_CERES-EBAF/zonal_mean in process [2300666] INFO [2300665] Running command ['/projects/NS2980K/conda/esmvaltool2.11.0/bin/python', '/projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvaltool/diag_scripts/validation.py', '/projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/run/validation_with_CERES-EBAF/lat_lon/settings.yml'] INFO [2300665] Writing output to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/work/validation_with_CERES-EBAF/lat_lon INFO [2300665] Writing plots to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/plots/validation_with_CERES-EBAF/lat_lon INFO [2300665] Writing log to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/run/validation_with_CERES-EBAF/lat_lon/log.txt INFO [2300665] To re-run this diagnostic script, run: cd /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/run/validation_with_CERES-EBAF/lat_lon; MPLBACKEND="Agg" /projects/NS2980K/conda/esmvaltool2.11.0/bin/python /projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvaltool/diag_scripts/validation.py /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/run/validation_with_CERES-EBAF/lat_lon/settings.yml INFO [2300667] Starting task validation_with_CERES-EBAF/meridional_mean in process [2300667] INFO [2300666] Running command ['/projects/NS2980K/conda/esmvaltool2.11.0/bin/python', '/projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvaltool/diag_scripts/validation.py', '/projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/run/validation_with_CERES-EBAF/zonal_mean/settings.yml'] INFO [2300666] Writing output to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/work/validation_with_CERES-EBAF/zonal_mean INFO [2300666] Writing plots to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/plots/validation_with_CERES-EBAF/zonal_mean INFO [2300666] Writing log to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/run/validation_with_CERES-EBAF/zonal_mean/log.txt INFO [2300666] To re-run this diagnostic script, run: cd /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/run/validation_with_CERES-EBAF/zonal_mean; MPLBACKEND="Agg" /projects/NS2980K/conda/esmvaltool2.11.0/bin/python /projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvaltool/diag_scripts/validation.py /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/run/validation_with_CERES-EBAF/zonal_mean/settings.yml INFO [2300667] Running command ['/projects/NS2980K/conda/esmvaltool2.11.0/bin/python', '/projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvaltool/diag_scripts/validation.py', '/projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/run/validation_with_CERES-EBAF/meridional_mean/settings.yml'] INFO [2300667] Writing output to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/work/validation_with_CERES-EBAF/meridional_mean INFO [2300667] Writing plots to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/plots/validation_with_CERES-EBAF/meridional_mean INFO [2300667] Writing log to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/run/validation_with_CERES-EBAF/meridional_mean/log.txt INFO [2300667] To re-run this diagnostic script, run: cd /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/run/validation_with_CERES-EBAF/meridional_mean; MPLBACKEND="Agg" /projects/NS2980K/conda/esmvaltool2.11.0/bin/python /projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvaltool/diag_scripts/validation.py /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/run/validation_with_CERES-EBAF/meridional_mean/settings.yml INFO [2300506] Progress: 3 tasks running, 0 tasks waiting for ancestors, 2/5 done INFO [2300667] Maximum memory used (estimate): 0.5 GB INFO [2300666] Maximum memory used (estimate): 0.5 GB INFO [2300667] Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. INFO [2300666] Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. INFO [2300666] Successfully completed task validation_with_CERES-EBAF/zonal_mean (priority 1) in 0:00:09.994930 INFO [2300667] Successfully completed task validation_with_CERES-EBAF/meridional_mean (priority 2) in 0:00:10.050997 INFO [2300506] Progress: 1 tasks running, 0 tasks waiting for ancestors, 4/5 done INFO [2300665] Maximum memory used (estimate): 0.6 GB INFO [2300665] Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. INFO [2300665] Successfully completed task validation_with_CERES-EBAF/lat_lon (priority 0) in 0:00:19.601847 INFO [2300506] Progress: 0 tasks running, 0 tasks waiting for ancestors, 5/5 done INFO [2300506] Successfully completed all tasks. INFO [2300506] Wrote recipe with version numbers and wildcards to: file:///projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/run/recipe_validation_filled.yml INFO [2300506] Wrote recipe output to: file:///projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/index.html INFO [2300506] It looks like you are connected to a remote machine via SSH. To show the output html file, you can try the following command on your local machine: server=10.11.120.250 && port=31415 && ssh -t -L ${port}:localhost:${port} tomast@${server} /projects/NS2980K/conda/esmvaltool2.11.0/bin/python -m http.server ${port} -d /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042 Then visit http://localhost:31415 in your browser INFO [2300506] If the port 31415 is already in use, you can replace it with any other free one (e.g., 12789). If you are connected through a jump host, replace the server IP address 10.11.120.250 with your SSH server name INFO [2300506] Ending the Earth System Model Evaluation Tool at time: 2025-01-31 16:51:18 UTC INFO [2300506] Time for running the recipe was: 0:00:35.966778 INFO [2300506] Maximum memory used (estimate): 3.7 GB INFO [2300506] Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. WARNING [2300506] Input data is not (fully) CMOR-compliant, see /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_validation_20250131_165042/run/cmor_log.txt for details INFO [2300506] Run was successful