2023-12-27 15:34:54,718 UTC [730848] INFO esmvalcore._main:492 ______________________________________________________________________ _____ ____ __ ____ __ _ _____ _ | ____/ ___|| \/ \ \ / /_ _| |_ _|__ ___ | | | _| \___ \| |\/| |\ \ / / _` | | | |/ _ \ / _ \| | | |___ ___) | | | | \ V / (_| | | | | (_) | (_) | | |_____|____/|_| |_| \_/ \__,_|_| |_|\___/ \___/|_| ______________________________________________________________________ ESMValTool - Earth System Model Evaluation Tool. http://www.esmvaltool.org CORE DEVELOPMENT TEAM AND CONTACTS: Birgit Hassler (Co-PI; DLR, Germany - birgit.hassler@dlr.de) Alistair Sellar (Co-PI; Met Office, UK - alistair.sellar@metoffice.gov.uk) Bouwe Andela (Netherlands eScience Center, The Netherlands - b.andela@esciencecenter.nl) Lee de Mora (PML, UK - ledm@pml.ac.uk) Niels Drost (Netherlands eScience Center, The Netherlands - n.drost@esciencecenter.nl) Veronika Eyring (DLR, Germany - veronika.eyring@dlr.de) Bettina Gier (UBremen, Germany - gier@uni-bremen.de) Remi Kazeroni (DLR, Germany - remi.kazeroni@dlr.de) Nikolay Koldunov (AWI, Germany - nikolay.koldunov@awi.de) Axel Lauer (DLR, Germany - axel.lauer@dlr.de) Saskia Loosveldt-Tomas (BSC, Spain - saskia.loosveldt@bsc.es) Ruth Lorenz (ETH Zurich, Switzerland - ruth.lorenz@env.ethz.ch) Benjamin Mueller (LMU, Germany - b.mueller@iggf.geo.uni-muenchen.de) Valeriu Predoi (URead, UK - valeriu.predoi@ncas.ac.uk) Mattia Righi (DLR, Germany - mattia.righi@dlr.de) Manuel Schlund (DLR, Germany - manuel.schlund@dlr.de) Breixo Solino Fernandez (DLR, Germany - breixo.solinofernandez@dlr.de) Javier Vegas-Regidor (BSC, Spain - javier.vegas@bsc.es) Klaus Zimmermann (SMHI, Sweden - klaus.zimmermann@smhi.se) For further help, please read the documentation at http://docs.esmvaltool.org. Have fun! 2023-12-27 15:34:54,718 UTC [730848] INFO esmvalcore._main:493 Package versions 2023-12-27 15:34:54,718 UTC [730848] INFO esmvalcore._main:494 ---------------- 2023-12-27 15:34:54,718 UTC [730848] INFO esmvalcore._main:495 ESMValCore: 2.10.0 2023-12-27 15:34:54,718 UTC [730848] INFO esmvalcore._main:497 ESMValTool: 2.10.0 2023-12-27 15:34:54,718 UTC [730848] INFO esmvalcore._main:498 ---------------- 2023-12-27 15:34:54,718 UTC [730848] INFO esmvalcore._main:499 Using config file /nird/home/tomast/.esmvaltool/config-user.yml 2023-12-27 15:34:54,718 UTC [730848] INFO esmvalcore._main:500 Writing program log files to: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/run/main_log.txt /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/run/main_log_debug.txt 2023-12-27 15:34:54,861 UTC [730848] INFO esmvalcore.config._esgf_pyclient:116 Using default ESGF configuration, configuration file /nird/home/tomast/.esmvaltool/esgf-pyclient.yml not present. 2023-12-27 15:34:54,861 UTC [730848] WARNING esmvalcore.config._esgf_pyclient:181 ESGF credentials missing, only data that is accessible without logging in will be available. See https://esgf.github.io/esgf-user-support/user_guide.html for instructions on how to create an account if you do not have one yet. Next, configure your system so esmvaltool can use your credentials. This can be done using the keyring package, or you can just enter them in /nird/home/tomast/.esmvaltool/esgf-pyclient.yml. keyring ======= First install the keyring package (requires a supported backend, see https://pypi.org/project/keyring/): $ pip install keyring Next, set your username and password by running the commands: $ keyring set ESGF hostname $ keyring set ESGF username $ keyring set ESGF password To check that you entered your credentials correctly, run: $ keyring get ESGF hostname $ keyring get ESGF username $ keyring get ESGF password configuration file ================== You can store the hostname, username, and password or your OpenID account in a plain text in the file /nird/home/tomast/.esmvaltool/esgf-pyclient.yml like this: logon: hostname: "your-hostname" username: "your-username" password: "your-password" or your can configure an interactive log in: logon: interactive: true Note that storing your password in plain text in the configuration file is less secure. On shared systems, make sure the permissions of the file are set so only you can read it, i.e. $ ls -l /nird/home/tomast/.esmvaltool/esgf-pyclient.yml shows permissions -rw-------. 2023-12-27 15:34:54,955 UTC [730848] INFO esmvalcore._main:91 Starting the Earth System Model Evaluation Tool at time: 2023-12-27 15:34:54 UTC 2023-12-27 15:34:54,955 UTC [730848] INFO esmvalcore._main:95 ---------------------------------------------------------------------- 2023-12-27 15:34:54,955 UTC [730848] INFO esmvalcore._main:96 RECIPE = /projects/NS2980K/home/tomast/ESMValTool/recipes-2.10.0/standard_recipes/examples/recipe_r.yml 2023-12-27 15:34:54,955 UTC [730848] INFO esmvalcore._main:97 RUNDIR = /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/run 2023-12-27 15:34:54,955 UTC [730848] INFO esmvalcore._main:98 WORKDIR = /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/work 2023-12-27 15:34:54,955 UTC [730848] INFO esmvalcore._main:99 PREPROCDIR = /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/preproc 2023-12-27 15:34:54,955 UTC [730848] INFO esmvalcore._main:100 PLOTDIR = /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/plots 2023-12-27 15:34:54,955 UTC [730848] INFO esmvalcore._main:101 ---------------------------------------------------------------------- 2023-12-27 15:34:54,955 UTC [730848] INFO esmvalcore._main:104 Running tasks using at most 256 processes 2023-12-27 15:34:54,955 UTC [730848] INFO esmvalcore._main:106 If your system hangs during execution, it may not have enough memory for keeping this number of tasks in memory. 2023-12-27 15:34:54,956 UTC [730848] INFO esmvalcore._main:109 If you experience memory problems, try reducing 'max_parallel_tasks' in your user configuration file. 2023-12-27 15:34:54,956 UTC [730848] WARNING esmvalcore.config._dask:18 Using the Dask basic scheduler. This may lead to slow computations and out-of-memory errors. Note that the basic scheduler may still be the best choice for preprocessor functions that are not lazy. In that case, you can safely ignore this warning. See https://docs.esmvaltool.org/projects/ESMValCore/en/latest/quickstart/configure.html#dask-distributed-configuration for more information. 2023-12-27 15:34:54,956 UTC [730848] DEBUG esmvalcore._recipe.check:54 Checking recipe against schema /projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/_recipe/recipe_schema.yml 2023-12-27 15:34:54,964 UTC [730848] DEBUG esmvalcore._recipe.to_datasets:343 Populating list of datasets for variable pr in diagnostic example 2023-12-27 15:34:54,964 UTC [730848] DEBUG esmvalcore._recipe.to_datasets:367 Found Dataset: pr, day, CMIP5, EC-EARTH, historical, r1i1p1 2023-12-27 15:34:54,965 UTC [730848] DEBUG esmvalcore._recipe.recipe:781 Retrieving diagnostics from recipe 2023-12-27 15:34:54,965 UTC [730848] DEBUG esmvalcore._recipe.recipe:809 Setting script for diagnostic example 2023-12-27 15:34:54,965 UTC [730848] INFO esmvalcore._recipe.recipe:999 Creating tasks from recipe 2023-12-27 15:34:54,965 UTC [730848] INFO esmvalcore._recipe.recipe:1008 Creating tasks for diagnostic example 2023-12-27 15:34:54,965 UTC [730848] INFO esmvalcore._recipe.recipe:930 Creating diagnostic task example/main 2023-12-27 15:34:54,966 UTC [730848] INFO esmvalcore._recipe.recipe:983 Creating preprocessor task example/pr 2023-12-27 15:34:54,967 UTC [730848] INFO esmvalcore._recipe.recipe:653 Creating preprocessor 'preproc' task for variable 'pr' 2023-12-27 15:34:54,967 UTC [730848] DEBUG esmvalcore.config._config:45 Loading extra facets from /projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config/extra_facets/cmip5-fx.yml 2023-12-27 15:34:54,967 UTC [730848] DEBUG esmvalcore.config._config:45 Loading extra facets from /projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config/extra_facets/cmip5-institutes.yml 2023-12-27 15:34:54,982 UTC [730848] DEBUG esmvalcore.config._config:45 Loading extra facets from /projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config/extra_facets/cmip5-product.yml 2023-12-27 15:34:54,985 UTC [730848] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/ICHEC/EC-EARTH/historical/day/atmos/day/r1i1p1/*/pr/pr_day_EC-EARTH_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/ICHEC/EC-EARTH/historical/day/atmos/day/r1i1p1/*/pr/pr_day_EC-EARTH_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/ICHEC/EC-EARTH/historical/day/atmos/day/r1i1p1/*/pr/pr_day_EC-EARTH_historical_r1i1p1*.nc 2023-12-27 15:34:55,205 UTC [730848] DEBUG esmvalcore._recipe.recipe:263 Using input files for variable pr of dataset EC-EARTH: /projects/NS9560K-datalake/ESGF/cmip5/output1/ICHEC/EC-EARTH/historical/day/atmos/day/r1i1p1/v20121115/pr/pr_day_EC-EARTH_historical_r1i1p1_19900101-19991231.nc 2023-12-27 15:34:55,205 UTC [730848] INFO esmvalcore._recipe.recipe:520 Found input files for Dataset: pr, day, CMIP5, EC-EARTH, historical, r1i1p1, v20121115 2023-12-27 15:34:55,206 UTC [730848] INFO esmvalcore._recipe.recipe:677 PreprocessingTask example/pr created. 2023-12-27 15:34:55,206 UTC [730848] DEBUG esmvalcore._recipe.recipe:678 PreprocessingTask example/pr will create the files: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/preproc/example/pr/CMIP5_EC-EARTH_day_historical_r1i1p1_pr_1997-1997.nc 2023-12-27 15:34:55,206 UTC [730848] DEBUG esmvalcore._recipe.recipe:858 Linking tasks for diagnostic example script main 2023-12-27 15:34:55,206 UTC [730848] DEBUG esmvalcore._recipe.recipe:867 Pattern example/pr matches ['example/pr'] 2023-12-27 15:34:55,206 UTC [730848] INFO esmvalcore._recipe.recipe:1048 These tasks will be executed: example/pr, example/main 2023-12-27 15:34:55,207 UTC [730848] DEBUG esmvalcore._main:128 Recipe summary: DiagnosticTask: example/main script: examples/diagnostic.R settings: {'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'log_level': 'info', 'output_file_type': 'png', 'parameter1': 1, 'plot_dir': '/projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/plots/example/main', 'profile_diagnostic': False, 'recipe': PosixPath('recipe_r.yml'), 'run_dir': '/projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/run/example/main', 'script': 'main', 'version': '2.10.0', 'work_dir': '/projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/work/example/main'} ancestors: PreprocessingTask: example/pr order: ['remove_supplementary_variables', 'save'] PreprocessorFile: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/preproc/example/pr/CMIP5_EC-EARTH_day_historical_r1i1p1_pr_1997-1997.nc input files: [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/ICHEC/EC-EARTH/historical/day/atmos/day/r1i1p1/v20121115/pr/pr_day_EC-EARTH_historical_r1i1p1_19900101-19991231.nc')] settings: {'remove_supplementary_variables': {}, 'save': {'compress': False, 'filename': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/preproc/example/pr/CMIP5_EC-EARTH_day_historical_r1i1p1_pr_1997-1997.nc')}} ancestors: None 2023-12-27 15:34:55,209 UTC [730848] INFO esmvalcore._recipe.recipe:1111 Wrote recipe with version numbers and wildcards to: file:///projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/run/recipe_r_filled.yml 2023-12-27 15:34:55,209 UTC [730848] DEBUG esmvalcore.esgf._download:535 All required data is available locally, not downloading anything. 2023-12-27 15:34:55,209 UTC [730848] INFO esmvalcore.config._dask:70 Using the Dask basic scheduler. 2023-12-27 15:34:55,209 UTC [730848] INFO esmvalcore._task:760 Running 2 tasks using 2 processes 2023-12-27 15:34:55,228 UTC [730984] INFO esmvalcore._task:261 Starting task example/pr in process [730984] 2023-12-27 15:34:55,326 UTC [730848] INFO esmvalcore._task:793 Progress: 1 tasks running, 1 tasks waiting for ancestors, 0/2 done 2023-12-27 15:34:56,401 UTC [730984] DEBUG esmvalcore.esgf._download:535 All required data is available locally, not downloading anything. 2023-12-27 15:34:56,401 UTC [730984] DEBUG esmvalcore.preprocessor:364 Running preprocessor step fix_file 2023-12-27 15:34:56,402 UTC [730984] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'fix_file' on the data LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/ICHEC/EC-EARTH/historical/day/atmos/day/r1i1p1/v20121115/pr/pr_day_EC-EARTH_historical_r1i1p1_19900101-19991231.nc') loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/ICHEC/EC-EARTH/historical/day/atmos/day/r1i1p1/v20121115/pr/pr_day_EC-EARTH_historical_r1i1p1_19900101-19991231.nc')] with function argument(s) output_dir = PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/preproc/fixed_files/pr_day_CMIP5_EC-EARTH_historical_r1i1p1_v20121115_'), add_unique_suffix = True, session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': None, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': True, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': False, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), preprocessor = 'preproc', reference_dataset = 'EC-EARTH', mip = 'day', dataset = 'EC-EARTH', project = 'CMIP5', exp = 'historical', ensemble = 'r1i1p1', short_name = 'pr', timerange = '1997/1997', variable_group = 'pr', diagnostic = 'example', recipe_dataset_index = 0, alias = 'EC-EARTH', product = ['output1', 'output2'], institute = ['ICHEC'], original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = ['atmos'], frequency = 'day', version = 'v20121115' 2023-12-27 15:34:56,416 UTC [730984] DEBUG esmvalcore.preprocessor:364 Running preprocessor step load 2023-12-27 15:34:56,416 UTC [730984] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'load' on the data LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/ICHEC/EC-EARTH/historical/day/atmos/day/r1i1p1/v20121115/pr/pr_day_EC-EARTH_historical_r1i1p1_19900101-19991231.nc') loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/ICHEC/EC-EARTH/historical/day/atmos/day/r1i1p1/v20121115/pr/pr_day_EC-EARTH_historical_r1i1p1_19900101-19991231.nc')] with function argument(s) ignore_warnings = None 2023-12-27 15:34:56,417 UTC [730984] DEBUG esmvalcore.preprocessor._io:145 Loading: /projects/NS9560K-datalake/ESGF/cmip5/output1/ICHEC/EC-EARTH/historical/day/atmos/day/r1i1p1/v20121115/pr/pr_day_EC-EARTH_historical_r1i1p1_19900101-19991231.nc 2023-12-27 15:35:54,061 UTC [730984] DEBUG esmvalcore.preprocessor._io:176 Done with loading /projects/NS9560K-datalake/ESGF/cmip5/output1/ICHEC/EC-EARTH/historical/day/atmos/day/r1i1p1/v20121115/pr/pr_day_EC-EARTH_historical_r1i1p1_19900101-19991231.nc 2023-12-27 15:35:54,062 UTC [730984] DEBUG esmvalcore.preprocessor:364 Running preprocessor step fix_metadata 2023-12-27 15:35:54,065 UTC [730984] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'fix_metadata' on the data [] loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/ICHEC/EC-EARTH/historical/day/atmos/day/r1i1p1/v20121115/pr/pr_day_EC-EARTH_historical_r1i1p1_19900101-19991231.nc')] with function argument(s) check_level = , session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': None, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': True, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': False, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), preprocessor = 'preproc', reference_dataset = 'EC-EARTH', mip = 'day', dataset = 'EC-EARTH', project = 'CMIP5', exp = 'historical', ensemble = 'r1i1p1', short_name = 'pr', timerange = '1997/1997', variable_group = 'pr', diagnostic = 'example', recipe_dataset_index = 0, alias = 'EC-EARTH', product = ['output1', 'output2'], institute = ['ICHEC'], original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = ['atmos'], frequency = 'day', version = 'v20121115' 2023-12-27 15:35:54,070 UTC [730984] DEBUG esmvalcore.preprocessor:364 Running preprocessor step concatenate 2023-12-27 15:35:54,071 UTC [730984] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'concatenate' on the data [] loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/ICHEC/EC-EARTH/historical/day/atmos/day/r1i1p1/v20121115/pr/pr_day_EC-EARTH_historical_r1i1p1_19900101-19991231.nc')] with function argument(s) check_level = 2023-12-27 15:35:54,071 UTC [730984] DEBUG esmvalcore.preprocessor:364 Running preprocessor step cmor_check_metadata 2023-12-27 15:35:54,072 UTC [730984] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'cmor_check_metadata' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/ICHEC/EC-EARTH/historical/day/atmos/day/r1i1p1/v20121115/pr/pr_day_EC-EARTH_historical_r1i1p1_19900101-19991231.nc')] with function argument(s) check_level = , cmor_table = 'CMIP5', mip = 'day', frequency = 'day', short_name = 'pr' 2023-12-27 15:35:54,076 UTC [730984] DEBUG esmvalcore.preprocessor:364 Running preprocessor step clip_timerange 2023-12-27 15:35:54,077 UTC [730984] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'clip_timerange' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/ICHEC/EC-EARTH/historical/day/atmos/day/r1i1p1/v20121115/pr/pr_day_EC-EARTH_historical_r1i1p1_19900101-19991231.nc')] with function argument(s) timerange = '1997/1997' 2023-12-27 15:35:54,090 UTC [730984] DEBUG esmvalcore.preprocessor:364 Running preprocessor step fix_data 2023-12-27 15:35:54,091 UTC [730984] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'fix_data' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/ICHEC/EC-EARTH/historical/day/atmos/day/r1i1p1/v20121115/pr/pr_day_EC-EARTH_historical_r1i1p1_19900101-19991231.nc')] with function argument(s) check_level = , session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': None, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': True, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': False, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), preprocessor = 'preproc', reference_dataset = 'EC-EARTH', mip = 'day', dataset = 'EC-EARTH', project = 'CMIP5', exp = 'historical', ensemble = 'r1i1p1', short_name = 'pr', timerange = '1997/1997', variable_group = 'pr', diagnostic = 'example', recipe_dataset_index = 0, alias = 'EC-EARTH', product = ['output1', 'output2'], institute = ['ICHEC'], original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = ['atmos'], frequency = 'day', version = 'v20121115' 2023-12-27 15:35:54,091 UTC [730984] DEBUG esmvalcore.preprocessor:364 Running preprocessor step cmor_check_data 2023-12-27 15:35:54,092 UTC [730984] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'cmor_check_data' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/ICHEC/EC-EARTH/historical/day/atmos/day/r1i1p1/v20121115/pr/pr_day_EC-EARTH_historical_r1i1p1_19900101-19991231.nc')] with function argument(s) check_level = , cmor_table = 'CMIP5', mip = 'day', frequency = 'day', short_name = 'pr' 2023-12-27 15:35:54,092 UTC [730984] DEBUG esmvalcore.preprocessor:364 Running preprocessor step add_supplementary_variables 2023-12-27 15:35:54,093 UTC [730984] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'add_supplementary_variables' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/ICHEC/EC-EARTH/historical/day/atmos/day/r1i1p1/v20121115/pr/pr_day_EC-EARTH_historical_r1i1p1_19900101-19991231.nc')] with function argument(s) supplementary_cubes = [] 2023-12-27 15:35:54,093 UTC [730984] DEBUG esmvalcore.preprocessor:364 Running preprocessor step save 2023-12-27 15:35:54,094 UTC [730984] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'save' on the data [] loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/ICHEC/EC-EARTH/historical/day/atmos/day/r1i1p1/v20121115/pr/pr_day_EC-EARTH_historical_r1i1p1_19900101-19991231.nc')] with function argument(s) compress = False, filename = PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/preproc/example/pr/CMIP5_EC-EARTH_day_historical_r1i1p1_pr_1997-1997.nc') 2023-12-27 15:35:54,095 UTC [730984] DEBUG esmvalcore.preprocessor._io:452 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 365; latitude: 160; longitude: 320) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean (interval: 3 hours) Attributes: CDI 'Climate Data Interface version 1.4.4 (http://code.zmaw.de/projects/cdi ...' CDO 'Climate Data Operators version 1.4.4 (http://code.zmaw.de/projects/cdo ...' Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_EC-EARTH_historical_r0i0p0.nc ...' branch_time 2125.0 cmor_version '2.8.0' contact 'Alastair McKinstry ' experiment 'historical' experiment_id 'historical' forcing 'Nat,Ant' frequency 'day' grid_type 'gaussian' initialization_method 1 institute_id 'ICHEC' institution 'EC-Earth (European Earth System Model)' model_id 'EC-EARTH' modeling_realm 'atmos' original_name 'TP=LSP+CP' parent_experiment 'pre-industrial control' parent_experiment_id 'piControl' parent_experiment_rip 'r1i1p1' physics_version 1 product 'output' project_id 'CMIP5' realization 1 references 'Model described by Hazeleger et al. (Bull. Amer. Meteor. Soc., 2010, 91, ...' table_id 'Table day (26 July 2011) 0640f2bc62aad4ebefa8ecc119311890' title 'EC-EARTH model output prepared for CMIP5 historical' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/preproc/example/pr/CMIP5_EC-EARTH_day_historical_r1i1p1_pr_1997-1997.nc 2023-12-27 15:36:10,838 UTC [730984] INFO esmvalcore._task:266 Successfully completed task example/pr (priority 1) in 0:01:15.610224 2023-12-27 15:36:10,882 UTC [730848] INFO esmvalcore._task:793 Progress: 0 tasks running, 1 tasks waiting for ancestors, 1/2 done 2023-12-27 15:36:10,885 UTC [730985] INFO esmvalcore._task:261 Starting task example/main in process [730985] 2023-12-27 15:36:10,887 UTC [730985] INFO esmvalcore._task:502 Running command ['/projects/NS2980K/conda/esmvaltool2.10.0/bin/Rscript', '/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvaltool/diag_scripts/examples/diagnostic.R', '/projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/run/example/main/settings.yml'] 2023-12-27 15:36:10,887 UTC [730985] DEBUG esmvalcore._task:503 in environment {'diag_scripts': '/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvaltool/diag_scripts'} 2023-12-27 15:36:10,888 UTC [730985] DEBUG esmvalcore._task:505 in current working directory: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/run/example/main 2023-12-27 15:36:10,888 UTC [730985] INFO esmvalcore._task:506 Writing output to /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/work/example/main 2023-12-27 15:36:10,888 UTC [730985] INFO esmvalcore._task:507 Writing plots to /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/plots/example/main 2023-12-27 15:36:10,888 UTC [730985] INFO esmvalcore._task:508 Writing log to /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/run/example/main/log.txt 2023-12-27 15:36:10,888 UTC [730985] INFO esmvalcore._task:518 To re-run this diagnostic script, run: cd /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/run/example/main; diag_scripts="/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvaltool/diag_scripts" /projects/NS2980K/conda/esmvaltool2.10.0/bin/Rscript /projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvaltool/diag_scripts/examples/diagnostic.R /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/run/example/main/settings.yml 2023-12-27 15:36:10,982 UTC [730848] INFO esmvalcore._task:793 Progress: 1 tasks running, 0 tasks waiting for ancestors, 1/2 done 2023-12-27 15:36:13,958 UTC [730985] INFO esmvalcore._task:128 Maximum memory used (estimate): 0.1 GB 2023-12-27 15:36:13,959 UTC [730985] INFO esmvalcore._task:130 Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. 2023-12-27 15:36:13,962 UTC [730985] DEBUG esmvalcore._task:591 Script examples/diagnostic.R completed successfully 2023-12-27 15:36:13,963 UTC [730985] WARNING esmvalcore._task:604 No provenance information was written to /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/run/example/main/diagnostic_provenance.yml. Unable to record provenance for files created by diagnostic script examples/diagnostic.R in task example/main 2023-12-27 15:36:13,963 UTC [730985] INFO esmvalcore._task:266 Successfully completed task example/main (priority 0) in 0:00:03.077457 2023-12-27 15:36:13,985 UTC [730848] INFO esmvalcore._task:793 Progress: 0 tasks running, 0 tasks waiting for ancestors, 2/2 done 2023-12-27 15:36:13,985 UTC [730848] INFO esmvalcore._task:798 Successfully completed all tasks. 2023-12-27 15:36:14,011 UTC [730848] INFO esmvalcore._recipe.recipe:1076 Wrote recipe with version numbers and wildcards to: file:///projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/run/recipe_r_filled.yml 2023-12-27 15:36:14,074 UTC [730848] INFO esmvalcore.experimental.recipe_output:244 Wrote recipe output to: file:///projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_r_20231227_153454/index.html 2023-12-27 15:36:14,075 UTC [730848] INFO esmvalcore._main:133 Ending the Earth System Model Evaluation Tool at time: 2023-12-27 15:36:14 UTC 2023-12-27 15:36:14,075 UTC [730848] INFO esmvalcore._main:136 Time for running the recipe was: 0:01:19.120111 2023-12-27 15:36:15,018 UTC [730848] INFO esmvalcore._task:128 Maximum memory used (estimate): 1.0 GB 2023-12-27 15:36:15,018 UTC [730848] INFO esmvalcore._task:130 Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. 2023-12-27 15:36:15,019 UTC [730848] INFO esmvalcore._main:470 Removing `preproc` directory containing preprocessed data 2023-12-27 15:36:15,020 UTC [730848] INFO esmvalcore._main:473 If this data is further needed, then set `remove_preproc_dir` to `false` in your user configuration file 2023-12-27 15:36:15,029 UTC [730848] INFO esmvalcore._main:450 Run was successful