2023-07-04 11:36:33,017 UTC [1346112] INFO esmvalcore._main:495 ______________________________________________________________________ _____ ____ __ ____ __ _ _____ _ | ____/ ___|| \/ \ \ / /_ _| |_ _|__ ___ | | | _| \___ \| |\/| |\ \ / / _` | | | |/ _ \ / _ \| | | |___ ___) | | | | \ V / (_| | | | | (_) | (_) | | |_____|____/|_| |_| \_/ \__,_|_| |_|\___/ \___/|_| ______________________________________________________________________ ESMValTool - Earth System Model Evaluation Tool. http://www.esmvaltool.org CORE DEVELOPMENT TEAM AND CONTACTS: Birgit Hassler (Co-PI; DLR, Germany - birgit.hassler@dlr.de) Alistair Sellar (Co-PI; Met Office, UK - alistair.sellar@metoffice.gov.uk) Bouwe Andela (Netherlands eScience Center, The Netherlands - b.andela@esciencecenter.nl) Lee de Mora (PML, UK - ledm@pml.ac.uk) Niels Drost (Netherlands eScience Center, The Netherlands - n.drost@esciencecenter.nl) Veronika Eyring (DLR, Germany - veronika.eyring@dlr.de) Bettina Gier (UBremen, Germany - gier@uni-bremen.de) Remi Kazeroni (DLR, Germany - remi.kazeroni@dlr.de) Nikolay Koldunov (AWI, Germany - nikolay.koldunov@awi.de) Axel Lauer (DLR, Germany - axel.lauer@dlr.de) Saskia Loosveldt-Tomas (BSC, Spain - saskia.loosveldt@bsc.es) Ruth Lorenz (ETH Zurich, Switzerland - ruth.lorenz@env.ethz.ch) Benjamin Mueller (LMU, Germany - b.mueller@iggf.geo.uni-muenchen.de) Valeriu Predoi (URead, UK - valeriu.predoi@ncas.ac.uk) Mattia Righi (DLR, Germany - mattia.righi@dlr.de) Manuel Schlund (DLR, Germany - manuel.schlund@dlr.de) Breixo Solino Fernandez (DLR, Germany - breixo.solinofernandez@dlr.de) Javier Vegas-Regidor (BSC, Spain - javier.vegas@bsc.es) Klaus Zimmermann (SMHI, Sweden - klaus.zimmermann@smhi.se) For further help, please read the documentation at http://docs.esmvaltool.org. Have fun! 2023-07-04 11:36:33,018 UTC [1346112] INFO esmvalcore._main:496 Package versions 2023-07-04 11:36:33,018 UTC [1346112] INFO esmvalcore._main:497 ---------------- 2023-07-04 11:36:33,018 UTC [1346112] INFO esmvalcore._main:498 ESMValCore: 2.8.0 2023-07-04 11:36:33,018 UTC [1346112] INFO esmvalcore._main:500 ESMValTool: 2.8.0 2023-07-04 11:36:33,018 UTC [1346112] INFO esmvalcore._main:501 ---------------- 2023-07-04 11:36:33,018 UTC [1346112] INFO esmvalcore._main:502 Using config file /nird/home/tomast/.esmvaltool/config-user.yml 2023-07-04 11:36:33,018 UTC [1346112] INFO esmvalcore._main:503 Writing program log files to: /projects/NS2980K/www/tomast/esmvaltool2.8.0/recipe_ocean_arctic_bgc_20230704_113632/run/main_log.txt /projects/NS2980K/www/tomast/esmvaltool2.8.0/recipe_ocean_arctic_bgc_20230704_113632/run/main_log_debug.txt 2023-07-04 11:36:33,345 UTC [1346112] INFO esmvalcore.config._esgf_pyclient:116 Using default ESGF configuration, configuration file /nird/home/tomast/.esmvaltool/esgf-pyclient.yml not present. 2023-07-04 11:36:33,345 UTC [1346112] WARNING esmvalcore.config._esgf_pyclient:181 ESGF credentials missing, only data that is accessible without logging in will be available. See https://esgf.github.io/esgf-user-support/user_guide.html for instructions on how to create an account if you do not have one yet. Next, configure your system so esmvaltool can use your credentials. This can be done using the keyring package, or you can just enter them in /nird/home/tomast/.esmvaltool/esgf-pyclient.yml. keyring ======= First install the keyring package (requires a supported backend, see https://pypi.org/project/keyring/): $ pip install keyring Next, set your username and password by running the commands: $ keyring set ESGF hostname $ keyring set ESGF username $ keyring set ESGF password To check that you entered your credentials correctly, run: $ keyring get ESGF hostname $ keyring get ESGF username $ keyring get ESGF password configuration file ================== You can store the hostname, username, and password or your OpenID account in a plain text in the file /nird/home/tomast/.esmvaltool/esgf-pyclient.yml like this: logon: hostname: "your-hostname" username: "your-username" password: "your-password" or your can configure an interactive log in: logon: interactive: true Note that storing your password in plain text in the configuration file is less secure. On shared systems, make sure the permissions of the file are set so only you can read it, i.e. $ ls -l /nird/home/tomast/.esmvaltool/esgf-pyclient.yml shows permissions -rw-------. 2023-07-04 11:36:35,763 UTC [1346112] INFO esmvalcore._main:85 Starting the Earth System Model Evaluation Tool at time: 2023-07-04 11:36:35 UTC 2023-07-04 11:36:35,763 UTC [1346112] INFO esmvalcore._main:89 ---------------------------------------------------------------------- 2023-07-04 11:36:35,763 UTC [1346112] INFO esmvalcore._main:90 RECIPE = /projects/NS2980K/home/tomast/ESMValTool/noresmvaltool/esmvaltool/new_recipes/recipe_ocean_arctic_bgc.yml 2023-07-04 11:36:35,763 UTC [1346112] INFO esmvalcore._main:91 RUNDIR = /projects/NS2980K/www/tomast/esmvaltool2.8.0/recipe_ocean_arctic_bgc_20230704_113632/run 2023-07-04 11:36:35,764 UTC [1346112] INFO esmvalcore._main:92 WORKDIR = /projects/NS2980K/www/tomast/esmvaltool2.8.0/recipe_ocean_arctic_bgc_20230704_113632/work 2023-07-04 11:36:35,764 UTC [1346112] INFO esmvalcore._main:93 PREPROCDIR = /projects/NS2980K/www/tomast/esmvaltool2.8.0/recipe_ocean_arctic_bgc_20230704_113632/preproc 2023-07-04 11:36:35,764 UTC [1346112] INFO esmvalcore._main:94 PLOTDIR = /projects/NS2980K/www/tomast/esmvaltool2.8.0/recipe_ocean_arctic_bgc_20230704_113632/plots 2023-07-04 11:36:35,764 UTC [1346112] INFO esmvalcore._main:95 ---------------------------------------------------------------------- 2023-07-04 11:36:35,764 UTC [1346112] INFO esmvalcore._main:98 Running tasks using at most 256 processes 2023-07-04 11:36:35,764 UTC [1346112] INFO esmvalcore._main:100 If your system hangs during execution, it may not have enough memory for keeping this number of tasks in memory. 2023-07-04 11:36:35,764 UTC [1346112] INFO esmvalcore._main:103 If you experience memory problems, try reducing 'max_parallel_tasks' in your user configuration file. 2023-07-04 11:36:35,765 UTC [1346112] DEBUG esmvalcore._recipe.check:52 Checking recipe against schema /diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_recipe/recipe_schema.yml 2023-07-04 11:36:35,797 UTC [1346112] DEBUG esmvalcore._recipe.to_datasets:344 Populating list of datasets for variable sic in diagnostic diag_ice_SHW 2023-07-04 11:36:35,798 UTC [1346112] DEBUG esmvalcore._recipe.to_datasets:368 Found Dataset: sic, OImon, CMIP5, NorESM1-M, historical, r1i1p1 2023-07-04 11:36:35,798 UTC [1346112] DEBUG esmvalcore._recipe.to_datasets:368 Found Dataset: sic, OImon, CMIP5, NorESM1-ME, historical, r1i1p1 2023-07-04 11:36:35,798 UTC [1346112] DEBUG esmvalcore._recipe.to_datasets:344 Populating list of datasets for variable sic in diagnostic diag_ice_SHS 2023-07-04 11:36:35,798 UTC [1346112] DEBUG esmvalcore._recipe.to_datasets:368 Found Dataset: sic, OImon, CMIP5, NorESM1-M, historical, r1i1p1 2023-07-04 11:36:35,799 UTC [1346112] DEBUG esmvalcore._recipe.to_datasets:368 Found Dataset: sic, OImon, CMIP5, NorESM1-ME, historical, r1i1p1 2023-07-04 11:36:35,799 UTC [1346112] DEBUG esmvalcore._recipe.recipe:1014 Retrieving diagnostics from recipe 2023-07-04 11:36:35,799 UTC [1346112] DEBUG esmvalcore._recipe.recipe:1042 Setting script for diagnostic diag_ice_SHW 2023-07-04 11:36:35,799 UTC [1346112] DEBUG esmvalcore._recipe.recipe:1042 Setting script for diagnostic diag_ice_SHS 2023-07-04 11:36:35,799 UTC [1346112] INFO esmvalcore._recipe.recipe:1232 Creating tasks from recipe 2023-07-04 11:36:35,799 UTC [1346112] INFO esmvalcore._recipe.recipe:1241 Creating tasks for diagnostic diag_ice_SHW 2023-07-04 11:36:35,799 UTC [1346112] INFO esmvalcore._recipe.recipe:1163 Creating diagnostic task diag_ice_SHW/Global_seaice_timeseries 2023-07-04 11:36:35,799 UTC [1346112] INFO esmvalcore._recipe.recipe:1216 Creating preprocessor task diag_ice_SHW/sic 2023-07-04 11:36:35,800 UTC [1346112] INFO esmvalcore._recipe.recipe:849 Creating preprocessor 'timeseries_SHW_ice_extent' task for variable 'sic' 2023-07-04 11:36:35,800 UTC [1346112] DEBUG esmvalcore.config._config:48 Loading extra facets from /diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/config/extra_facets/cmip5-fx.yml 2023-07-04 11:36:35,801 UTC [1346112] DEBUG esmvalcore.config._config:48 Loading extra facets from /diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/config/extra_facets/cmip5-institutes.yml 2023-07-04 11:36:35,830 UTC [1346112] DEBUG esmvalcore.config._config:48 Loading extra facets from /diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/config/extra_facets/cmip5-product.yml 2023-07-04 11:36:35,839 UTC [1346112] DEBUG esmvalcore.local:458 Looking for files matching: /projects/NS9034K/CMIP5/output1/NCC/NorESM1-M/historical/mon/ocean/OImon/r1i1p1/*/sic/sic_OImon_NorESM1-M_historical_r1i1p1*.nc /projects/NS9034K/CMIP5/output1/NCC/NorESM1-M/historical/mon/seaIce/OImon/r1i1p1/*/sic/sic_OImon_NorESM1-M_historical_r1i1p1*.nc /projects/NS9034K/CMIP5/output2/NCC/NorESM1-M/historical/mon/ocean/OImon/r1i1p1/*/sic/sic_OImon_NorESM1-M_historical_r1i1p1*.nc /projects/NS9034K/CMIP5/output2/NCC/NorESM1-M/historical/mon/seaIce/OImon/r1i1p1/*/sic/sic_OImon_NorESM1-M_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/NCC/NorESM1-M/historical/mon/ocean/OImon/r1i1p1/*/sic/sic_OImon_NorESM1-M_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/NCC/NorESM1-M/historical/mon/seaIce/OImon/r1i1p1/*/sic/sic_OImon_NorESM1-M_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output2/NCC/NorESM1-M/historical/mon/ocean/OImon/r1i1p1/*/sic/sic_OImon_NorESM1-M_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output2/NCC/NorESM1-M/historical/mon/seaIce/OImon/r1i1p1/*/sic/sic_OImon_NorESM1-M_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/NCC/NorESM1-M/historical/mon/ocean/OImon/r1i1p1/*/sic/sic_OImon_NorESM1-M_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/NCC/NorESM1-M/historical/mon/seaIce/OImon/r1i1p1/*/sic/sic_OImon_NorESM1-M_historical_r1i1p1*.nc 2023-07-04 11:36:35,860 UTC [1346112] DEBUG esmvalcore._recipe.recipe:446 Using input files for variable sic of dataset NorESM1-M: /projects/NS9034K/CMIP5/output1/NCC/NorESM1-M/historical/mon/seaIce/OImon/r1i1p1/v20120227/sic/sic_OImon_NorESM1-M_historical_r1i1p1_185001-200512.nc 2023-07-04 11:36:35,860 UTC [1346112] INFO esmvalcore._recipe.recipe:717 Found input files for Dataset: sic, OImon, CMIP5, NorESM1-M, historical, r1i1p1, v20120227 2023-07-04 11:36:35,861 UTC [1346112] DEBUG esmvalcore.local:458 Looking for files matching: /projects/NS9034K/CMIP5/output1/NCC/NorESM1-ME/historical/mon/ocean/OImon/r1i1p1/*/sic/sic_OImon_NorESM1-ME_historical_r1i1p1*.nc /projects/NS9034K/CMIP5/output1/NCC/NorESM1-ME/historical/mon/seaIce/OImon/r1i1p1/*/sic/sic_OImon_NorESM1-ME_historical_r1i1p1*.nc /projects/NS9034K/CMIP5/output2/NCC/NorESM1-ME/historical/mon/ocean/OImon/r1i1p1/*/sic/sic_OImon_NorESM1-ME_historical_r1i1p1*.nc /projects/NS9034K/CMIP5/output2/NCC/NorESM1-ME/historical/mon/seaIce/OImon/r1i1p1/*/sic/sic_OImon_NorESM1-ME_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/NCC/NorESM1-ME/historical/mon/ocean/OImon/r1i1p1/*/sic/sic_OImon_NorESM1-ME_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/NCC/NorESM1-ME/historical/mon/seaIce/OImon/r1i1p1/*/sic/sic_OImon_NorESM1-ME_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output2/NCC/NorESM1-ME/historical/mon/ocean/OImon/r1i1p1/*/sic/sic_OImon_NorESM1-ME_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output2/NCC/NorESM1-ME/historical/mon/seaIce/OImon/r1i1p1/*/sic/sic_OImon_NorESM1-ME_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/NCC/NorESM1-ME/historical/mon/ocean/OImon/r1i1p1/*/sic/sic_OImon_NorESM1-ME_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/NCC/NorESM1-ME/historical/mon/seaIce/OImon/r1i1p1/*/sic/sic_OImon_NorESM1-ME_historical_r1i1p1*.nc 2023-07-04 11:36:35,863 UTC [1346112] DEBUG esmvalcore._recipe.recipe:446 Using input files for variable sic of dataset NorESM1-ME: /projects/NS9034K/CMIP5/output1/NCC/NorESM1-ME/historical/mon/seaIce/OImon/r1i1p1/v20120225/sic/sic_OImon_NorESM1-ME_historical_r1i1p1_185001-200512.nc 2023-07-04 11:36:35,863 UTC [1346112] INFO esmvalcore._recipe.recipe:717 Found input files for Dataset: sic, OImon, CMIP5, NorESM1-ME, historical, r1i1p1, v20120225 2023-07-04 11:36:35,865 UTC [1346112] INFO esmvalcore._recipe.recipe:873 PreprocessingTask diag_ice_SHW/sic created. 2023-07-04 11:36:35,865 UTC [1346112] DEBUG esmvalcore._recipe.recipe:874 PreprocessingTask diag_ice_SHW/sic will create the files: /projects/NS2980K/www/tomast/esmvaltool2.8.0/recipe_ocean_arctic_bgc_20230704_113632/preproc/diag_ice_SHW/sic/CMIP5_NorESM1-ME_OImon_historical_r1i1p1_sic_1989-2004.nc /projects/NS2980K/www/tomast/esmvaltool2.8.0/recipe_ocean_arctic_bgc_20230704_113632/preproc/diag_ice_SHW/sic/CMIP5_NorESM1-M_OImon_historical_r1i1p1_sic_1989-2004.nc 2023-07-04 11:36:35,865 UTC [1346112] INFO esmvalcore._recipe.recipe:1241 Creating tasks for diagnostic diag_ice_SHS 2023-07-04 11:36:35,865 UTC [1346112] INFO esmvalcore._recipe.recipe:1163 Creating diagnostic task diag_ice_SHS/Global_seaice_timeseries 2023-07-04 11:36:36,838 UTC [1346112] INFO esmvalcore._task:126 Maximum memory used (estimate): 0.5 GB 2023-07-04 11:36:36,839 UTC [1346112] INFO esmvalcore._task:128 Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. 2023-07-04 11:36:36,839 UTC [1346112] ERROR esmvalcore._main:533 Program terminated abnormally, see stack trace below for more information: Traceback (most recent call last): File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_main.py", line 521, in run fire.Fire(ESMValTool()) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/fire/core.py", line 141, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/fire/core.py", line 475, in _Fire component, remaining_args = _CallAndUpdateTrace( File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_main.py", line 408, in run self._run(recipe, session) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_main.py", line 450, in _run process_recipe(recipe_file=recipe, session=session) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_main.py", line 119, in process_recipe recipe = read_recipe_file(recipe_file, session) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_recipe/recipe.py", line 82, in read_recipe_file return Recipe(raw_recipe, session, recipe_file=filename) File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_recipe/recipe.py", line 917, in __init__ self.tasks = self.initialize_tasks() File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_recipe/recipe.py", line 1279, in initialize_tasks tasks = self._create_tasks() File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_recipe/recipe.py", line 1244, in _create_tasks new_tasks = self._create_diagnostic_tasks(diagnostic_name, File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_recipe/recipe.py", line 1164, in _create_diagnostic_tasks task = DiagnosticTask( File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_task.py", line 343, in __init__ self.cmd = self._initialize_cmd() File "/diagnostics/esmvaltool/2.8.0/lib/python3.10/site-packages/esmvalcore/_task.py", line 356, in _initialize_cmd raise DiagnosticError(f"{err_msg}: file does not exist.") esmvalcore._task.DiagnosticError: Cannot execute script '/projects/NS2980K/home/tomast/ESMValTool/noresmvaltool/esmvaltool/diag_scripts/diagnostic_polar.py' (/projects/NS2980K/home/tomast/ESMValTool/noresmvaltool/esmvaltool/diag_scripts/diagnostic_polar.py): file does not exist. 2023-07-04 11:36:36,894 UTC [1346112] INFO esmvalcore._main:537 If you have a question or need help, please start a new discussion on https://github.com/ESMValGroup/ESMValTool/discussions If you suspect this is a bug, please open an issue on https://github.com/ESMValGroup/ESMValTool/issues To make it easier to find out what the problem is, please consider attaching the files run/recipe_*.yml and run/main_log_debug.txt from the output directory.