2024-03-26 14:02:19,157 UTC [2602179] INFO esmvalcore._main:492 ______________________________________________________________________ _____ ____ __ ____ __ _ _____ _ | ____/ ___|| \/ \ \ / /_ _| |_ _|__ ___ | | | _| \___ \| |\/| |\ \ / / _` | | | |/ _ \ / _ \| | | |___ ___) | | | | \ V / (_| | | | | (_) | (_) | | |_____|____/|_| |_| \_/ \__,_|_| |_|\___/ \___/|_| ______________________________________________________________________ ESMValTool - Earth System Model Evaluation Tool. http://www.esmvaltool.org CORE DEVELOPMENT TEAM AND CONTACTS: Birgit Hassler (Co-PI; DLR, Germany - birgit.hassler@dlr.de) Alistair Sellar (Co-PI; Met Office, UK - alistair.sellar@metoffice.gov.uk) Bouwe Andela (Netherlands eScience Center, The Netherlands - b.andela@esciencecenter.nl) Lee de Mora (PML, UK - ledm@pml.ac.uk) Niels Drost (Netherlands eScience Center, The Netherlands - n.drost@esciencecenter.nl) Veronika Eyring (DLR, Germany - veronika.eyring@dlr.de) Bettina Gier (UBremen, Germany - gier@uni-bremen.de) Remi Kazeroni (DLR, Germany - remi.kazeroni@dlr.de) Nikolay Koldunov (AWI, Germany - nikolay.koldunov@awi.de) Axel Lauer (DLR, Germany - axel.lauer@dlr.de) Saskia Loosveldt-Tomas (BSC, Spain - saskia.loosveldt@bsc.es) Ruth Lorenz (ETH Zurich, Switzerland - ruth.lorenz@env.ethz.ch) Benjamin Mueller (LMU, Germany - b.mueller@iggf.geo.uni-muenchen.de) Valeriu Predoi (URead, UK - valeriu.predoi@ncas.ac.uk) Mattia Righi (DLR, Germany - mattia.righi@dlr.de) Manuel Schlund (DLR, Germany - manuel.schlund@dlr.de) Breixo Solino Fernandez (DLR, Germany - breixo.solinofernandez@dlr.de) Javier Vegas-Regidor (BSC, Spain - javier.vegas@bsc.es) Klaus Zimmermann (SMHI, Sweden - klaus.zimmermann@smhi.se) For further help, please read the documentation at http://docs.esmvaltool.org. Have fun! 2024-03-26 14:02:19,157 UTC [2602179] INFO esmvalcore._main:493 Package versions 2024-03-26 14:02:19,157 UTC [2602179] INFO esmvalcore._main:494 ---------------- 2024-03-26 14:02:19,157 UTC [2602179] INFO esmvalcore._main:495 ESMValCore: 2.10.0 2024-03-26 14:02:19,158 UTC [2602179] INFO esmvalcore._main:497 ESMValTool: 2.10.0 2024-03-26 14:02:19,158 UTC [2602179] INFO esmvalcore._main:498 ---------------- 2024-03-26 14:02:19,158 UTC [2602179] INFO esmvalcore._main:499 Using config file /nird/home/tomast/.esmvaltool/config-user.yml 2024-03-26 14:02:19,158 UTC [2602179] INFO esmvalcore._main:500 Writing program log files to: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_seaice_NorESM2_20240326_140219/run/main_log.txt /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_seaice_NorESM2_20240326_140219/run/main_log_debug.txt 2024-03-26 14:02:20,732 UTC [2602179] INFO esmvalcore.config._esgf_pyclient:116 Using default ESGF configuration, configuration file /nird/home/tomast/.esmvaltool/esgf-pyclient.yml not present. 2024-03-26 14:02:20,733 UTC [2602179] WARNING esmvalcore.config._esgf_pyclient:181 ESGF credentials missing, only data that is accessible without logging in will be available. See https://esgf.github.io/esgf-user-support/user_guide.html for instructions on how to create an account if you do not have one yet. Next, configure your system so esmvaltool can use your credentials. This can be done using the keyring package, or you can just enter them in /nird/home/tomast/.esmvaltool/esgf-pyclient.yml. keyring ======= First install the keyring package (requires a supported backend, see https://pypi.org/project/keyring/): $ pip install keyring Next, set your username and password by running the commands: $ keyring set ESGF hostname $ keyring set ESGF username $ keyring set ESGF password To check that you entered your credentials correctly, run: $ keyring get ESGF hostname $ keyring get ESGF username $ keyring get ESGF password configuration file ================== You can store the hostname, username, and password or your OpenID account in a plain text in the file /nird/home/tomast/.esmvaltool/esgf-pyclient.yml like this: logon: hostname: "your-hostname" username: "your-username" password: "your-password" or your can configure an interactive log in: logon: interactive: true Note that storing your password in plain text in the configuration file is less secure. On shared systems, make sure the permissions of the file are set so only you can read it, i.e. $ ls -l /nird/home/tomast/.esmvaltool/esgf-pyclient.yml shows permissions -rw-------. 2024-03-26 14:02:22,290 UTC [2602179] INFO esmvalcore._main:91 Starting the Earth System Model Evaluation Tool at time: 2024-03-26 14:02:22 UTC 2024-03-26 14:02:22,291 UTC [2602179] INFO esmvalcore._main:95 ---------------------------------------------------------------------- 2024-03-26 14:02:22,291 UTC [2602179] INFO esmvalcore._main:96 RECIPE = /projects/NS2980K/home/tomast/ESMValTool/recipes-2.10.0/standard_recipes_NorESM/seaice/recipe_seaice_NorESM2.yml 2024-03-26 14:02:22,291 UTC [2602179] INFO esmvalcore._main:97 RUNDIR = /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_seaice_NorESM2_20240326_140219/run 2024-03-26 14:02:22,291 UTC [2602179] INFO esmvalcore._main:98 WORKDIR = /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_seaice_NorESM2_20240326_140219/work 2024-03-26 14:02:22,291 UTC [2602179] INFO esmvalcore._main:99 PREPROCDIR = /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_seaice_NorESM2_20240326_140219/preproc 2024-03-26 14:02:22,291 UTC [2602179] INFO esmvalcore._main:100 PLOTDIR = /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_seaice_NorESM2_20240326_140219/plots 2024-03-26 14:02:22,291 UTC [2602179] INFO esmvalcore._main:101 ---------------------------------------------------------------------- 2024-03-26 14:02:22,291 UTC [2602179] INFO esmvalcore._main:104 Running tasks using at most 16 processes 2024-03-26 14:02:22,291 UTC [2602179] INFO esmvalcore._main:106 If your system hangs during execution, it may not have enough memory for keeping this number of tasks in memory. 2024-03-26 14:02:22,291 UTC [2602179] INFO esmvalcore._main:109 If you experience memory problems, try reducing 'max_parallel_tasks' in your user configuration file. 2024-03-26 14:02:22,291 UTC [2602179] WARNING esmvalcore.config._dask:18 Using the Dask basic scheduler. This may lead to slow computations and out-of-memory errors. Note that the basic scheduler may still be the best choice for preprocessor functions that are not lazy. In that case, you can safely ignore this warning. See https://docs.esmvaltool.org/projects/ESMValCore/en/latest/quickstart/configure.html#dask-distributed-configuration for more information. 2024-03-26 14:02:22,292 UTC [2602179] DEBUG esmvalcore._recipe.check:54 Checking recipe against schema /projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/_recipe/recipe_schema.yml 2024-03-26 14:02:22,303 UTC [2602179] DEBUG esmvalcore._recipe.to_datasets:343 Populating list of datasets for variable sithick in diagnostic NegativeSeaIceFeedback 2024-03-26 14:02:22,303 UTC [2602179] DEBUG esmvalcore._recipe.to_datasets:367 Found Dataset: sithick, SImon, CMIP6, NorESM2-LM, historical, r1i1p1f1 2024-03-26 14:02:22,303 UTC [2602179] DEBUG esmvalcore._recipe.to_datasets:367 Found Dataset: sithick, SImon, CMIP6, NorESM2-MM, historical, r1i1p1f1 2024-03-26 14:02:22,303 UTC [2602179] DEBUG esmvalcore._recipe.to_datasets:343 Populating list of datasets for variable areacello in diagnostic NegativeSeaIceFeedback 2024-03-26 14:02:22,304 UTC [2602179] DEBUG esmvalcore._recipe.to_datasets:367 Found Dataset: areacello, Ofx, CMIP6, NorESM2-LM, historical, r1i1p1f1 2024-03-26 14:02:22,304 UTC [2602179] DEBUG esmvalcore._recipe.to_datasets:367 Found Dataset: areacello, Ofx, CMIP6, NorESM2-MM, historical, r1i1p1f1 2024-03-26 14:02:22,304 UTC [2602179] DEBUG esmvalcore._recipe.recipe:781 Retrieving diagnostics from recipe 2024-03-26 14:02:22,304 UTC [2602179] DEBUG esmvalcore._recipe.recipe:809 Setting script for diagnostic NegativeSeaIceFeedback 2024-03-26 14:02:22,304 UTC [2602179] INFO esmvalcore._recipe.recipe:999 Creating tasks from recipe 2024-03-26 14:02:22,304 UTC [2602179] INFO esmvalcore._recipe.recipe:1008 Creating tasks for diagnostic NegativeSeaIceFeedback 2024-03-26 14:02:22,304 UTC [2602179] INFO esmvalcore._recipe.recipe:930 Creating diagnostic task NegativeSeaIceFeedback/negative_seaice_feedback 2024-03-26 14:02:22,305 UTC [2602179] INFO esmvalcore._recipe.recipe:983 Creating preprocessor task NegativeSeaIceFeedback/sithick 2024-03-26 14:02:22,305 UTC [2602179] INFO esmvalcore._recipe.recipe:653 Creating preprocessor 'default' task for variable 'sithick' 2024-03-26 14:02:22,307 UTC [2602179] INFO esmvalcore._recipe.recipe:983 Creating preprocessor task NegativeSeaIceFeedback/areacello 2024-03-26 14:02:22,308 UTC [2602179] INFO esmvalcore._recipe.recipe:653 Creating preprocessor 'default' task for variable 'areacello' 2024-03-26 14:02:22,308 UTC [2602179] ERROR esmvalcore._recipe.recipe:727 Could not create all tasks 2024-03-26 14:02:22,308 UTC [2602179] ERROR esmvalcore._recipe.recipe:729 Missing data for preprocessor NegativeSeaIceFeedback/sithick: - Dataset key 'grid' must be specified for {'mip': 'SImon', 'project': 'CMIP6', 'exp': 'historical', 'dataset': 'NorESM2-LM', 'ensemble': 'r1i1p1f1', 'short_name': 'sithick', 'timerange': '1979/2004', 'variable_group': 'sithick', 'diagnostic': 'NegativeSeaIceFeedback', 'recipe_dataset_index': 0, 'alias': 'NorESM2-LM', 'institute': ['NCC'], 'activity': 'CMIP', 'original_short_name': 'sithick', 'standard_name': 'sea_ice_thickness', 'long_name': 'Sea Ice Thickness', 'units': 'm', 'modeling_realm': ['seaIce', 'ocean'], 'frequency': 'mon'}, check your recipe entry - Dataset key 'grid' must be specified for {'mip': 'SImon', 'project': 'CMIP6', 'exp': 'historical', 'dataset': 'NorESM2-MM', 'ensemble': 'r1i1p1f1', 'short_name': 'sithick', 'timerange': '1979/2004', 'variable_group': 'sithick', 'diagnostic': 'NegativeSeaIceFeedback', 'recipe_dataset_index': 1, 'alias': 'NorESM2-MM', 'institute': ['NCC'], 'activity': 'CMIP', 'original_short_name': 'sithick', 'standard_name': 'sea_ice_thickness', 'long_name': 'Sea Ice Thickness', 'units': 'm', 'modeling_realm': ['seaIce', 'ocean'], 'frequency': 'mon'}, check your recipe entry 2024-03-26 14:02:22,308 UTC [2602179] ERROR esmvalcore._recipe.recipe:729 Missing data for preprocessor NegativeSeaIceFeedback/areacello: - Dataset key 'grid' must be specified for {'mip': 'Ofx', 'project': 'CMIP6', 'exp': 'historical', 'dataset': 'NorESM2-LM', 'ensemble': 'r1i1p1f1', 'short_name': 'areacello', 'variable_group': 'areacello', 'diagnostic': 'NegativeSeaIceFeedback', 'recipe_dataset_index': 0, 'alias': 'NorESM2-LM', 'institute': ['NCC'], 'activity': 'CMIP', 'original_short_name': 'areacello', 'standard_name': 'cell_area', 'long_name': 'Grid-Cell Area for Ocean Variables', 'units': 'm2', 'modeling_realm': ['ocean'], 'frequency': 'fx'}, check your recipe entry - Dataset key 'grid' must be specified for {'mip': 'Ofx', 'project': 'CMIP6', 'exp': 'historical', 'dataset': 'NorESM2-MM', 'ensemble': 'r1i1p1f1', 'short_name': 'areacello', 'variable_group': 'areacello', 'diagnostic': 'NegativeSeaIceFeedback', 'recipe_dataset_index': 1, 'alias': 'NorESM2-MM', 'institute': ['NCC'], 'activity': 'CMIP', 'original_short_name': 'areacello', 'standard_name': 'cell_area', 'long_name': 'Grid-Cell Area for Ocean Variables', 'units': 'm2', 'modeling_realm': ['ocean'], 'frequency': 'fx'}, check your recipe entry 2024-03-26 14:02:22,860 UTC [2602179] INFO esmvalcore._task:128 Maximum memory used (estimate): 0.3 GB 2024-03-26 14:02:22,860 UTC [2602179] INFO esmvalcore._task:130 Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. 2024-03-26 14:02:22,861 UTC [2602179] ERROR esmvalcore._main:523 Could not create all tasks 2024-03-26 14:02:22,861 UTC [2602179] DEBUG esmvalcore._main:524 Stack trace for debugging: Traceback (most recent call last): File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/_main.py", line 518, in run fire.Fire(ESMValTool()) File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/fire/core.py", line 141, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/fire/core.py", line 475, in _Fire component, remaining_args = _CallAndUpdateTrace( ^^^^^^^^^^^^^^^^^^^^ File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^ File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/_main.py", line 405, in run self._run(recipe, session) File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/_main.py", line 447, in _run process_recipe(recipe_file=recipe, session=session) File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/_main.py", line 127, in process_recipe recipe = read_recipe_file(recipe_file, session) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/_recipe/recipe.py", line 73, in read_recipe_file return Recipe(raw_recipe, session, recipe_file=filename) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/_recipe/recipe.py", line 720, in __init__ self.tasks = self.initialize_tasks() ^^^^^^^^^^^^^^^^^^^^^^^ File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/_recipe/recipe.py", line 1046, in initialize_tasks tasks = self._create_tasks() ^^^^^^^^^^^^^^^^^^^^ File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/_recipe/recipe.py", line 1034, in _create_tasks raise recipe_error esmvalcore.exceptions.RecipeError: Could not create all tasks