INFO [920192] ______________________________________________________________________ _____ ____ __ ____ __ _ _____ _ | ____/ ___|| \/ \ \ / /_ _| |_ _|__ ___ | | | _| \___ \| |\/| |\ \ / / _` | | | |/ _ \ / _ \| | | |___ ___) | | | | \ V / (_| | | | | (_) | (_) | | |_____|____/|_| |_| \_/ \__,_|_| |_|\___/ \___/|_| ______________________________________________________________________ ESMValTool - Earth System Model Evaluation Tool. http://www.esmvaltool.org CORE DEVELOPMENT TEAM AND CONTACTS: Birgit Hassler (Co-PI; DLR, Germany - birgit.hassler@dlr.de) Alistair Sellar (Co-PI; Met Office, UK - alistair.sellar@metoffice.gov.uk) Bouwe Andela (Netherlands eScience Center, The Netherlands - b.andela@esciencecenter.nl) Lee de Mora (PML, UK - ledm@pml.ac.uk) Niels Drost (Netherlands eScience Center, The Netherlands - n.drost@esciencecenter.nl) Veronika Eyring (DLR, Germany - veronika.eyring@dlr.de) Bettina Gier (UBremen, Germany - gier@uni-bremen.de) Remi Kazeroni (DLR, Germany - remi.kazeroni@dlr.de) Nikolay Koldunov (AWI, Germany - nikolay.koldunov@awi.de) Axel Lauer (DLR, Germany - axel.lauer@dlr.de) Saskia Loosveldt-Tomas (BSC, Spain - saskia.loosveldt@bsc.es) Ruth Lorenz (ETH Zurich, Switzerland - ruth.lorenz@env.ethz.ch) Benjamin Mueller (LMU, Germany - b.mueller@iggf.geo.uni-muenchen.de) Valeriu Predoi (URead, UK - valeriu.predoi@ncas.ac.uk) Mattia Righi (DLR, Germany - mattia.righi@dlr.de) Manuel Schlund (DLR, Germany - manuel.schlund@dlr.de) Breixo Solino Fernandez (DLR, Germany - breixo.solinofernandez@dlr.de) Javier Vegas-Regidor (BSC, Spain - javier.vegas@bsc.es) Klaus Zimmermann (SMHI, Sweden - klaus.zimmermann@smhi.se) For further help, please read the documentation at http://docs.esmvaltool.org. Have fun! INFO [920192] Package versions INFO [920192] ---------------- INFO [920192] ESMValCore: 2.10.0 INFO [920192] ESMValTool: 2.10.0 INFO [920192] ---------------- INFO [920192] Using config file /nird/home/tomast/.esmvaltool/config-user.yml INFO [920192] Writing program log files to: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_landcover_20231227_231102/run/main_log.txt /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_landcover_20231227_231102/run/main_log_debug.txt INFO [920192] Using default ESGF configuration, configuration file /nird/home/tomast/.esmvaltool/esgf-pyclient.yml not present. WARNING [920192] ESGF credentials missing, only data that is accessible without logging in will be available. See https://esgf.github.io/esgf-user-support/user_guide.html for instructions on how to create an account if you do not have one yet. Next, configure your system so esmvaltool can use your credentials. This can be done using the keyring package, or you can just enter them in /nird/home/tomast/.esmvaltool/esgf-pyclient.yml. keyring ======= First install the keyring package (requires a supported backend, see https://pypi.org/project/keyring/): $ pip install keyring Next, set your username and password by running the commands: $ keyring set ESGF hostname $ keyring set ESGF username $ keyring set ESGF password To check that you entered your credentials correctly, run: $ keyring get ESGF hostname $ keyring get ESGF username $ keyring get ESGF password configuration file ================== You can store the hostname, username, and password or your OpenID account in a plain text in the file /nird/home/tomast/.esmvaltool/esgf-pyclient.yml like this: logon: hostname: "your-hostname" username: "your-username" password: "your-password" or your can configure an interactive log in: logon: interactive: true Note that storing your password in plain text in the configuration file is less secure. On shared systems, make sure the permissions of the file are set so only you can read it, i.e. $ ls -l /nird/home/tomast/.esmvaltool/esgf-pyclient.yml shows permissions -rw-------. INFO [920192] Starting the Earth System Model Evaluation Tool at time: 2023-12-27 23:11:03 UTC INFO [920192] ---------------------------------------------------------------------- INFO [920192] RECIPE = /projects/NS2980K/home/tomast/ESMValTool/recipes-2.10.0/standard_recipes/land/recipe_landcover.yml INFO [920192] RUNDIR = /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_landcover_20231227_231102/run INFO [920192] WORKDIR = /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_landcover_20231227_231102/work INFO [920192] PREPROCDIR = /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_landcover_20231227_231102/preproc INFO [920192] PLOTDIR = /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_landcover_20231227_231102/plots INFO [920192] ---------------------------------------------------------------------- INFO [920192] Running tasks using at most 256 processes INFO [920192] If your system hangs during execution, it may not have enough memory for keeping this number of tasks in memory. INFO [920192] If you experience memory problems, try reducing 'max_parallel_tasks' in your user configuration file. WARNING [920192] Using the Dask basic scheduler. This may lead to slow computations and out-of-memory errors. Note that the basic scheduler may still be the best choice for preprocessor functions that are not lazy. In that case, you can safely ignore this warning. See https://docs.esmvaltool.org/projects/ESMValCore/en/latest/quickstart/configure.html#dask-distributed-configuration for more information. INFO [920192] Creating tasks from recipe INFO [920192] Creating tasks for diagnostic landcover INFO [920192] Creating diagnostic task landcover/landcover INFO [920192] Creating preprocessor task landcover/baresoilFrac INFO [920192] Creating preprocessor 'prep0' task for variable 'baresoilFrac' ERROR [920192] No input files found for Dataset: {'diagnostic': 'landcover', 'variable_group': 'baresoilFrac', 'dataset': 'ESACCI-LANDCOVER', 'project': 'OBS', 'mip': 'Lmon', 'short_name': 'baresoilFrac', 'alias': 'OBS', 'frequency': 'mon', 'long_name': 'Bare Soil Fraction', 'modeling_realm': ['land'], 'original_short_name': 'baresoilFrac', 'preprocessor': 'prep0', 'recipe_dataset_index': 1, 'reference_dataset': 'ESACCI-LANDCOVER', 'standard_name': 'area_fraction', 'tier': 2, 'timerange': '2008/2012', 'type': 'sat', 'units': '%', 'version': 'L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg'} session: 'recipe_landcover_20231227_231102' ERROR [920192] Looked for files matching /projects/NS9560K-datalake/ESGF/obsdata/Tier2/ESACCI-LANDCOVER/OBS_ESACCI-LANDCOVER_sat_L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg_Lmon_baresoilFrac[_.]*nc /scratch/tomast/ESGF/obsdata/Tier2/ESACCI-LANDCOVER/OBS_ESACCI-LANDCOVER_sat_L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg_Lmon_baresoilFrac[_.]*nc ERROR [920192] Set 'log_level' to 'debug' to get more information INFO [920192] Found input files for Dataset: baresoilFrac, Lmon, CMIP5, MPI-ESM-LR, rcp85, r1i1p1, v20120625 ERROR [920192] No input files found for Dataset: {'diagnostic': 'landcover', 'variable_group': 'baresoilFrac', 'dataset': 'ESACCI-LANDCOVER', 'project': 'OBS', 'mip': 'Lmon', 'short_name': 'baresoilFrac', 'alias': 'OBS', 'frequency': 'mon', 'long_name': 'Bare Soil Fraction', 'modeling_realm': ['land'], 'original_short_name': 'baresoilFrac', 'preprocessor': 'prep0', 'recipe_dataset_index': 1, 'reference_dataset': 'ESACCI-LANDCOVER', 'standard_name': 'area_fraction', 'tier': 2, 'timerange': '2008/2012', 'type': 'sat', 'units': '%', 'version': 'L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg'} session: 'recipe_landcover_20231227_231102' ERROR [920192] Looked for files matching /projects/NS9560K-datalake/ESGF/obsdata/Tier2/ESACCI-LANDCOVER/OBS_ESACCI-LANDCOVER_sat_L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg_Lmon_baresoilFrac[_.]*nc /scratch/tomast/ESGF/obsdata/Tier2/ESACCI-LANDCOVER/OBS_ESACCI-LANDCOVER_sat_L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg_Lmon_baresoilFrac[_.]*nc ERROR [920192] Set 'log_level' to 'debug' to get more information INFO [920192] Creating preprocessor task landcover/grassFrac INFO [920192] Creating preprocessor 'prep0' task for variable 'grassFrac' ERROR [920192] No input files found for Dataset: {'diagnostic': 'landcover', 'variable_group': 'grassFrac', 'dataset': 'ESACCI-LANDCOVER', 'project': 'OBS', 'mip': 'Lmon', 'short_name': 'grassFrac', 'alias': 'OBS', 'frequency': 'mon', 'long_name': 'Natural Grass Fraction', 'modeling_realm': ['land'], 'original_short_name': 'grassFrac', 'preprocessor': 'prep0', 'recipe_dataset_index': 1, 'reference_dataset': 'ESACCI-LANDCOVER', 'standard_name': 'area_fraction', 'tier': 2, 'timerange': '2008/2012', 'type': 'sat', 'units': '%', 'version': 'L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg'} session: 'recipe_landcover_20231227_231102' ERROR [920192] Looked for files matching /projects/NS9560K-datalake/ESGF/obsdata/Tier2/ESACCI-LANDCOVER/OBS_ESACCI-LANDCOVER_sat_L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg_Lmon_grassFrac[_.]*nc /scratch/tomast/ESGF/obsdata/Tier2/ESACCI-LANDCOVER/OBS_ESACCI-LANDCOVER_sat_L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg_Lmon_grassFrac[_.]*nc ERROR [920192] Set 'log_level' to 'debug' to get more information INFO [920192] Found input files for Dataset: grassFrac, Lmon, CMIP5, MPI-ESM-LR, rcp85, r1i1p1, v20120625 ERROR [920192] No input files found for Dataset: {'diagnostic': 'landcover', 'variable_group': 'grassFrac', 'dataset': 'ESACCI-LANDCOVER', 'project': 'OBS', 'mip': 'Lmon', 'short_name': 'grassFrac', 'alias': 'OBS', 'frequency': 'mon', 'long_name': 'Natural Grass Fraction', 'modeling_realm': ['land'], 'original_short_name': 'grassFrac', 'preprocessor': 'prep0', 'recipe_dataset_index': 1, 'reference_dataset': 'ESACCI-LANDCOVER', 'standard_name': 'area_fraction', 'tier': 2, 'timerange': '2008/2012', 'type': 'sat', 'units': '%', 'version': 'L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg'} session: 'recipe_landcover_20231227_231102' ERROR [920192] Looked for files matching /projects/NS9560K-datalake/ESGF/obsdata/Tier2/ESACCI-LANDCOVER/OBS_ESACCI-LANDCOVER_sat_L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg_Lmon_grassFrac[_.]*nc /scratch/tomast/ESGF/obsdata/Tier2/ESACCI-LANDCOVER/OBS_ESACCI-LANDCOVER_sat_L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg_Lmon_grassFrac[_.]*nc ERROR [920192] Set 'log_level' to 'debug' to get more information INFO [920192] Creating preprocessor task landcover/treeFrac INFO [920192] Creating preprocessor 'prep0' task for variable 'treeFrac' ERROR [920192] No input files found for Dataset: {'diagnostic': 'landcover', 'variable_group': 'treeFrac', 'dataset': 'ESACCI-LANDCOVER', 'project': 'OBS', 'mip': 'Lmon', 'short_name': 'treeFrac', 'alias': 'OBS', 'frequency': 'mon', 'long_name': 'Tree Cover Fraction', 'modeling_realm': ['land'], 'original_short_name': 'treeFrac', 'preprocessor': 'prep0', 'recipe_dataset_index': 1, 'reference_dataset': 'ESACCI-LANDCOVER', 'standard_name': 'area_fraction', 'tier': 2, 'timerange': '2008/2012', 'type': 'sat', 'units': '%', 'version': 'L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg'} session: 'recipe_landcover_20231227_231102' ERROR [920192] Looked for files matching /projects/NS9560K-datalake/ESGF/obsdata/Tier2/ESACCI-LANDCOVER/OBS_ESACCI-LANDCOVER_sat_L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg_Lmon_treeFrac[_.]*nc /scratch/tomast/ESGF/obsdata/Tier2/ESACCI-LANDCOVER/OBS_ESACCI-LANDCOVER_sat_L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg_Lmon_treeFrac[_.]*nc ERROR [920192] Set 'log_level' to 'debug' to get more information INFO [920192] Found input files for Dataset: treeFrac, Lmon, CMIP5, MPI-ESM-LR, rcp85, r1i1p1, v20120625 ERROR [920192] No input files found for Dataset: {'diagnostic': 'landcover', 'variable_group': 'treeFrac', 'dataset': 'ESACCI-LANDCOVER', 'project': 'OBS', 'mip': 'Lmon', 'short_name': 'treeFrac', 'alias': 'OBS', 'frequency': 'mon', 'long_name': 'Tree Cover Fraction', 'modeling_realm': ['land'], 'original_short_name': 'treeFrac', 'preprocessor': 'prep0', 'recipe_dataset_index': 1, 'reference_dataset': 'ESACCI-LANDCOVER', 'standard_name': 'area_fraction', 'tier': 2, 'timerange': '2008/2012', 'type': 'sat', 'units': '%', 'version': 'L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg'} session: 'recipe_landcover_20231227_231102' ERROR [920192] Looked for files matching /projects/NS9560K-datalake/ESGF/obsdata/Tier2/ESACCI-LANDCOVER/OBS_ESACCI-LANDCOVER_sat_L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg_Lmon_treeFrac[_.]*nc /scratch/tomast/ESGF/obsdata/Tier2/ESACCI-LANDCOVER/OBS_ESACCI-LANDCOVER_sat_L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg_Lmon_treeFrac[_.]*nc ERROR [920192] Set 'log_level' to 'debug' to get more information INFO [920192] Creating preprocessor task landcover/shrubFrac INFO [920192] Creating preprocessor 'prep0' task for variable 'shrubFrac' ERROR [920192] No input files found for Dataset: {'diagnostic': 'landcover', 'variable_group': 'shrubFrac', 'dataset': 'ESACCI-LANDCOVER', 'project': 'OBS', 'mip': 'Lmon', 'short_name': 'shrubFrac', 'alias': 'OBS', 'frequency': 'mon', 'long_name': 'Shrub Fraction', 'modeling_realm': ['land'], 'original_short_name': 'shrubFrac', 'preprocessor': 'prep0', 'recipe_dataset_index': 1, 'reference_dataset': 'ESACCI-LANDCOVER', 'standard_name': 'area_fraction', 'tier': 2, 'timerange': '2008/2012', 'type': 'sat', 'units': '%', 'version': 'L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg'} session: 'recipe_landcover_20231227_231102' ERROR [920192] Looked for files matching /projects/NS9560K-datalake/ESGF/obsdata/Tier2/ESACCI-LANDCOVER/OBS_ESACCI-LANDCOVER_sat_L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg_Lmon_shrubFrac[_.]*nc /scratch/tomast/ESGF/obsdata/Tier2/ESACCI-LANDCOVER/OBS_ESACCI-LANDCOVER_sat_L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg_Lmon_shrubFrac[_.]*nc ERROR [920192] Set 'log_level' to 'debug' to get more information INFO [920192] Found input files for Dataset: shrubFrac, Lmon, CMIP5, MPI-ESM-LR, rcp85, r1i1p1, v20120625 ERROR [920192] No input files found for Dataset: {'diagnostic': 'landcover', 'variable_group': 'shrubFrac', 'dataset': 'ESACCI-LANDCOVER', 'project': 'OBS', 'mip': 'Lmon', 'short_name': 'shrubFrac', 'alias': 'OBS', 'frequency': 'mon', 'long_name': 'Shrub Fraction', 'modeling_realm': ['land'], 'original_short_name': 'shrubFrac', 'preprocessor': 'prep0', 'recipe_dataset_index': 1, 'reference_dataset': 'ESACCI-LANDCOVER', 'standard_name': 'area_fraction', 'tier': 2, 'timerange': '2008/2012', 'type': 'sat', 'units': '%', 'version': 'L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg'} session: 'recipe_landcover_20231227_231102' ERROR [920192] Looked for files matching /projects/NS9560K-datalake/ESGF/obsdata/Tier2/ESACCI-LANDCOVER/OBS_ESACCI-LANDCOVER_sat_L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg_Lmon_shrubFrac[_.]*nc /scratch/tomast/ESGF/obsdata/Tier2/ESACCI-LANDCOVER/OBS_ESACCI-LANDCOVER_sat_L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg_Lmon_shrubFrac[_.]*nc ERROR [920192] Set 'log_level' to 'debug' to get more information INFO [920192] Creating preprocessor task landcover/cropFrac INFO [920192] Creating preprocessor 'prep0' task for variable 'cropFrac' ERROR [920192] No input files found for Dataset: {'diagnostic': 'landcover', 'variable_group': 'cropFrac', 'dataset': 'ESACCI-LANDCOVER', 'project': 'OBS', 'mip': 'Lmon', 'short_name': 'cropFrac', 'alias': 'OBS', 'frequency': 'mon', 'long_name': 'Crop Fraction', 'modeling_realm': ['land'], 'original_short_name': 'cropFrac', 'preprocessor': 'prep0', 'recipe_dataset_index': 1, 'reference_dataset': 'ESACCI-LANDCOVER', 'standard_name': 'area_fraction', 'tier': 2, 'timerange': '2008/2012', 'type': 'sat', 'units': '%', 'version': 'L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg'} session: 'recipe_landcover_20231227_231102' ERROR [920192] Looked for files matching /projects/NS9560K-datalake/ESGF/obsdata/Tier2/ESACCI-LANDCOVER/OBS_ESACCI-LANDCOVER_sat_L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg_Lmon_cropFrac[_.]*nc /scratch/tomast/ESGF/obsdata/Tier2/ESACCI-LANDCOVER/OBS_ESACCI-LANDCOVER_sat_L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg_Lmon_cropFrac[_.]*nc ERROR [920192] Set 'log_level' to 'debug' to get more information INFO [920192] Found input files for Dataset: cropFrac, Lmon, CMIP5, MPI-ESM-LR, rcp85, r1i1p1, v20120625 ERROR [920192] No input files found for Dataset: {'diagnostic': 'landcover', 'variable_group': 'cropFrac', 'dataset': 'ESACCI-LANDCOVER', 'project': 'OBS', 'mip': 'Lmon', 'short_name': 'cropFrac', 'alias': 'OBS', 'frequency': 'mon', 'long_name': 'Crop Fraction', 'modeling_realm': ['land'], 'original_short_name': 'cropFrac', 'preprocessor': 'prep0', 'recipe_dataset_index': 1, 'reference_dataset': 'ESACCI-LANDCOVER', 'standard_name': 'area_fraction', 'tier': 2, 'timerange': '2008/2012', 'type': 'sat', 'units': '%', 'version': 'L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg'} session: 'recipe_landcover_20231227_231102' ERROR [920192] Looked for files matching /projects/NS9560K-datalake/ESGF/obsdata/Tier2/ESACCI-LANDCOVER/OBS_ESACCI-LANDCOVER_sat_L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg_Lmon_cropFrac[_.]*nc /scratch/tomast/ESGF/obsdata/Tier2/ESACCI-LANDCOVER/OBS_ESACCI-LANDCOVER_sat_L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg_Lmon_cropFrac[_.]*nc ERROR [920192] Set 'log_level' to 'debug' to get more information ERROR [920192] Could not create all tasks ERROR [920192] Missing data for preprocessor landcover/baresoilFrac: - Missing data for Dataset: baresoilFrac, Lmon, OBS, ESACCI-LANDCOVER, L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg ERROR [920192] Missing data for preprocessor landcover/grassFrac: - Missing data for Dataset: grassFrac, Lmon, OBS, ESACCI-LANDCOVER, L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg ERROR [920192] Missing data for preprocessor landcover/treeFrac: - Missing data for Dataset: treeFrac, Lmon, OBS, ESACCI-LANDCOVER, L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg ERROR [920192] Missing data for preprocessor landcover/shrubFrac: - Missing data for Dataset: shrubFrac, Lmon, OBS, ESACCI-LANDCOVER, L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg ERROR [920192] Missing data for preprocessor landcover/cropFrac: - Missing data for Dataset: cropFrac, Lmon, OBS, ESACCI-LANDCOVER, L4-LCCS-Map-300m-P5Y-aggregated-0.500000Deg INFO [920192] Maximum memory used (estimate): 0.3 GB INFO [920192] Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. ERROR [920192] Could not create all tasks