2023-12-27 23:09:44,882 UTC [919785] INFO esmvalcore._main:492 ______________________________________________________________________ _____ ____ __ ____ __ _ _____ _ | ____/ ___|| \/ \ \ / /_ _| |_ _|__ ___ | | | _| \___ \| |\/| |\ \ / / _` | | | |/ _ \ / _ \| | | |___ ___) | | | | \ V / (_| | | | | (_) | (_) | | |_____|____/|_| |_| \_/ \__,_|_| |_|\___/ \___/|_| ______________________________________________________________________ ESMValTool - Earth System Model Evaluation Tool. http://www.esmvaltool.org CORE DEVELOPMENT TEAM AND CONTACTS: Birgit Hassler (Co-PI; DLR, Germany - birgit.hassler@dlr.de) Alistair Sellar (Co-PI; Met Office, UK - alistair.sellar@metoffice.gov.uk) Bouwe Andela (Netherlands eScience Center, The Netherlands - b.andela@esciencecenter.nl) Lee de Mora (PML, UK - ledm@pml.ac.uk) Niels Drost (Netherlands eScience Center, The Netherlands - n.drost@esciencecenter.nl) Veronika Eyring (DLR, Germany - veronika.eyring@dlr.de) Bettina Gier (UBremen, Germany - gier@uni-bremen.de) Remi Kazeroni (DLR, Germany - remi.kazeroni@dlr.de) Nikolay Koldunov (AWI, Germany - nikolay.koldunov@awi.de) Axel Lauer (DLR, Germany - axel.lauer@dlr.de) Saskia Loosveldt-Tomas (BSC, Spain - saskia.loosveldt@bsc.es) Ruth Lorenz (ETH Zurich, Switzerland - ruth.lorenz@env.ethz.ch) Benjamin Mueller (LMU, Germany - b.mueller@iggf.geo.uni-muenchen.de) Valeriu Predoi (URead, UK - valeriu.predoi@ncas.ac.uk) Mattia Righi (DLR, Germany - mattia.righi@dlr.de) Manuel Schlund (DLR, Germany - manuel.schlund@dlr.de) Breixo Solino Fernandez (DLR, Germany - breixo.solinofernandez@dlr.de) Javier Vegas-Regidor (BSC, Spain - javier.vegas@bsc.es) Klaus Zimmermann (SMHI, Sweden - klaus.zimmermann@smhi.se) For further help, please read the documentation at http://docs.esmvaltool.org. Have fun! 2023-12-27 23:09:44,882 UTC [919785] INFO esmvalcore._main:493 Package versions 2023-12-27 23:09:44,882 UTC [919785] INFO esmvalcore._main:494 ---------------- 2023-12-27 23:09:44,882 UTC [919785] INFO esmvalcore._main:495 ESMValCore: 2.10.0 2023-12-27 23:09:44,883 UTC [919785] INFO esmvalcore._main:497 ESMValTool: 2.10.0 2023-12-27 23:09:44,883 UTC [919785] INFO esmvalcore._main:498 ---------------- 2023-12-27 23:09:44,883 UTC [919785] INFO esmvalcore._main:499 Using config file /nird/home/tomast/.esmvaltool/config-user.yml 2023-12-27 23:09:44,883 UTC [919785] INFO esmvalcore._main:500 Writing program log files to: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_albedolandcover_20231227_230944/run/main_log.txt /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_albedolandcover_20231227_230944/run/main_log_debug.txt 2023-12-27 23:09:45,054 UTC [919785] INFO esmvalcore.config._esgf_pyclient:116 Using default ESGF configuration, configuration file /nird/home/tomast/.esmvaltool/esgf-pyclient.yml not present. 2023-12-27 23:09:45,055 UTC [919785] WARNING esmvalcore.config._esgf_pyclient:181 ESGF credentials missing, only data that is accessible without logging in will be available. See https://esgf.github.io/esgf-user-support/user_guide.html for instructions on how to create an account if you do not have one yet. Next, configure your system so esmvaltool can use your credentials. This can be done using the keyring package, or you can just enter them in /nird/home/tomast/.esmvaltool/esgf-pyclient.yml. keyring ======= First install the keyring package (requires a supported backend, see https://pypi.org/project/keyring/): $ pip install keyring Next, set your username and password by running the commands: $ keyring set ESGF hostname $ keyring set ESGF username $ keyring set ESGF password To check that you entered your credentials correctly, run: $ keyring get ESGF hostname $ keyring get ESGF username $ keyring get ESGF password configuration file ================== You can store the hostname, username, and password or your OpenID account in a plain text in the file /nird/home/tomast/.esmvaltool/esgf-pyclient.yml like this: logon: hostname: "your-hostname" username: "your-username" password: "your-password" or your can configure an interactive log in: logon: interactive: true Note that storing your password in plain text in the configuration file is less secure. On shared systems, make sure the permissions of the file are set so only you can read it, i.e. $ ls -l /nird/home/tomast/.esmvaltool/esgf-pyclient.yml shows permissions -rw-------. 2023-12-27 23:09:45,167 UTC [919785] INFO esmvalcore._main:91 Starting the Earth System Model Evaluation Tool at time: 2023-12-27 23:09:45 UTC 2023-12-27 23:09:45,168 UTC [919785] INFO esmvalcore._main:95 ---------------------------------------------------------------------- 2023-12-27 23:09:45,168 UTC [919785] INFO esmvalcore._main:96 RECIPE = /projects/NS2980K/home/tomast/ESMValTool/recipes-2.10.0/standard_recipes/land/recipe_albedolandcover.yml 2023-12-27 23:09:45,168 UTC [919785] INFO esmvalcore._main:97 RUNDIR = /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_albedolandcover_20231227_230944/run 2023-12-27 23:09:45,168 UTC [919785] INFO esmvalcore._main:98 WORKDIR = /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_albedolandcover_20231227_230944/work 2023-12-27 23:09:45,168 UTC [919785] INFO esmvalcore._main:99 PREPROCDIR = /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_albedolandcover_20231227_230944/preproc 2023-12-27 23:09:45,168 UTC [919785] INFO esmvalcore._main:100 PLOTDIR = /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_albedolandcover_20231227_230944/plots 2023-12-27 23:09:45,169 UTC [919785] INFO esmvalcore._main:101 ---------------------------------------------------------------------- 2023-12-27 23:09:45,169 UTC [919785] INFO esmvalcore._main:104 Running tasks using at most 256 processes 2023-12-27 23:09:45,169 UTC [919785] INFO esmvalcore._main:106 If your system hangs during execution, it may not have enough memory for keeping this number of tasks in memory. 2023-12-27 23:09:45,169 UTC [919785] INFO esmvalcore._main:109 If you experience memory problems, try reducing 'max_parallel_tasks' in your user configuration file. 2023-12-27 23:09:45,169 UTC [919785] WARNING esmvalcore.config._dask:18 Using the Dask basic scheduler. This may lead to slow computations and out-of-memory errors. Note that the basic scheduler may still be the best choice for preprocessor functions that are not lazy. In that case, you can safely ignore this warning. See https://docs.esmvaltool.org/projects/ESMValCore/en/latest/quickstart/configure.html#dask-distributed-configuration for more information. 2023-12-27 23:09:45,170 UTC [919785] DEBUG esmvalcore._recipe.check:54 Checking recipe against schema /projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/_recipe/recipe_schema.yml 2023-12-27 23:09:45,197 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:343 Populating list of datasets for variable treeFrac in diagnostic albedolandcover_cmip5 2023-12-27 23:09:45,198 UTC [919785] DEBUG esmvalcore.config._config:45 Loading extra facets from /projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config/extra_facets/cmip5-fx.yml 2023-12-27 23:09:45,198 UTC [919785] DEBUG esmvalcore.config._config:45 Loading extra facets from /projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config/extra_facets/cmip5-institutes.yml 2023-12-27 23:09:45,214 UTC [919785] DEBUG esmvalcore.config._config:45 Loading extra facets from /projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config/extra_facets/cmip5-product.yml 2023-12-27 23:09:45,218 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc /scratch/tomast/ESGF/cmip5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc 2023-12-27 23:09:46,528 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:367 Found Dataset: treeFrac, Lmon, CMIP5, MPI-ESM-LR, historical, r1i1p1, supplementaries: sftlf, fx, r0i0p0 2023-12-27 23:09:46,528 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:343 Populating list of datasets for variable snc in diagnostic albedolandcover_cmip5 2023-12-27 23:09:46,528 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc /scratch/tomast/ESGF/cmip5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc 2023-12-27 23:09:46,532 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:367 Found Dataset: snc, LImon, CMIP5, MPI-ESM-LR, historical, r1i1p1, supplementaries: sftlf, fx, r0i0p0 2023-12-27 23:09:46,532 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:343 Populating list of datasets for variable alb in diagnostic albedolandcover_cmip5 2023-12-27 23:09:46,532 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/*/alb/alb_Amon_MPI-ESM-LR_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/*/alb/alb_Amon_MPI-ESM-LR_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/*/alb/alb_Amon_MPI-ESM-LR_historical_r1i1p1*.nc 2023-12-27 23:09:46,533 UTC [919785] DEBUG esmvalcore.esgf._search:116 Searching https://esgf.ceda.ac.uk/esg-search for datasets using facets={'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'Amon', 'product': 'output1,output2', 'variable': 'alb'} 2023-12-27 23:09:46,533 UTC [919785] DEBUG requests_cache.backends:63 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2023-12-27 23:09:46,533 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2023-12-27 23:09:46,533 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2023-12-27 23:09:46,534 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: None 2023-12-27 23:09:46,534 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2023-12-27 23:09:46,537 UTC [919785] DEBUG requests_cache.policy.actions:91 Cache directives from request headers: CacheDirectives() 2023-12-27 23:09:46,537 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-read cache checks: Passed 2023-12-27 23:09:46,537 UTC [919785] DEBUG requests_cache.policy.actions:205 Post-read cache actions: CacheActions(expire_after=86400, send_request=True) 2023-12-27 23:09:46,538 UTC [919785] DEBUG urllib3.connectionpool:1052 Starting new HTTPS connection (1): esgf.ceda.ac.uk:443 2023-12-27 23:09:46,766 UTC [919785] DEBUG urllib3.connectionpool:546 https://esgf.ceda.ac.uk:443 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=CMIP5&model=MPI-ESM-LR&ensemble=r1i1p1&experiment=historical&time_frequency=mon&institute=MPI-M&cmor_table=Amon&product=output1%2Coutput2&variable=alb HTTP/1.1" 200 1232 2023-12-27 23:09:46,767 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-write cache checks: Passed 2023-12-27 23:09:46,777 UTC [919785] DEBUG requests_cache.backends.base:106 Closing backend connections 2023-12-27 23:09:46,779 UTC [919785] DEBUG esmvalcore.esgf._search:154 Found the following files matching facets {'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'Amon', 'product': 'output1,output2', 'variable': 'alb'}: none 2023-12-27 23:09:46,780 UTC [919785] DEBUG esmvalcore.esgf._search:364 Selected files: 2023-12-27 23:09:46,780 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc /scratch/tomast/ESGF/cmip5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc 2023-12-27 23:09:46,785 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:367 Found Dataset: alb, Amon, CMIP5, MPI-ESM-LR, historical, r1i1p1, supplementaries: sftlf, fx, r0i0p0 2023-12-27 23:09:46,785 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:343 Populating list of datasets for variable cropFrac in diagnostic albedolandcover_cmip5 2023-12-27 23:09:46,785 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc /scratch/tomast/ESGF/cmip5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc 2023-12-27 23:09:46,789 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:367 Found Dataset: cropFrac, Lmon, CMIP5, MPI-ESM-LR, historical, r1i1p1, supplementaries: sftlf, fx, r0i0p0 2023-12-27 23:09:46,789 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:343 Populating list of datasets for variable grassFrac in diagnostic albedolandcover_cmip5 2023-12-27 23:09:46,789 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc /scratch/tomast/ESGF/cmip5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc 2023-12-27 23:09:46,792 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:367 Found Dataset: grassFrac, Lmon, CMIP5, MPI-ESM-LR, historical, r1i1p1, supplementaries: sftlf, fx, r0i0p0 2023-12-27 23:09:46,792 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:343 Populating list of datasets for variable shrubFrac in diagnostic albedolandcover_cmip5 2023-12-27 23:09:46,793 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc /scratch/tomast/ESGF/cmip5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc 2023-12-27 23:09:46,795 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:367 Found Dataset: shrubFrac, Lmon, CMIP5, MPI-ESM-LR, historical, r1i1p1, supplementaries: sftlf, fx, r0i0p0 2023-12-27 23:09:46,795 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:343 Populating list of datasets for variable pastureFrac in diagnostic albedolandcover_cmip5 2023-12-27 23:09:46,796 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc /scratch/tomast/ESGF/cmip5/output1/*/MPI-ESM-LR/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_MPI-ESM-LR_*_**.nc 2023-12-27 23:09:46,799 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:367 Found Dataset: pastureFrac, Lmon, CMIP5, MPI-ESM-LR, historical, r1i1p1, supplementaries: sftlf, fx, r0i0p0 2023-12-27 23:09:46,799 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:343 Populating list of datasets for variable treeFrac in diagnostic albedolandcover_cmip6 2023-12-27 23:09:46,801 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/*/*/HadGEM3-GC31-LL/*/*/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_*_*_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/*/*/HadGEM3-GC31-LL/*/*/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_*_*_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/*/*/HadGEM3-GC31-LL/*/*/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_*_*_gn*.nc /scratch/tomast/ESGF/CMIP6/*/*/HadGEM3-GC31-LL/*/*/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_*_*_gn*.nc 2023-12-27 23:09:47,117 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:367 Found Dataset: treeFrac, Lmon, CMIP6, HadGEM3-GC31-LL, historical, r1i1p1f3, gn, supplementaries: sftlf, fx, CMIP, piControl, r1i1p1f1 2023-12-27 23:09:47,117 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:343 Populating list of datasets for variable snc in diagnostic albedolandcover_cmip6 2023-12-27 23:09:47,117 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/*/*/HadGEM3-GC31-LL/*/*/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_*_*_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/*/*/HadGEM3-GC31-LL/*/*/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_*_*_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/*/*/HadGEM3-GC31-LL/*/*/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_*_*_gn*.nc /scratch/tomast/ESGF/CMIP6/*/*/HadGEM3-GC31-LL/*/*/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_*_*_gn*.nc 2023-12-27 23:09:47,127 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:367 Found Dataset: snc, LImon, CMIP6, HadGEM3-GC31-LL, historical, r1i1p1f3, gn, supplementaries: sftlf, fx, CMIP, piControl, r1i1p1f1 2023-12-27 23:09:47,127 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:343 Populating list of datasets for variable alb in diagnostic albedolandcover_cmip6 2023-12-27 23:09:47,127 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9034K/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc 2023-12-27 23:09:47,129 UTC [919785] DEBUG esmvalcore.esgf._search:116 Searching https://esgf.ceda.ac.uk/esg-search for datasets using facets={'project': 'CMIP6', 'activity_drs': 'CMIP', 'source_id': 'HadGEM3-GC31-LL', 'member_id': 'r1i1p1f3', 'experiment_id': 'historical', 'institution_id': 'MOHC,NERC', 'grid_label': 'gn', 'table_id': 'Amon', 'variable': 'alb'} 2023-12-27 23:09:47,129 UTC [919785] DEBUG requests_cache.backends:63 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2023-12-27 23:09:47,129 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2023-12-27 23:09:47,129 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2023-12-27 23:09:47,130 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: None 2023-12-27 23:09:47,130 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2023-12-27 23:09:47,132 UTC [919785] DEBUG requests_cache.policy.actions:91 Cache directives from request headers: CacheDirectives() 2023-12-27 23:09:47,132 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-read cache checks: Passed 2023-12-27 23:09:47,132 UTC [919785] DEBUG requests_cache.policy.actions:205 Post-read cache actions: CacheActions(expire_after=86400, send_request=True) 2023-12-27 23:09:47,133 UTC [919785] DEBUG urllib3.connectionpool:1052 Starting new HTTPS connection (1): esgf.ceda.ac.uk:443 2023-12-27 23:09:47,323 UTC [919785] DEBUG urllib3.connectionpool:546 https://esgf.ceda.ac.uk:443 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=CMIP6&activity_drs=CMIP&source_id=HadGEM3-GC31-LL&member_id=r1i1p1f3&experiment_id=historical&institution_id=MOHC%2CNERC&grid_label=gn&table_id=Amon&variable=alb HTTP/1.1" 200 1250 2023-12-27 23:09:47,323 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-write cache checks: Passed 2023-12-27 23:09:47,467 UTC [919785] DEBUG requests_cache.backends.base:106 Closing backend connections 2023-12-27 23:09:47,468 UTC [919785] DEBUG esmvalcore.esgf._search:154 Found the following files matching facets {'project': 'CMIP6', 'activity_drs': 'CMIP', 'source_id': 'HadGEM3-GC31-LL', 'member_id': 'r1i1p1f3', 'experiment_id': 'historical', 'institution_id': 'MOHC,NERC', 'grid_label': 'gn', 'table_id': 'Amon', 'variable': 'alb'}: none 2023-12-27 23:09:47,469 UTC [919785] DEBUG esmvalcore.esgf._search:364 Selected files: 2023-12-27 23:09:47,469 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/*/*/HadGEM3-GC31-LL/*/*/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_*_*_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/*/*/HadGEM3-GC31-LL/*/*/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_*_*_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/*/*/HadGEM3-GC31-LL/*/*/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_*_*_gn*.nc /scratch/tomast/ESGF/CMIP6/*/*/HadGEM3-GC31-LL/*/*/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_*_*_gn*.nc 2023-12-27 23:09:47,478 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:367 Found Dataset: alb, Amon, CMIP6, HadGEM3-GC31-LL, historical, r1i1p1f3, gn, supplementaries: sftlf, fx, CMIP, piControl, r1i1p1f1 2023-12-27 23:09:47,478 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:343 Populating list of datasets for variable grassFrac in diagnostic albedolandcover_cmip6 2023-12-27 23:09:47,478 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/*/*/HadGEM3-GC31-LL/*/*/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_*_*_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/*/*/HadGEM3-GC31-LL/*/*/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_*_*_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/*/*/HadGEM3-GC31-LL/*/*/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_*_*_gn*.nc /scratch/tomast/ESGF/CMIP6/*/*/HadGEM3-GC31-LL/*/*/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_*_*_gn*.nc 2023-12-27 23:09:47,486 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:367 Found Dataset: grassFrac, Lmon, CMIP6, HadGEM3-GC31-LL, historical, r1i1p1f3, gn, supplementaries: sftlf, fx, CMIP, piControl, r1i1p1f1 2023-12-27 23:09:47,486 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:343 Populating list of datasets for variable shrubFrac in diagnostic albedolandcover_cmip6 2023-12-27 23:09:47,486 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/*/*/HadGEM3-GC31-LL/*/*/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_*_*_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/*/*/HadGEM3-GC31-LL/*/*/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_*_*_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/*/*/HadGEM3-GC31-LL/*/*/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_*_*_gn*.nc /scratch/tomast/ESGF/CMIP6/*/*/HadGEM3-GC31-LL/*/*/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_*_*_gn*.nc 2023-12-27 23:09:47,494 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:367 Found Dataset: shrubFrac, Lmon, CMIP6, HadGEM3-GC31-LL, historical, r1i1p1f3, gn, supplementaries: sftlf, fx, CMIP, piControl, r1i1p1f1 2023-12-27 23:09:47,494 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:343 Populating list of datasets for variable albDiffiTr13 in diagnostic albedolandcover_obs 2023-12-27 23:09:47,495 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9560K-datalake/ESGF/obsdata/Tier2/Duveiller2018/OBS_Duveiller2018_clim_*_fx_sftlf[_.]*nc /scratch/tomast/ESGF/obsdata/Tier2/Duveiller2018/OBS_Duveiller2018_clim_*_fx_sftlf[_.]*nc 2023-12-27 23:09:47,495 UTC [919785] INFO esmvalcore.dataset:315 For Dataset: albDiffiTr13, Amon, OBS, Duveiller2018, v2018, supplementaries: sftlf, *, *: ignoring supplementary variable 'sftlf', unable to expand wildcards 'mip', 'version'. 2023-12-27 23:09:47,495 UTC [919785] DEBUG esmvalcore._recipe.to_datasets:367 Found Dataset: albDiffiTr13, Amon, OBS, Duveiller2018, v2018 2023-12-27 23:09:47,496 UTC [919785] DEBUG esmvalcore._recipe.recipe:781 Retrieving diagnostics from recipe 2023-12-27 23:09:47,496 UTC [919785] DEBUG esmvalcore._recipe.recipe:809 Setting script for diagnostic albedolandcover_cmip5 2023-12-27 23:09:47,496 UTC [919785] DEBUG esmvalcore._recipe.recipe:809 Setting script for diagnostic albedolandcover_cmip6 2023-12-27 23:09:47,496 UTC [919785] DEBUG esmvalcore._recipe.recipe:809 Setting script for diagnostic albedolandcover_obs 2023-12-27 23:09:47,496 UTC [919785] INFO esmvalcore._recipe.recipe:999 Creating tasks from recipe 2023-12-27 23:09:47,496 UTC [919785] INFO esmvalcore._recipe.recipe:1008 Creating tasks for diagnostic albedolandcover_cmip5 2023-12-27 23:09:47,497 UTC [919785] INFO esmvalcore._recipe.recipe:930 Creating diagnostic task albedolandcover_cmip5/albedolandcover 2023-12-27 23:09:47,498 UTC [919785] INFO esmvalcore._recipe.recipe:983 Creating preprocessor task albedolandcover_cmip5/treeFrac 2023-12-27 23:09:47,498 UTC [919785] INFO esmvalcore._recipe.recipe:653 Creating preprocessor 'pp_cmip' task for variable 'treeFrac' 2023-12-27 23:09:47,501 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/*/treeFrac/treeFrac_Lmon_MPI-ESM-LR_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/*/treeFrac/treeFrac_Lmon_MPI-ESM-LR_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/*/treeFrac/treeFrac_Lmon_MPI-ESM-LR_historical_r1i1p1*.nc 2023-12-27 23:09:47,502 UTC [919785] DEBUG esmvalcore.esgf._search:116 Searching https://esgf.ceda.ac.uk/esg-search for datasets using facets={'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'Lmon', 'product': 'output1,output2', 'variable': 'treeFrac'} 2023-12-27 23:09:47,502 UTC [919785] DEBUG requests_cache.backends:63 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2023-12-27 23:09:47,502 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2023-12-27 23:09:47,502 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2023-12-27 23:09:47,502 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: None 2023-12-27 23:09:47,502 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2023-12-27 23:09:47,504 UTC [919785] DEBUG requests_cache.policy.actions:91 Cache directives from request headers: CacheDirectives() 2023-12-27 23:09:47,504 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-read cache checks: Passed 2023-12-27 23:09:47,504 UTC [919785] DEBUG requests_cache.policy.actions:205 Post-read cache actions: CacheActions(expire_after=86400, send_request=True) 2023-12-27 23:09:47,505 UTC [919785] DEBUG urllib3.connectionpool:1052 Starting new HTTPS connection (1): esgf.ceda.ac.uk:443 2023-12-27 23:09:47,690 UTC [919785] DEBUG urllib3.connectionpool:546 https://esgf.ceda.ac.uk:443 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=CMIP5&model=MPI-ESM-LR&ensemble=r1i1p1&experiment=historical&time_frequency=mon&institute=MPI-M&cmor_table=Lmon&product=output1%2Coutput2&variable=treeFrac HTTP/1.1" 200 None 2023-12-27 23:09:47,724 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-write cache checks: Passed 2023-12-27 23:09:47,791 UTC [919785] DEBUG requests_cache.backends.base:106 Closing backend connections 2023-12-27 23:09:47,792 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'modeling_realm' from 'None' to 'land' for cmip5.output1.MPI-M.MPI-ESM-LR.historical.mon.land.Lmon.r1i1p1.v20120625.treeFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc 2023-12-27 23:09:47,792 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'version' from 'None' to 'v20120625' for cmip5.output1.MPI-M.MPI-ESM-LR.historical.mon.land.Lmon.r1i1p1.v20120625.treeFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc 2023-12-27 23:09:47,793 UTC [919785] DEBUG esmvalcore.esgf._search:154 Found the following files matching facets {'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'Lmon', 'product': 'output1,output2', 'variable': 'treeFrac'}: ESGFFile:cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/v20120625/treeFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf.nci.org.au', 'esgf1.dkrz.de'] 2023-12-27 23:09:47,793 UTC [919785] DEBUG esmvalcore.esgf._search:364 Selected files: ESGFFile:cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/v20120625/treeFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf.nci.org.au', 'esgf1.dkrz.de'] 2023-12-27 23:09:47,794 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc 2023-12-27 23:09:47,795 UTC [919785] DEBUG esmvalcore._recipe.recipe:263 Using input files for variable treeFrac of dataset MPI-ESM-LR: /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/v20120625/treeFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc (will be downloaded) /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/v20120315/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0.nc 2023-12-27 23:09:47,795 UTC [919785] INFO esmvalcore._recipe.recipe:520 Found input files for Dataset: treeFrac, Lmon, CMIP5, MPI-ESM-LR, historical, r1i1p1, v20120625, supplementaries: sftlf, fx, r0i0p0, v20120315 2023-12-27 23:09:47,796 UTC [919785] INFO esmvalcore._recipe.recipe:677 PreprocessingTask albedolandcover_cmip5/treeFrac created. 2023-12-27 23:09:47,796 UTC [919785] DEBUG esmvalcore._recipe.recipe:678 PreprocessingTask albedolandcover_cmip5/treeFrac will create the files: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_albedolandcover_20231227_230944/preproc/albedolandcover_cmip5/treeFrac/CMIP5_MPI-ESM-LR_Lmon_historical_r1i1p1_treeFrac_2000-2004.nc 2023-12-27 23:09:47,796 UTC [919785] INFO esmvalcore._recipe.recipe:983 Creating preprocessor task albedolandcover_cmip5/snc 2023-12-27 23:09:47,796 UTC [919785] INFO esmvalcore._recipe.recipe:653 Creating preprocessor 'pp_cmip' task for variable 'snc' 2023-12-27 23:09:47,798 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/LImon/r1i1p1/*/snc/snc_LImon_MPI-ESM-LR_historical_r1i1p1*.nc /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/mon/landIce/LImon/r1i1p1/*/snc/snc_LImon_MPI-ESM-LR_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/LImon/r1i1p1/*/snc/snc_LImon_MPI-ESM-LR_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/landIce/LImon/r1i1p1/*/snc/snc_LImon_MPI-ESM-LR_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/LImon/r1i1p1/*/snc/snc_LImon_MPI-ESM-LR_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/landIce/LImon/r1i1p1/*/snc/snc_LImon_MPI-ESM-LR_historical_r1i1p1*.nc 2023-12-27 23:09:47,798 UTC [919785] DEBUG esmvalcore.esgf._search:116 Searching https://esgf.ceda.ac.uk/esg-search for datasets using facets={'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'LImon', 'product': 'output1,output2', 'variable': 'snc'} 2023-12-27 23:09:47,798 UTC [919785] DEBUG requests_cache.backends:63 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2023-12-27 23:09:47,799 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2023-12-27 23:09:47,799 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2023-12-27 23:09:47,799 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: None 2023-12-27 23:09:47,799 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2023-12-27 23:09:47,801 UTC [919785] DEBUG requests_cache.policy.actions:91 Cache directives from request headers: CacheDirectives() 2023-12-27 23:09:47,801 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-read cache checks: Passed 2023-12-27 23:09:47,801 UTC [919785] DEBUG requests_cache.policy.actions:205 Post-read cache actions: CacheActions(expire_after=86400, send_request=True) 2023-12-27 23:09:47,801 UTC [919785] DEBUG urllib3.connectionpool:1052 Starting new HTTPS connection (1): esgf.ceda.ac.uk:443 2023-12-27 23:09:47,980 UTC [919785] DEBUG urllib3.connectionpool:546 https://esgf.ceda.ac.uk:443 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=CMIP5&model=MPI-ESM-LR&ensemble=r1i1p1&experiment=historical&time_frequency=mon&institute=MPI-M&cmor_table=LImon&product=output1%2Coutput2&variable=snc HTTP/1.1" 200 None 2023-12-27 23:09:48,014 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-write cache checks: Passed 2023-12-27 23:09:48,135 UTC [919785] DEBUG requests_cache.backends.base:106 Closing backend connections 2023-12-27 23:09:48,138 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'modeling_realm' from 'None' to 'landIce' for cmip5.output1.MPI-M.MPI-ESM-LR.historical.mon.landIce.LImon.r1i1p1.v20111006.snc_LImon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc 2023-12-27 23:09:48,138 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'version' from 'None' to 'v20111006' for cmip5.output1.MPI-M.MPI-ESM-LR.historical.mon.landIce.LImon.r1i1p1.v20111006.snc_LImon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc 2023-12-27 23:09:48,138 UTC [919785] DEBUG esmvalcore.esgf._search:154 Found the following files matching facets {'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'LImon', 'product': 'output1,output2', 'variable': 'snc'}: ESGFFile:cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/landIce/LImon/r1i1p1/v20111006/snc_LImon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf.nci.org.au', 'esgf1.dkrz.de'] 2023-12-27 23:09:48,138 UTC [919785] DEBUG esmvalcore.esgf._search:364 Selected files: ESGFFile:cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/landIce/LImon/r1i1p1/v20111006/snc_LImon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf.nci.org.au', 'esgf1.dkrz.de'] 2023-12-27 23:09:48,141 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc 2023-12-27 23:09:48,142 UTC [919785] DEBUG esmvalcore._recipe.recipe:263 Using input files for variable snc of dataset MPI-ESM-LR: /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/landIce/LImon/r1i1p1/v20111006/snc_LImon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc (will be downloaded) /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/v20120315/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0.nc 2023-12-27 23:09:48,142 UTC [919785] INFO esmvalcore._recipe.recipe:520 Found input files for Dataset: snc, LImon, CMIP5, MPI-ESM-LR, historical, r1i1p1, v20111006, supplementaries: sftlf, fx, r0i0p0, v20120315 2023-12-27 23:09:48,143 UTC [919785] INFO esmvalcore._recipe.recipe:677 PreprocessingTask albedolandcover_cmip5/snc created. 2023-12-27 23:09:48,143 UTC [919785] DEBUG esmvalcore._recipe.recipe:678 PreprocessingTask albedolandcover_cmip5/snc will create the files: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_albedolandcover_20231227_230944/preproc/albedolandcover_cmip5/snc/CMIP5_MPI-ESM-LR_LImon_historical_r1i1p1_snc_2000-2004.nc 2023-12-27 23:09:48,143 UTC [919785] INFO esmvalcore._recipe.recipe:983 Creating preprocessor task albedolandcover_cmip5/alb 2023-12-27 23:09:48,143 UTC [919785] INFO esmvalcore._recipe.recipe:653 Creating preprocessor 'pp_cmip' task for variable 'alb' 2023-12-27 23:09:48,143 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/*/alb/alb_Amon_MPI-ESM-LR_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/*/alb/alb_Amon_MPI-ESM-LR_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/*/alb/alb_Amon_MPI-ESM-LR_historical_r1i1p1*.nc 2023-12-27 23:09:48,144 UTC [919785] DEBUG esmvalcore.esgf._search:116 Searching https://esgf.ceda.ac.uk/esg-search for datasets using facets={'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'Amon', 'product': 'output1,output2', 'variable': 'alb'} 2023-12-27 23:09:48,144 UTC [919785] DEBUG requests_cache.backends:63 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2023-12-27 23:09:48,144 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2023-12-27 23:09:48,144 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2023-12-27 23:09:48,144 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: None 2023-12-27 23:09:48,145 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2023-12-27 23:09:48,146 UTC [919785] DEBUG requests_cache.policy.actions:91 Cache directives from request headers: CacheDirectives() 2023-12-27 23:09:48,146 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-read cache checks: Passed 2023-12-27 23:09:48,149 UTC [919785] DEBUG requests_cache.policy.actions:205 Post-read cache actions: CacheActions(expire_after=86400) 2023-12-27 23:09:48,149 UTC [919785] DEBUG requests_cache.backends.base:106 Closing backend connections 2023-12-27 23:09:48,150 UTC [919785] DEBUG esmvalcore.esgf._search:154 Found the following files matching facets {'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'Amon', 'product': 'output1,output2', 'variable': 'alb'}: none 2023-12-27 23:09:48,150 UTC [919785] DEBUG esmvalcore.esgf._search:364 Selected files: 2023-12-27 23:09:48,152 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/*/alb/alb_Amon_MPI-ESM-LR_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/*/alb/alb_Amon_MPI-ESM-LR_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/*/alb/alb_Amon_MPI-ESM-LR_historical_r1i1p1*.nc 2023-12-27 23:09:48,152 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/*/rsdscs/rsdscs_Amon_MPI-ESM-LR_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/*/rsdscs/rsdscs_Amon_MPI-ESM-LR_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/*/rsdscs/rsdscs_Amon_MPI-ESM-LR_historical_r1i1p1*.nc 2023-12-27 23:09:48,153 UTC [919785] DEBUG esmvalcore.esgf._search:116 Searching https://esgf.ceda.ac.uk/esg-search for datasets using facets={'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'Amon', 'product': 'output1,output2', 'variable': 'rsdscs'} 2023-12-27 23:09:48,153 UTC [919785] DEBUG requests_cache.backends:63 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2023-12-27 23:09:48,153 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2023-12-27 23:09:48,153 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2023-12-27 23:09:48,153 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: None 2023-12-27 23:09:48,154 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2023-12-27 23:09:48,155 UTC [919785] DEBUG requests_cache.policy.actions:91 Cache directives from request headers: CacheDirectives() 2023-12-27 23:09:48,155 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-read cache checks: Passed 2023-12-27 23:09:48,155 UTC [919785] DEBUG requests_cache.policy.actions:205 Post-read cache actions: CacheActions(expire_after=86400, send_request=True) 2023-12-27 23:09:48,156 UTC [919785] DEBUG urllib3.connectionpool:1052 Starting new HTTPS connection (1): esgf.ceda.ac.uk:443 2023-12-27 23:09:48,309 UTC [919785] DEBUG urllib3.connectionpool:546 https://esgf.ceda.ac.uk:443 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=CMIP5&model=MPI-ESM-LR&ensemble=r1i1p1&experiment=historical&time_frequency=mon&institute=MPI-M&cmor_table=Amon&product=output1%2Coutput2&variable=rsdscs HTTP/1.1" 200 None 2023-12-27 23:09:48,343 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-write cache checks: Passed 2023-12-27 23:09:48,407 UTC [919785] DEBUG requests_cache.backends.base:106 Closing backend connections 2023-12-27 23:09:48,409 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'modeling_realm' from 'None' to 'atmos' for cmip5.output1.MPI-M.MPI-ESM-LR.historical.mon.atmos.Amon.r1i1p1.v20120315.rsdscs_Amon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc 2023-12-27 23:09:48,409 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'version' from 'None' to 'v20120315' for cmip5.output1.MPI-M.MPI-ESM-LR.historical.mon.atmos.Amon.r1i1p1.v20120315.rsdscs_Amon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc 2023-12-27 23:09:48,409 UTC [919785] DEBUG esmvalcore.esgf._search:154 Found the following files matching facets {'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'Amon', 'product': 'output1,output2', 'variable': 'rsdscs'}: ESGFFile:cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/v20120315/rsdscs_Amon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf.nci.org.au', 'esgf1.dkrz.de'] 2023-12-27 23:09:48,409 UTC [919785] DEBUG esmvalcore.esgf._search:364 Selected files: ESGFFile:cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/v20120315/rsdscs_Amon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf.nci.org.au', 'esgf1.dkrz.de'] 2023-12-27 23:09:48,409 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc 2023-12-27 23:09:48,411 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/*/rsuscs/rsuscs_Amon_MPI-ESM-LR_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/*/rsuscs/rsuscs_Amon_MPI-ESM-LR_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/*/rsuscs/rsuscs_Amon_MPI-ESM-LR_historical_r1i1p1*.nc 2023-12-27 23:09:48,411 UTC [919785] DEBUG esmvalcore.esgf._search:116 Searching https://esgf.ceda.ac.uk/esg-search for datasets using facets={'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'Amon', 'product': 'output1,output2', 'variable': 'rsuscs'} 2023-12-27 23:09:48,411 UTC [919785] DEBUG requests_cache.backends:63 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2023-12-27 23:09:48,411 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2023-12-27 23:09:48,411 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2023-12-27 23:09:48,412 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: None 2023-12-27 23:09:48,412 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2023-12-27 23:09:48,413 UTC [919785] DEBUG requests_cache.policy.actions:91 Cache directives from request headers: CacheDirectives() 2023-12-27 23:09:48,413 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-read cache checks: Passed 2023-12-27 23:09:48,432 UTC [919785] DEBUG requests_cache.policy.actions:205 Post-read cache actions: CacheActions(expire_after=86400, send_request=True) 2023-12-27 23:09:48,432 UTC [919785] DEBUG urllib3.connectionpool:1052 Starting new HTTPS connection (1): esgf.ceda.ac.uk:443 2023-12-27 23:11:35,030 UTC [919785] DEBUG esmvalcore.esgf._search:129 Unable to connect to https://esgf.ceda.ac.uk/esg-search due to HTTPSConnectionPool(host='esgf.ceda.ac.uk', port=443): Max retries exceeded with url: /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=CMIP5&model=MPI-ESM-LR&ensemble=r1i1p1&experiment=historical&time_frequency=mon&institute=MPI-M&cmor_table=Amon&product=output1%2Coutput2&variable=rsuscs (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1006)'))) 2023-12-27 23:11:35,030 UTC [919785] DEBUG esmvalcore.esgf._search:116 Searching https://esgf-node.llnl.gov/esg-search for datasets using facets={'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'Amon', 'product': 'output1,output2', 'variable': 'rsuscs'} 2023-12-27 23:11:35,030 UTC [919785] DEBUG requests_cache.backends:63 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2023-12-27 23:11:35,030 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2023-12-27 23:11:35,031 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2023-12-27 23:11:35,031 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: None 2023-12-27 23:11:35,031 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2023-12-27 23:11:35,033 UTC [919785] DEBUG requests_cache.policy.actions:91 Cache directives from request headers: CacheDirectives() 2023-12-27 23:11:35,033 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-read cache checks: Passed 2023-12-27 23:11:35,033 UTC [919785] DEBUG requests_cache.policy.actions:205 Post-read cache actions: CacheActions(expire_after=86400, send_request=True) 2023-12-27 23:11:35,034 UTC [919785] DEBUG urllib3.connectionpool:1052 Starting new HTTPS connection (1): esgf-node.llnl.gov:443 2023-12-27 23:11:36,259 UTC [919785] DEBUG urllib3.connectionpool:546 https://esgf-node.llnl.gov:443 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=CMIP5&model=MPI-ESM-LR&ensemble=r1i1p1&experiment=historical&time_frequency=mon&institute=MPI-M&cmor_table=Amon&product=output1%2Coutput2&variable=rsuscs HTTP/1.1" 200 None 2023-12-27 23:11:36,260 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-write cache checks: Passed 2023-12-27 23:11:36,322 UTC [919785] DEBUG requests_cache.backends.base:106 Closing backend connections 2023-12-27 23:11:36,322 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'modeling_realm' from 'None' to 'atmos' for cmip5.output1.MPI-M.MPI-ESM-LR.historical.mon.atmos.Amon.r1i1p1.v20120315.rsuscs_Amon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc 2023-12-27 23:11:36,322 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'version' from 'None' to 'v20120315' for cmip5.output1.MPI-M.MPI-ESM-LR.historical.mon.atmos.Amon.r1i1p1.v20120315.rsuscs_Amon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc 2023-12-27 23:11:36,323 UTC [919785] DEBUG esmvalcore.esgf._search:154 Found the following files matching facets {'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'Amon', 'product': 'output1,output2', 'variable': 'rsuscs'}: ESGFFile:cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/v20120315/rsuscs_Amon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf.nci.org.au', 'esgf1.dkrz.de'] 2023-12-27 23:11:36,323 UTC [919785] DEBUG esmvalcore.esgf._search:364 Selected files: ESGFFile:cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/v20120315/rsuscs_Amon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf.nci.org.au', 'esgf1.dkrz.de'] 2023-12-27 23:11:36,323 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc 2023-12-27 23:11:36,324 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc 2023-12-27 23:11:36,325 UTC [919785] DEBUG esmvalcore._recipe.recipe:263 Using input files for variable rsdscs of dataset MPI-ESM-LR: /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/v20120315/rsdscs_Amon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc (will be downloaded) /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/v20120315/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0.nc 2023-12-27 23:11:36,325 UTC [919785] DEBUG esmvalcore._recipe.recipe:263 Using input files for variable rsuscs of dataset MPI-ESM-LR: /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/v20120315/rsuscs_Amon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc (will be downloaded) /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/v20120315/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0.nc 2023-12-27 23:11:36,325 UTC [919785] INFO esmvalcore._recipe.recipe:520 Found input files for Dataset: alb, Amon, CMIP5, MPI-ESM-LR, historical, r1i1p1, v20120315, supplementaries: sftlf, fx, r0i0p0 2023-12-27 23:11:36,326 UTC [919785] INFO esmvalcore._recipe.recipe:677 PreprocessingTask albedolandcover_cmip5/alb created. 2023-12-27 23:11:36,326 UTC [919785] DEBUG esmvalcore._recipe.recipe:678 PreprocessingTask albedolandcover_cmip5/alb will create the files: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_albedolandcover_20231227_230944/preproc/albedolandcover_cmip5/alb/CMIP5_MPI-ESM-LR_Amon_historical_r1i1p1_alb_2000-2004.nc 2023-12-27 23:11:36,326 UTC [919785] INFO esmvalcore._recipe.recipe:983 Creating preprocessor task albedolandcover_cmip5/cropFrac 2023-12-27 23:11:36,326 UTC [919785] INFO esmvalcore._recipe.recipe:653 Creating preprocessor 'pp_cmip' task for variable 'cropFrac' 2023-12-27 23:11:36,330 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/*/cropFrac/cropFrac_Lmon_MPI-ESM-LR_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/*/cropFrac/cropFrac_Lmon_MPI-ESM-LR_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/*/cropFrac/cropFrac_Lmon_MPI-ESM-LR_historical_r1i1p1*.nc 2023-12-27 23:11:36,330 UTC [919785] DEBUG esmvalcore.esgf._search:116 Searching https://esgf-node.llnl.gov/esg-search for datasets using facets={'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'Lmon', 'product': 'output1,output2', 'variable': 'cropFrac'} 2023-12-27 23:11:36,331 UTC [919785] DEBUG requests_cache.backends:63 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2023-12-27 23:11:36,331 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2023-12-27 23:11:36,331 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2023-12-27 23:11:36,331 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: None 2023-12-27 23:11:36,331 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2023-12-27 23:11:36,333 UTC [919785] DEBUG requests_cache.policy.actions:91 Cache directives from request headers: CacheDirectives() 2023-12-27 23:11:36,333 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-read cache checks: Passed 2023-12-27 23:11:36,333 UTC [919785] DEBUG requests_cache.policy.actions:205 Post-read cache actions: CacheActions(expire_after=86400, send_request=True) 2023-12-27 23:11:36,334 UTC [919785] DEBUG urllib3.connectionpool:1052 Starting new HTTPS connection (1): esgf-node.llnl.gov:443 2023-12-27 23:11:37,300 UTC [919785] DEBUG urllib3.connectionpool:546 https://esgf-node.llnl.gov:443 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=CMIP5&model=MPI-ESM-LR&ensemble=r1i1p1&experiment=historical&time_frequency=mon&institute=MPI-M&cmor_table=Lmon&product=output1%2Coutput2&variable=cropFrac HTTP/1.1" 200 None 2023-12-27 23:11:37,302 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-write cache checks: Passed 2023-12-27 23:11:37,354 UTC [919785] DEBUG requests_cache.backends.base:106 Closing backend connections 2023-12-27 23:11:37,354 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'modeling_realm' from 'None' to 'land' for cmip5.output1.MPI-M.MPI-ESM-LR.historical.mon.land.Lmon.r1i1p1.v20120625.cropFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc 2023-12-27 23:11:37,355 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'version' from 'None' to 'v20120625' for cmip5.output1.MPI-M.MPI-ESM-LR.historical.mon.land.Lmon.r1i1p1.v20120625.cropFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc 2023-12-27 23:11:37,355 UTC [919785] DEBUG esmvalcore.esgf._search:154 Found the following files matching facets {'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'Lmon', 'product': 'output1,output2', 'variable': 'cropFrac'}: ESGFFile:cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/v20120625/cropFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf.nci.org.au', 'esgf1.dkrz.de'] 2023-12-27 23:11:37,355 UTC [919785] DEBUG esmvalcore.esgf._search:364 Selected files: ESGFFile:cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/v20120625/cropFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf.nci.org.au', 'esgf1.dkrz.de'] 2023-12-27 23:11:37,355 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc 2023-12-27 23:11:37,356 UTC [919785] DEBUG esmvalcore._recipe.recipe:263 Using input files for variable cropFrac of dataset MPI-ESM-LR: /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/v20120625/cropFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc (will be downloaded) /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/v20120315/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0.nc 2023-12-27 23:11:37,356 UTC [919785] INFO esmvalcore._recipe.recipe:520 Found input files for Dataset: cropFrac, Lmon, CMIP5, MPI-ESM-LR, historical, r1i1p1, v20120625, supplementaries: sftlf, fx, r0i0p0, v20120315 2023-12-27 23:11:37,357 UTC [919785] INFO esmvalcore._recipe.recipe:677 PreprocessingTask albedolandcover_cmip5/cropFrac created. 2023-12-27 23:11:37,357 UTC [919785] DEBUG esmvalcore._recipe.recipe:678 PreprocessingTask albedolandcover_cmip5/cropFrac will create the files: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_albedolandcover_20231227_230944/preproc/albedolandcover_cmip5/cropFrac/CMIP5_MPI-ESM-LR_Lmon_historical_r1i1p1_cropFrac_2000-2004.nc 2023-12-27 23:11:37,357 UTC [919785] INFO esmvalcore._recipe.recipe:983 Creating preprocessor task albedolandcover_cmip5/grassFrac 2023-12-27 23:11:37,357 UTC [919785] INFO esmvalcore._recipe.recipe:653 Creating preprocessor 'pp_cmip' task for variable 'grassFrac' 2023-12-27 23:11:37,359 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/*/grassFrac/grassFrac_Lmon_MPI-ESM-LR_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/*/grassFrac/grassFrac_Lmon_MPI-ESM-LR_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/*/grassFrac/grassFrac_Lmon_MPI-ESM-LR_historical_r1i1p1*.nc 2023-12-27 23:11:37,360 UTC [919785] DEBUG esmvalcore.esgf._search:116 Searching https://esgf-node.llnl.gov/esg-search for datasets using facets={'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'Lmon', 'product': 'output1,output2', 'variable': 'grassFrac'} 2023-12-27 23:11:37,360 UTC [919785] DEBUG requests_cache.backends:63 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2023-12-27 23:11:37,360 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2023-12-27 23:11:37,360 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2023-12-27 23:11:37,360 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: None 2023-12-27 23:11:37,361 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2023-12-27 23:11:37,362 UTC [919785] DEBUG requests_cache.policy.actions:91 Cache directives from request headers: CacheDirectives() 2023-12-27 23:11:37,362 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-read cache checks: Passed 2023-12-27 23:11:37,381 UTC [919785] DEBUG requests_cache.policy.actions:205 Post-read cache actions: CacheActions(expire_after=86400, send_request=True) 2023-12-27 23:11:37,382 UTC [919785] DEBUG urllib3.connectionpool:1052 Starting new HTTPS connection (1): esgf-node.llnl.gov:443 2023-12-27 23:11:38,089 UTC [919785] DEBUG urllib3.connectionpool:546 https://esgf-node.llnl.gov:443 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=CMIP5&model=MPI-ESM-LR&ensemble=r1i1p1&experiment=historical&time_frequency=mon&institute=MPI-M&cmor_table=Lmon&product=output1%2Coutput2&variable=grassFrac HTTP/1.1" 200 None 2023-12-27 23:11:38,092 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-write cache checks: Passed 2023-12-27 23:11:38,187 UTC [919785] DEBUG requests_cache.backends.base:106 Closing backend connections 2023-12-27 23:11:38,187 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'modeling_realm' from 'None' to 'land' for cmip5.output1.MPI-M.MPI-ESM-LR.historical.mon.land.Lmon.r1i1p1.v20120625.grassFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc 2023-12-27 23:11:38,188 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'version' from 'None' to 'v20120625' for cmip5.output1.MPI-M.MPI-ESM-LR.historical.mon.land.Lmon.r1i1p1.v20120625.grassFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc 2023-12-27 23:11:38,188 UTC [919785] DEBUG esmvalcore.esgf._search:154 Found the following files matching facets {'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'Lmon', 'product': 'output1,output2', 'variable': 'grassFrac'}: ESGFFile:cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/v20120625/grassFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf.nci.org.au', 'esgf1.dkrz.de'] 2023-12-27 23:11:38,188 UTC [919785] DEBUG esmvalcore.esgf._search:364 Selected files: ESGFFile:cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/v20120625/grassFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf.nci.org.au', 'esgf1.dkrz.de'] 2023-12-27 23:11:38,188 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc 2023-12-27 23:11:38,189 UTC [919785] DEBUG esmvalcore._recipe.recipe:263 Using input files for variable grassFrac of dataset MPI-ESM-LR: /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/v20120625/grassFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc (will be downloaded) /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/v20120315/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0.nc 2023-12-27 23:11:38,189 UTC [919785] INFO esmvalcore._recipe.recipe:520 Found input files for Dataset: grassFrac, Lmon, CMIP5, MPI-ESM-LR, historical, r1i1p1, v20120625, supplementaries: sftlf, fx, r0i0p0, v20120315 2023-12-27 23:11:38,190 UTC [919785] INFO esmvalcore._recipe.recipe:677 PreprocessingTask albedolandcover_cmip5/grassFrac created. 2023-12-27 23:11:38,190 UTC [919785] DEBUG esmvalcore._recipe.recipe:678 PreprocessingTask albedolandcover_cmip5/grassFrac will create the files: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_albedolandcover_20231227_230944/preproc/albedolandcover_cmip5/grassFrac/CMIP5_MPI-ESM-LR_Lmon_historical_r1i1p1_grassFrac_2000-2004.nc 2023-12-27 23:11:38,190 UTC [919785] INFO esmvalcore._recipe.recipe:983 Creating preprocessor task albedolandcover_cmip5/shrubFrac 2023-12-27 23:11:38,190 UTC [919785] INFO esmvalcore._recipe.recipe:653 Creating preprocessor 'pp_cmip' task for variable 'shrubFrac' 2023-12-27 23:11:38,192 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/*/shrubFrac/shrubFrac_Lmon_MPI-ESM-LR_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/*/shrubFrac/shrubFrac_Lmon_MPI-ESM-LR_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/*/shrubFrac/shrubFrac_Lmon_MPI-ESM-LR_historical_r1i1p1*.nc 2023-12-27 23:11:38,193 UTC [919785] DEBUG esmvalcore.esgf._search:116 Searching https://esgf-node.llnl.gov/esg-search for datasets using facets={'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'Lmon', 'product': 'output1,output2', 'variable': 'shrubFrac'} 2023-12-27 23:11:38,193 UTC [919785] DEBUG requests_cache.backends:63 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2023-12-27 23:11:38,193 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2023-12-27 23:11:38,193 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2023-12-27 23:11:38,193 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: None 2023-12-27 23:11:38,194 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2023-12-27 23:11:38,195 UTC [919785] DEBUG requests_cache.policy.actions:91 Cache directives from request headers: CacheDirectives() 2023-12-27 23:11:38,195 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-read cache checks: Passed 2023-12-27 23:11:38,195 UTC [919785] DEBUG requests_cache.policy.actions:205 Post-read cache actions: CacheActions(expire_after=86400, send_request=True) 2023-12-27 23:11:38,196 UTC [919785] DEBUG urllib3.connectionpool:1052 Starting new HTTPS connection (1): esgf-node.llnl.gov:443 2023-12-27 23:11:38,914 UTC [919785] DEBUG urllib3.connectionpool:546 https://esgf-node.llnl.gov:443 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=CMIP5&model=MPI-ESM-LR&ensemble=r1i1p1&experiment=historical&time_frequency=mon&institute=MPI-M&cmor_table=Lmon&product=output1%2Coutput2&variable=shrubFrac HTTP/1.1" 200 None 2023-12-27 23:11:38,916 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-write cache checks: Passed 2023-12-27 23:11:38,966 UTC [919785] DEBUG requests_cache.backends.base:106 Closing backend connections 2023-12-27 23:11:38,966 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'modeling_realm' from 'None' to 'land' for cmip5.output1.MPI-M.MPI-ESM-LR.historical.mon.land.Lmon.r1i1p1.v20120625.shrubFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc 2023-12-27 23:11:38,966 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'version' from 'None' to 'v20120625' for cmip5.output1.MPI-M.MPI-ESM-LR.historical.mon.land.Lmon.r1i1p1.v20120625.shrubFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc 2023-12-27 23:11:38,966 UTC [919785] DEBUG esmvalcore.esgf._search:154 Found the following files matching facets {'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'Lmon', 'product': 'output1,output2', 'variable': 'shrubFrac'}: ESGFFile:cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/v20120625/shrubFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf.nci.org.au', 'esgf1.dkrz.de'] 2023-12-27 23:11:38,967 UTC [919785] DEBUG esmvalcore.esgf._search:364 Selected files: ESGFFile:cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/v20120625/shrubFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf.nci.org.au', 'esgf1.dkrz.de'] 2023-12-27 23:11:38,967 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc 2023-12-27 23:11:38,968 UTC [919785] DEBUG esmvalcore._recipe.recipe:263 Using input files for variable shrubFrac of dataset MPI-ESM-LR: /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/v20120625/shrubFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc (will be downloaded) /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/v20120315/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0.nc 2023-12-27 23:11:38,968 UTC [919785] INFO esmvalcore._recipe.recipe:520 Found input files for Dataset: shrubFrac, Lmon, CMIP5, MPI-ESM-LR, historical, r1i1p1, v20120625, supplementaries: sftlf, fx, r0i0p0, v20120315 2023-12-27 23:11:38,969 UTC [919785] INFO esmvalcore._recipe.recipe:677 PreprocessingTask albedolandcover_cmip5/shrubFrac created. 2023-12-27 23:11:38,969 UTC [919785] DEBUG esmvalcore._recipe.recipe:678 PreprocessingTask albedolandcover_cmip5/shrubFrac will create the files: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_albedolandcover_20231227_230944/preproc/albedolandcover_cmip5/shrubFrac/CMIP5_MPI-ESM-LR_Lmon_historical_r1i1p1_shrubFrac_2000-2004.nc 2023-12-27 23:11:38,969 UTC [919785] INFO esmvalcore._recipe.recipe:983 Creating preprocessor task albedolandcover_cmip5/pastureFrac 2023-12-27 23:11:38,969 UTC [919785] INFO esmvalcore._recipe.recipe:653 Creating preprocessor 'pp_cmip' task for variable 'pastureFrac' 2023-12-27 23:11:38,971 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/*/pastureFrac/pastureFrac_Lmon_MPI-ESM-LR_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/*/pastureFrac/pastureFrac_Lmon_MPI-ESM-LR_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/*/pastureFrac/pastureFrac_Lmon_MPI-ESM-LR_historical_r1i1p1*.nc 2023-12-27 23:11:38,971 UTC [919785] DEBUG esmvalcore.esgf._search:116 Searching https://esgf-node.llnl.gov/esg-search for datasets using facets={'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'Lmon', 'product': 'output1,output2', 'variable': 'pastureFrac'} 2023-12-27 23:11:38,971 UTC [919785] DEBUG requests_cache.backends:63 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2023-12-27 23:11:38,971 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2023-12-27 23:11:38,971 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2023-12-27 23:11:38,972 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: None 2023-12-27 23:11:38,972 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2023-12-27 23:11:38,973 UTC [919785] DEBUG requests_cache.policy.actions:91 Cache directives from request headers: CacheDirectives() 2023-12-27 23:11:38,973 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-read cache checks: Passed 2023-12-27 23:11:38,973 UTC [919785] DEBUG requests_cache.policy.actions:205 Post-read cache actions: CacheActions(expire_after=86400, send_request=True) 2023-12-27 23:11:38,974 UTC [919785] DEBUG urllib3.connectionpool:1052 Starting new HTTPS connection (1): esgf-node.llnl.gov:443 2023-12-27 23:11:39,885 UTC [919785] DEBUG urllib3.connectionpool:546 https://esgf-node.llnl.gov:443 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=CMIP5&model=MPI-ESM-LR&ensemble=r1i1p1&experiment=historical&time_frequency=mon&institute=MPI-M&cmor_table=Lmon&product=output1%2Coutput2&variable=pastureFrac HTTP/1.1" 200 None 2023-12-27 23:11:39,888 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-write cache checks: Passed 2023-12-27 23:11:39,949 UTC [919785] DEBUG requests_cache.backends.base:106 Closing backend connections 2023-12-27 23:11:39,950 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'modeling_realm' from 'None' to 'land' for cmip5.output1.MPI-M.MPI-ESM-LR.historical.mon.land.Lmon.r1i1p1.v20120625.pastureFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc 2023-12-27 23:11:39,950 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'version' from 'None' to 'v20120625' for cmip5.output1.MPI-M.MPI-ESM-LR.historical.mon.land.Lmon.r1i1p1.v20120625.pastureFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc 2023-12-27 23:11:39,950 UTC [919785] DEBUG esmvalcore.esgf._search:154 Found the following files matching facets {'project': 'CMIP5', 'model': 'MPI-ESM-LR', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'time_frequency': 'mon', 'institute': 'MPI-M', 'cmor_table': 'Lmon', 'product': 'output1,output2', 'variable': 'pastureFrac'}: ESGFFile:cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/v20120625/pastureFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf.nci.org.au', 'esgf1.dkrz.de'] 2023-12-27 23:11:39,950 UTC [919785] DEBUG esmvalcore.esgf._search:364 Selected files: ESGFFile:cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/v20120625/pastureFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf.nci.org.au', 'esgf1.dkrz.de'] 2023-12-27 23:11:39,950 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0*.nc 2023-12-27 23:11:39,952 UTC [919785] DEBUG esmvalcore._recipe.recipe:263 Using input files for variable pastureFrac of dataset MPI-ESM-LR: /scratch/tomast/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/land/Lmon/r1i1p1/v20120625/pastureFrac_Lmon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc (will be downloaded) /projects/NS9560K-datalake/ESGF/cmip5/output1/MPI-M/MPI-ESM-LR/historical/fx/atmos/fx/r0i0p0/v20120315/sftlf/sftlf_fx_MPI-ESM-LR_historical_r0i0p0.nc 2023-12-27 23:11:39,952 UTC [919785] INFO esmvalcore._recipe.recipe:520 Found input files for Dataset: pastureFrac, Lmon, CMIP5, MPI-ESM-LR, historical, r1i1p1, v20120625, supplementaries: sftlf, fx, r0i0p0, v20120315 2023-12-27 23:11:39,952 UTC [919785] INFO esmvalcore._recipe.recipe:677 PreprocessingTask albedolandcover_cmip5/pastureFrac created. 2023-12-27 23:11:39,952 UTC [919785] DEBUG esmvalcore._recipe.recipe:678 PreprocessingTask albedolandcover_cmip5/pastureFrac will create the files: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_albedolandcover_20231227_230944/preproc/albedolandcover_cmip5/pastureFrac/CMIP5_MPI-ESM-LR_Lmon_historical_r1i1p1_pastureFrac_2000-2004.nc 2023-12-27 23:11:39,953 UTC [919785] INFO esmvalcore._recipe.recipe:1008 Creating tasks for diagnostic albedolandcover_cmip6 2023-12-27 23:11:39,953 UTC [919785] INFO esmvalcore._recipe.recipe:930 Creating diagnostic task albedolandcover_cmip6/albedolandcover 2023-12-27 23:11:39,953 UTC [919785] INFO esmvalcore._recipe.recipe:983 Creating preprocessor task albedolandcover_cmip6/treeFrac 2023-12-27 23:11:39,953 UTC [919785] INFO esmvalcore._recipe.recipe:653 Creating preprocessor 'pp_cmip' task for variable 'treeFrac' 2023-12-27 23:11:39,955 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/treeFrac/gn/*/treeFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9034K/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/treeFrac/gn/*/treeFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/treeFrac/gn/*/treeFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/treeFrac/gn/*/treeFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/treeFrac/gn/*/treeFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/treeFrac/gn/*/treeFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/treeFrac/gn/*/treeFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/treeFrac/gn/*/treeFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc 2023-12-27 23:11:40,172 UTC [919785] DEBUG esmvalcore.esgf._search:116 Searching https://esgf-node.llnl.gov/esg-search for datasets using facets={'project': 'CMIP6', 'activity_drs': 'CMIP', 'source_id': 'HadGEM3-GC31-LL', 'member_id': 'r1i1p1f3', 'experiment_id': 'historical', 'institution_id': 'MOHC,NERC', 'grid_label': 'gn', 'table_id': 'Lmon', 'variable': 'treeFrac'} 2023-12-27 23:11:40,173 UTC [919785] DEBUG requests_cache.backends:63 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2023-12-27 23:11:40,173 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2023-12-27 23:11:40,173 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2023-12-27 23:11:40,173 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: None 2023-12-27 23:11:40,173 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2023-12-27 23:11:40,175 UTC [919785] DEBUG requests_cache.policy.actions:91 Cache directives from request headers: CacheDirectives() 2023-12-27 23:11:40,175 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-read cache checks: Passed 2023-12-27 23:11:40,194 UTC [919785] DEBUG requests_cache.policy.actions:205 Post-read cache actions: CacheActions(expire_after=86400, send_request=True) 2023-12-27 23:11:40,194 UTC [919785] DEBUG urllib3.connectionpool:1052 Starting new HTTPS connection (1): esgf-node.llnl.gov:443 2023-12-27 23:11:41,132 UTC [919785] DEBUG urllib3.connectionpool:546 https://esgf-node.llnl.gov:443 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=CMIP6&activity_drs=CMIP&source_id=HadGEM3-GC31-LL&member_id=r1i1p1f3&experiment_id=historical&institution_id=MOHC%2CNERC&grid_label=gn&table_id=Lmon&variable=treeFrac HTTP/1.1" 200 None 2023-12-27 23:11:41,300 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-write cache checks: Passed 2023-12-27 23:11:41,354 UTC [919785] DEBUG requests_cache.backends.base:106 Closing backend connections 2023-12-27 23:11:41,355 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'version' from 'None' to 'v20190624' for CMIP6.CMIP.MOHC.HadGEM3-GC31-LL.historical.r1i1p1f3.Lmon.treeFrac.gn.v20190624.treeFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_185001-194912.nc 2023-12-27 23:11:41,355 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'version' from 'None' to 'v20190624' for CMIP6.CMIP.MOHC.HadGEM3-GC31-LL.historical.r1i1p1f3.Lmon.treeFrac.gn.v20190624.treeFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_195001-201412.nc 2023-12-27 23:11:41,355 UTC [919785] DEBUG esmvalcore.esgf._search:154 Found the following files matching facets {'project': 'CMIP6', 'activity_drs': 'CMIP', 'source_id': 'HadGEM3-GC31-LL', 'member_id': 'r1i1p1f3', 'experiment_id': 'historical', 'institution_id': 'MOHC,NERC', 'grid_label': 'gn', 'table_id': 'Lmon', 'variable': 'treeFrac'}: ESGFFile:CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/treeFrac/gn/v20190624/treeFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_185001-194912.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf3.dkrz.de'] ESGFFile:CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/treeFrac/gn/v20190624/treeFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_195001-201412.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf3.dkrz.de'] 2023-12-27 23:11:41,355 UTC [919785] DEBUG esmvalcore.esgf._search:364 Selected files: ESGFFile:CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/treeFrac/gn/v20190624/treeFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_195001-201412.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf3.dkrz.de'] 2023-12-27 23:11:41,356 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc 2023-12-27 23:11:41,357 UTC [919785] DEBUG esmvalcore._recipe.recipe:263 Using input files for variable treeFrac of dataset HadGEM3-GC31-LL: /scratch/tomast/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/treeFrac/gn/v20190624/treeFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_195001-201412.nc (will be downloaded) /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/v20190709/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn.nc 2023-12-27 23:11:41,357 UTC [919785] INFO esmvalcore._recipe.recipe:520 Found input files for Dataset: treeFrac, Lmon, CMIP6, HadGEM3-GC31-LL, CMIP, historical, r1i1p1f3, gn, v20190624, supplementaries: sftlf, fx, piControl, r1i1p1f1, v20190709 2023-12-27 23:11:41,358 UTC [919785] INFO esmvalcore._recipe.recipe:677 PreprocessingTask albedolandcover_cmip6/treeFrac created. 2023-12-27 23:11:41,358 UTC [919785] DEBUG esmvalcore._recipe.recipe:678 PreprocessingTask albedolandcover_cmip6/treeFrac will create the files: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_albedolandcover_20231227_230944/preproc/albedolandcover_cmip6/treeFrac/CMIP6_HadGEM3-GC31-LL_Lmon_historical_r1i1p1f3_treeFrac_gn_2000-2004.nc 2023-12-27 23:11:41,358 UTC [919785] INFO esmvalcore._recipe.recipe:983 Creating preprocessor task albedolandcover_cmip6/snc 2023-12-27 23:11:41,358 UTC [919785] INFO esmvalcore._recipe.recipe:653 Creating preprocessor 'pp_cmip' task for variable 'snc' 2023-12-27 23:11:41,360 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/LImon/snc/gn/*/snc_LImon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9034K/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/LImon/snc/gn/*/snc_LImon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/LImon/snc/gn/*/snc_LImon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/LImon/snc/gn/*/snc_LImon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/LImon/snc/gn/*/snc_LImon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/LImon/snc/gn/*/snc_LImon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/LImon/snc/gn/*/snc_LImon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/LImon/snc/gn/*/snc_LImon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc 2023-12-27 23:11:41,360 UTC [919785] DEBUG esmvalcore.esgf._search:116 Searching https://esgf-node.llnl.gov/esg-search for datasets using facets={'project': 'CMIP6', 'activity_drs': 'CMIP', 'source_id': 'HadGEM3-GC31-LL', 'member_id': 'r1i1p1f3', 'experiment_id': 'historical', 'institution_id': 'MOHC,NERC', 'grid_label': 'gn', 'table_id': 'LImon', 'variable': 'snc'} 2023-12-27 23:11:41,360 UTC [919785] DEBUG requests_cache.backends:63 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2023-12-27 23:11:41,360 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2023-12-27 23:11:41,360 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2023-12-27 23:11:41,361 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: None 2023-12-27 23:11:41,361 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2023-12-27 23:11:41,362 UTC [919785] DEBUG requests_cache.policy.actions:91 Cache directives from request headers: CacheDirectives() 2023-12-27 23:11:41,362 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-read cache checks: Passed 2023-12-27 23:11:41,362 UTC [919785] DEBUG requests_cache.policy.actions:205 Post-read cache actions: CacheActions(expire_after=86400, send_request=True) 2023-12-27 23:11:41,363 UTC [919785] DEBUG urllib3.connectionpool:1052 Starting new HTTPS connection (1): esgf-node.llnl.gov:443 2023-12-27 23:11:42,086 UTC [919785] DEBUG urllib3.connectionpool:546 https://esgf-node.llnl.gov:443 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=CMIP6&activity_drs=CMIP&source_id=HadGEM3-GC31-LL&member_id=r1i1p1f3&experiment_id=historical&institution_id=MOHC%2CNERC&grid_label=gn&table_id=LImon&variable=snc HTTP/1.1" 200 None 2023-12-27 23:11:42,254 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-write cache checks: Passed 2023-12-27 23:11:42,309 UTC [919785] DEBUG requests_cache.backends.base:106 Closing backend connections 2023-12-27 23:11:42,309 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'version' from 'None' to 'v20190624' for CMIP6.CMIP.MOHC.HadGEM3-GC31-LL.historical.r1i1p1f3.LImon.snc.gn.v20190624.snc_LImon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_185001-194912.nc 2023-12-27 23:11:42,309 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'version' from 'None' to 'v20190624' for CMIP6.CMIP.MOHC.HadGEM3-GC31-LL.historical.r1i1p1f3.LImon.snc.gn.v20190624.snc_LImon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_195001-201412.nc 2023-12-27 23:11:42,310 UTC [919785] DEBUG esmvalcore.esgf._search:154 Found the following files matching facets {'project': 'CMIP6', 'activity_drs': 'CMIP', 'source_id': 'HadGEM3-GC31-LL', 'member_id': 'r1i1p1f3', 'experiment_id': 'historical', 'institution_id': 'MOHC,NERC', 'grid_label': 'gn', 'table_id': 'LImon', 'variable': 'snc'}: ESGFFile:CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/LImon/snc/gn/v20190624/snc_LImon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_185001-194912.nc on hosts ['aims3.llnl.gov', 'esgf-data04.diasjp.net', 'esgf.ceda.ac.uk', 'esgf3.dkrz.de'] ESGFFile:CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/LImon/snc/gn/v20190624/snc_LImon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_195001-201412.nc on hosts ['aims3.llnl.gov', 'esgf-data04.diasjp.net', 'esgf.ceda.ac.uk', 'esgf3.dkrz.de'] 2023-12-27 23:11:42,310 UTC [919785] DEBUG esmvalcore.esgf._search:364 Selected files: ESGFFile:CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/LImon/snc/gn/v20190624/snc_LImon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_195001-201412.nc on hosts ['aims3.llnl.gov', 'esgf-data04.diasjp.net', 'esgf.ceda.ac.uk', 'esgf3.dkrz.de'] 2023-12-27 23:11:42,310 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc 2023-12-27 23:11:42,311 UTC [919785] DEBUG esmvalcore._recipe.recipe:263 Using input files for variable snc of dataset HadGEM3-GC31-LL: /scratch/tomast/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/LImon/snc/gn/v20190624/snc_LImon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_195001-201412.nc (will be downloaded) /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/v20190709/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn.nc 2023-12-27 23:11:42,311 UTC [919785] INFO esmvalcore._recipe.recipe:520 Found input files for Dataset: snc, LImon, CMIP6, HadGEM3-GC31-LL, CMIP, historical, r1i1p1f3, gn, v20190624, supplementaries: sftlf, fx, piControl, r1i1p1f1, v20190709 2023-12-27 23:11:42,312 UTC [919785] INFO esmvalcore._recipe.recipe:677 PreprocessingTask albedolandcover_cmip6/snc created. 2023-12-27 23:11:42,312 UTC [919785] DEBUG esmvalcore._recipe.recipe:678 PreprocessingTask albedolandcover_cmip6/snc will create the files: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_albedolandcover_20231227_230944/preproc/albedolandcover_cmip6/snc/CMIP6_HadGEM3-GC31-LL_LImon_historical_r1i1p1f3_snc_gn_2000-2004.nc 2023-12-27 23:11:42,312 UTC [919785] INFO esmvalcore._recipe.recipe:983 Creating preprocessor task albedolandcover_cmip6/alb 2023-12-27 23:11:42,312 UTC [919785] INFO esmvalcore._recipe.recipe:653 Creating preprocessor 'pp_cmip' task for variable 'alb' 2023-12-27 23:11:42,312 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9034K/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc 2023-12-27 23:11:42,313 UTC [919785] DEBUG esmvalcore.esgf._search:116 Searching https://esgf-node.llnl.gov/esg-search for datasets using facets={'project': 'CMIP6', 'activity_drs': 'CMIP', 'source_id': 'HadGEM3-GC31-LL', 'member_id': 'r1i1p1f3', 'experiment_id': 'historical', 'institution_id': 'MOHC,NERC', 'grid_label': 'gn', 'table_id': 'Amon', 'variable': 'alb'} 2023-12-27 23:11:42,313 UTC [919785] DEBUG requests_cache.backends:63 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2023-12-27 23:11:42,313 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2023-12-27 23:11:42,313 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2023-12-27 23:11:42,313 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: None 2023-12-27 23:11:42,313 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2023-12-27 23:11:42,315 UTC [919785] DEBUG requests_cache.policy.actions:91 Cache directives from request headers: CacheDirectives() 2023-12-27 23:11:42,315 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-read cache checks: Passed 2023-12-27 23:11:42,329 UTC [919785] DEBUG requests_cache.policy.actions:205 Post-read cache actions: CacheActions(expire_after=86400, send_request=True) 2023-12-27 23:11:42,329 UTC [919785] DEBUG urllib3.connectionpool:1052 Starting new HTTPS connection (1): esgf-node.llnl.gov:443 2023-12-27 23:11:43,477 UTC [919785] DEBUG urllib3.connectionpool:546 https://esgf-node.llnl.gov:443 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=CMIP6&activity_drs=CMIP&source_id=HadGEM3-GC31-LL&member_id=r1i1p1f3&experiment_id=historical&institution_id=MOHC%2CNERC&grid_label=gn&table_id=Amon&variable=alb HTTP/1.1" 200 1048 2023-12-27 23:11:43,479 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-write cache checks: Passed 2023-12-27 23:11:43,541 UTC [919785] DEBUG requests_cache.backends.base:106 Closing backend connections 2023-12-27 23:11:43,542 UTC [919785] DEBUG esmvalcore.esgf._search:154 Found the following files matching facets {'project': 'CMIP6', 'activity_drs': 'CMIP', 'source_id': 'HadGEM3-GC31-LL', 'member_id': 'r1i1p1f3', 'experiment_id': 'historical', 'institution_id': 'MOHC,NERC', 'grid_label': 'gn', 'table_id': 'Amon', 'variable': 'alb'}: none 2023-12-27 23:11:43,542 UTC [919785] DEBUG esmvalcore.esgf._search:364 Selected files: 2023-12-27 23:11:43,544 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9034K/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/alb/gn/*/alb_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc 2023-12-27 23:11:43,544 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/rsdscs/gn/*/rsdscs_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9034K/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/rsdscs/gn/*/rsdscs_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/rsdscs/gn/*/rsdscs_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/rsdscs/gn/*/rsdscs_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/rsdscs/gn/*/rsdscs_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/rsdscs/gn/*/rsdscs_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/rsdscs/gn/*/rsdscs_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/rsdscs/gn/*/rsdscs_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc 2023-12-27 23:11:43,551 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc 2023-12-27 23:11:43,551 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/rsuscs/gn/*/rsuscs_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9034K/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/rsuscs/gn/*/rsuscs_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/rsuscs/gn/*/rsuscs_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/rsuscs/gn/*/rsuscs_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/rsuscs/gn/*/rsuscs_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/rsuscs/gn/*/rsuscs_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/rsuscs/gn/*/rsuscs_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/rsuscs/gn/*/rsuscs_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc 2023-12-27 23:11:43,558 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc 2023-12-27 23:11:43,558 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc 2023-12-27 23:11:43,559 UTC [919785] DEBUG esmvalcore._recipe.recipe:263 Using input files for variable rsdscs of dataset HadGEM3-GC31-LL: /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/rsdscs/gn/v20190624/rsdscs_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_195001-201412.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/v20190709/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn.nc 2023-12-27 23:11:43,559 UTC [919785] DEBUG esmvalcore._recipe.recipe:263 Using input files for variable rsuscs of dataset HadGEM3-GC31-LL: /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Amon/rsuscs/gn/v20190624/rsuscs_Amon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_195001-201412.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/v20190709/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn.nc 2023-12-27 23:11:43,559 UTC [919785] INFO esmvalcore._recipe.recipe:520 Found input files for Dataset: alb, Amon, CMIP6, HadGEM3-GC31-LL, CMIP, historical, r1i1p1f3, gn, v20190624, supplementaries: sftlf, fx, piControl, r1i1p1f1, v20190709 2023-12-27 23:11:43,560 UTC [919785] INFO esmvalcore._recipe.recipe:677 PreprocessingTask albedolandcover_cmip6/alb created. 2023-12-27 23:11:43,560 UTC [919785] DEBUG esmvalcore._recipe.recipe:678 PreprocessingTask albedolandcover_cmip6/alb will create the files: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_albedolandcover_20231227_230944/preproc/albedolandcover_cmip6/alb/CMIP6_HadGEM3-GC31-LL_Amon_historical_r1i1p1f3_alb_gn_2000-2004.nc 2023-12-27 23:11:43,560 UTC [919785] INFO esmvalcore._recipe.recipe:983 Creating preprocessor task albedolandcover_cmip6/grassFrac 2023-12-27 23:11:43,560 UTC [919785] INFO esmvalcore._recipe.recipe:653 Creating preprocessor 'pp_cmip' task for variable 'grassFrac' 2023-12-27 23:11:43,562 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/grassFrac/gn/*/grassFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9034K/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/grassFrac/gn/*/grassFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/grassFrac/gn/*/grassFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/grassFrac/gn/*/grassFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/grassFrac/gn/*/grassFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/grassFrac/gn/*/grassFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/grassFrac/gn/*/grassFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/grassFrac/gn/*/grassFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc 2023-12-27 23:11:43,562 UTC [919785] DEBUG esmvalcore.esgf._search:116 Searching https://esgf-node.llnl.gov/esg-search for datasets using facets={'project': 'CMIP6', 'activity_drs': 'CMIP', 'source_id': 'HadGEM3-GC31-LL', 'member_id': 'r1i1p1f3', 'experiment_id': 'historical', 'institution_id': 'MOHC,NERC', 'grid_label': 'gn', 'table_id': 'Lmon', 'variable': 'grassFrac'} 2023-12-27 23:11:43,562 UTC [919785] DEBUG requests_cache.backends:63 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2023-12-27 23:11:43,562 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2023-12-27 23:11:43,562 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2023-12-27 23:11:43,563 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: None 2023-12-27 23:11:43,563 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2023-12-27 23:11:43,564 UTC [919785] DEBUG requests_cache.policy.actions:91 Cache directives from request headers: CacheDirectives() 2023-12-27 23:11:43,564 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-read cache checks: Passed 2023-12-27 23:11:43,564 UTC [919785] DEBUG requests_cache.policy.actions:205 Post-read cache actions: CacheActions(expire_after=86400, send_request=True) 2023-12-27 23:11:43,565 UTC [919785] DEBUG urllib3.connectionpool:1052 Starting new HTTPS connection (1): esgf-node.llnl.gov:443 2023-12-27 23:11:44,430 UTC [919785] DEBUG urllib3.connectionpool:546 https://esgf-node.llnl.gov:443 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=CMIP6&activity_drs=CMIP&source_id=HadGEM3-GC31-LL&member_id=r1i1p1f3&experiment_id=historical&institution_id=MOHC%2CNERC&grid_label=gn&table_id=Lmon&variable=grassFrac HTTP/1.1" 200 None 2023-12-27 23:11:44,598 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-write cache checks: Passed 2023-12-27 23:11:44,665 UTC [919785] DEBUG requests_cache.backends.base:106 Closing backend connections 2023-12-27 23:11:44,666 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'version' from 'None' to 'v20190624' for CMIP6.CMIP.MOHC.HadGEM3-GC31-LL.historical.r1i1p1f3.Lmon.grassFrac.gn.v20190624.grassFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_185001-194912.nc 2023-12-27 23:11:44,666 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'version' from 'None' to 'v20190624' for CMIP6.CMIP.MOHC.HadGEM3-GC31-LL.historical.r1i1p1f3.Lmon.grassFrac.gn.v20190624.grassFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_195001-201412.nc 2023-12-27 23:11:44,666 UTC [919785] DEBUG esmvalcore.esgf._search:154 Found the following files matching facets {'project': 'CMIP6', 'activity_drs': 'CMIP', 'source_id': 'HadGEM3-GC31-LL', 'member_id': 'r1i1p1f3', 'experiment_id': 'historical', 'institution_id': 'MOHC,NERC', 'grid_label': 'gn', 'table_id': 'Lmon', 'variable': 'grassFrac'}: ESGFFile:CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/grassFrac/gn/v20190624/grassFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_185001-194912.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf3.dkrz.de'] ESGFFile:CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/grassFrac/gn/v20190624/grassFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_195001-201412.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf3.dkrz.de'] 2023-12-27 23:11:44,667 UTC [919785] DEBUG esmvalcore.esgf._search:364 Selected files: ESGFFile:CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/grassFrac/gn/v20190624/grassFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_195001-201412.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf3.dkrz.de'] 2023-12-27 23:11:44,667 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc 2023-12-27 23:11:44,668 UTC [919785] DEBUG esmvalcore._recipe.recipe:263 Using input files for variable grassFrac of dataset HadGEM3-GC31-LL: /scratch/tomast/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/grassFrac/gn/v20190624/grassFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_195001-201412.nc (will be downloaded) /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/v20190709/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn.nc 2023-12-27 23:11:44,668 UTC [919785] INFO esmvalcore._recipe.recipe:520 Found input files for Dataset: grassFrac, Lmon, CMIP6, HadGEM3-GC31-LL, CMIP, historical, r1i1p1f3, gn, v20190624, supplementaries: sftlf, fx, piControl, r1i1p1f1, v20190709 2023-12-27 23:11:44,669 UTC [919785] INFO esmvalcore._recipe.recipe:677 PreprocessingTask albedolandcover_cmip6/grassFrac created. 2023-12-27 23:11:44,669 UTC [919785] DEBUG esmvalcore._recipe.recipe:678 PreprocessingTask albedolandcover_cmip6/grassFrac will create the files: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_albedolandcover_20231227_230944/preproc/albedolandcover_cmip6/grassFrac/CMIP6_HadGEM3-GC31-LL_Lmon_historical_r1i1p1f3_grassFrac_gn_2000-2004.nc 2023-12-27 23:11:44,669 UTC [919785] INFO esmvalcore._recipe.recipe:983 Creating preprocessor task albedolandcover_cmip6/shrubFrac 2023-12-27 23:11:44,669 UTC [919785] INFO esmvalcore._recipe.recipe:653 Creating preprocessor 'pp_cmip' task for variable 'shrubFrac' 2023-12-27 23:11:44,671 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/shrubFrac/gn/*/shrubFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9034K/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/shrubFrac/gn/*/shrubFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/shrubFrac/gn/*/shrubFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/shrubFrac/gn/*/shrubFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/shrubFrac/gn/*/shrubFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/shrubFrac/gn/*/shrubFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/shrubFrac/gn/*/shrubFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/NERC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/shrubFrac/gn/*/shrubFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn*.nc 2023-12-27 23:11:44,671 UTC [919785] DEBUG esmvalcore.esgf._search:116 Searching https://esgf-node.llnl.gov/esg-search for datasets using facets={'project': 'CMIP6', 'activity_drs': 'CMIP', 'source_id': 'HadGEM3-GC31-LL', 'member_id': 'r1i1p1f3', 'experiment_id': 'historical', 'institution_id': 'MOHC,NERC', 'grid_label': 'gn', 'table_id': 'Lmon', 'variable': 'shrubFrac'} 2023-12-27 23:11:44,671 UTC [919785] DEBUG requests_cache.backends:63 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2023-12-27 23:11:44,671 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2023-12-27 23:11:44,671 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2023-12-27 23:11:44,672 UTC [919785] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: None 2023-12-27 23:11:44,672 UTC [919785] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2023-12-27 23:11:44,673 UTC [919785] DEBUG requests_cache.policy.actions:91 Cache directives from request headers: CacheDirectives() 2023-12-27 23:11:44,673 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-read cache checks: Passed 2023-12-27 23:11:44,674 UTC [919785] DEBUG requests_cache.policy.actions:205 Post-read cache actions: CacheActions(expire_after=86400, send_request=True) 2023-12-27 23:11:44,674 UTC [919785] DEBUG urllib3.connectionpool:1052 Starting new HTTPS connection (1): esgf-node.llnl.gov:443 2023-12-27 23:11:45,384 UTC [919785] DEBUG urllib3.connectionpool:546 https://esgf-node.llnl.gov:443 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=CMIP6&activity_drs=CMIP&source_id=HadGEM3-GC31-LL&member_id=r1i1p1f3&experiment_id=historical&institution_id=MOHC%2CNERC&grid_label=gn&table_id=Lmon&variable=shrubFrac HTTP/1.1" 200 None 2023-12-27 23:11:45,553 UTC [919785] DEBUG requests_cache.policy.actions:344 Pre-write cache checks: Passed 2023-12-27 23:11:45,658 UTC [919785] DEBUG requests_cache.backends.base:106 Closing backend connections 2023-12-27 23:11:45,659 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'version' from 'None' to 'v20190624' for CMIP6.CMIP.MOHC.HadGEM3-GC31-LL.historical.r1i1p1f3.Lmon.shrubFrac.gn.v20190624.shrubFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_185001-194912.nc 2023-12-27 23:11:45,659 UTC [919785] DEBUG esmvalcore.esgf._download:272 Correcting facet 'version' from 'None' to 'v20190624' for CMIP6.CMIP.MOHC.HadGEM3-GC31-LL.historical.r1i1p1f3.Lmon.shrubFrac.gn.v20190624.shrubFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_195001-201412.nc 2023-12-27 23:11:45,659 UTC [919785] DEBUG esmvalcore.esgf._search:154 Found the following files matching facets {'project': 'CMIP6', 'activity_drs': 'CMIP', 'source_id': 'HadGEM3-GC31-LL', 'member_id': 'r1i1p1f3', 'experiment_id': 'historical', 'institution_id': 'MOHC,NERC', 'grid_label': 'gn', 'table_id': 'Lmon', 'variable': 'shrubFrac'}: ESGFFile:CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/shrubFrac/gn/v20190624/shrubFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_185001-194912.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf3.dkrz.de'] ESGFFile:CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/shrubFrac/gn/v20190624/shrubFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_195001-201412.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf3.dkrz.de'] 2023-12-27 23:11:45,659 UTC [919785] DEBUG esmvalcore.esgf._search:364 Selected files: ESGFFile:CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/shrubFrac/gn/v20190624/shrubFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_195001-201412.nc on hosts ['aims3.llnl.gov', 'esgf.ceda.ac.uk', 'esgf3.dkrz.de'] 2023-12-27 23:11:45,659 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /projects/NS9252K/KeyCLIM_CMOR/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc /scratch/tomast/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/*/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn*.nc 2023-12-27 23:11:45,661 UTC [919785] DEBUG esmvalcore._recipe.recipe:263 Using input files for variable shrubFrac of dataset HadGEM3-GC31-LL: /scratch/tomast/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/historical/r1i1p1f3/Lmon/shrubFrac/gn/v20190624/shrubFrac_Lmon_HadGEM3-GC31-LL_historical_r1i1p1f3_gn_195001-201412.nc (will be downloaded) /projects/NS9560K-datalake/ESGF/CMIP6/CMIP/MOHC/HadGEM3-GC31-LL/piControl/r1i1p1f1/fx/sftlf/gn/v20190709/sftlf_fx_HadGEM3-GC31-LL_piControl_r1i1p1f1_gn.nc 2023-12-27 23:11:45,661 UTC [919785] INFO esmvalcore._recipe.recipe:520 Found input files for Dataset: shrubFrac, Lmon, CMIP6, HadGEM3-GC31-LL, CMIP, historical, r1i1p1f3, gn, v20190624, supplementaries: sftlf, fx, piControl, r1i1p1f1, v20190709 2023-12-27 23:11:45,661 UTC [919785] INFO esmvalcore._recipe.recipe:677 PreprocessingTask albedolandcover_cmip6/shrubFrac created. 2023-12-27 23:11:45,662 UTC [919785] DEBUG esmvalcore._recipe.recipe:678 PreprocessingTask albedolandcover_cmip6/shrubFrac will create the files: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_albedolandcover_20231227_230944/preproc/albedolandcover_cmip6/shrubFrac/CMIP6_HadGEM3-GC31-LL_Lmon_historical_r1i1p1f3_shrubFrac_gn_2000-2004.nc 2023-12-27 23:11:45,662 UTC [919785] INFO esmvalcore._recipe.recipe:1008 Creating tasks for diagnostic albedolandcover_obs 2023-12-27 23:11:45,662 UTC [919785] INFO esmvalcore._recipe.recipe:930 Creating diagnostic task albedolandcover_obs/albedolandcover 2023-12-27 23:11:45,662 UTC [919785] INFO esmvalcore._recipe.recipe:983 Creating preprocessor task albedolandcover_obs/albDiffiTr13 2023-12-27 23:11:45,662 UTC [919785] INFO esmvalcore._recipe.recipe:653 Creating preprocessor 'pp_obs' task for variable 'albDiffiTr13' 2023-12-27 23:11:45,662 UTC [919785] WARNING esmvalcore._recipe.check:206 Preprocessor function mask_landsea works best when at least one supplementary variable of ['sftlf', 'sftof'] is defined in the recipe for Dataset: {'diagnostic': 'albedolandcover_obs', 'variable_group': 'albDiffiTr13', 'dataset': 'Duveiller2018', 'project': 'OBS', 'mip': 'Amon', 'short_name': 'albDiffiTr13', 'alias': 'Duveiller2018', 'frequency': 'mon', 'long_name': 'Difference in Surface Albedo for Vegetation Cover Transition ' 'from Forest to Crops and Grasses', 'modeling_realm': ['atmos'], 'original_short_name': 'albDiffiTr13', 'preprocessor': 'pp_obs', 'recipe_dataset_index': 0, 'standard_name': '', 'tier': 2, 'timerange': '2010/2010', 'type': 'clim', 'units': '1', 'version': 'v2018'} session: 'recipe_albedolandcover_20231227_230944'. 2023-12-27 23:11:45,663 UTC [919785] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9560K-datalake/ESGF/obsdata/Tier2/Duveiller2018/OBS_Duveiller2018_clim_v2018_Amon_albDiffiTr13[_.]*nc /scratch/tomast/ESGF/obsdata/Tier2/Duveiller2018/OBS_Duveiller2018_clim_v2018_Amon_albDiffiTr13[_.]*nc 2023-12-27 23:11:45,663 UTC [919785] ERROR esmvalcore._recipe.check:120 No input files found for Dataset: {'diagnostic': 'albedolandcover_obs', 'variable_group': 'albDiffiTr13', 'dataset': 'Duveiller2018', 'project': 'OBS', 'mip': 'Amon', 'short_name': 'albDiffiTr13', 'alias': 'Duveiller2018', 'frequency': 'mon', 'long_name': 'Difference in Surface Albedo for Vegetation Cover Transition ' 'from Forest to Crops and Grasses', 'modeling_realm': ['atmos'], 'original_short_name': 'albDiffiTr13', 'preprocessor': 'pp_obs', 'recipe_dataset_index': 0, 'standard_name': '', 'tier': 2, 'timerange': '2010/2010', 'type': 'clim', 'units': '1', 'version': 'v2018'} session: 'recipe_albedolandcover_20231227_230944' 2023-12-27 23:11:45,664 UTC [919785] ERROR esmvalcore._recipe.check:126 Looked for files matching /projects/NS9560K-datalake/ESGF/obsdata/Tier2/Duveiller2018/OBS_Duveiller2018_clim_v2018_Amon_albDiffiTr13[_.]*nc /scratch/tomast/ESGF/obsdata/Tier2/Duveiller2018/OBS_Duveiller2018_clim_v2018_Amon_albDiffiTr13[_.]*nc 2023-12-27 23:11:45,664 UTC [919785] ERROR esmvalcore._recipe.check:127 Set 'log_level' to 'debug' to get more information 2023-12-27 23:11:45,664 UTC [919785] ERROR esmvalcore._recipe.recipe:727 Could not create all tasks 2023-12-27 23:11:45,664 UTC [919785] ERROR esmvalcore._recipe.recipe:729 Missing data for preprocessor albedolandcover_obs/albDiffiTr13: - Missing data for Dataset: albDiffiTr13, Amon, OBS, Duveiller2018, v2018 2023-12-27 23:11:46,265 UTC [919785] INFO esmvalcore._task:128 Maximum memory used (estimate): 0.3 GB 2023-12-27 23:11:46,266 UTC [919785] INFO esmvalcore._task:130 Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. 2023-12-27 23:11:46,266 UTC [919785] ERROR esmvalcore._main:523 Could not create all tasks 2023-12-27 23:11:46,267 UTC [919785] DEBUG esmvalcore._main:524 Stack trace for debugging: Traceback (most recent call last): File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/_main.py", line 518, in run fire.Fire(ESMValTool()) File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/fire/core.py", line 141, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/fire/core.py", line 475, in _Fire component, remaining_args = _CallAndUpdateTrace( ^^^^^^^^^^^^^^^^^^^^ File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^ File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/_main.py", line 405, in run self._run(recipe, session) File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/_main.py", line 447, in _run process_recipe(recipe_file=recipe, session=session) File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/_main.py", line 127, in process_recipe recipe = read_recipe_file(recipe_file, session) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/_recipe/recipe.py", line 73, in read_recipe_file return Recipe(raw_recipe, session, recipe_file=filename) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/_recipe/recipe.py", line 720, in __init__ self.tasks = self.initialize_tasks() ^^^^^^^^^^^^^^^^^^^^^^^ File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/_recipe/recipe.py", line 1046, in initialize_tasks tasks = self._create_tasks() ^^^^^^^^^^^^^^^^^^^^ File "/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/_recipe/recipe.py", line 1034, in _create_tasks raise recipe_error esmvalcore.exceptions.RecipeError: Could not create all tasks