2023-12-27 23:41:53,928 UTC [929581] INFO esmvalcore._main:492 ______________________________________________________________________ _____ ____ __ ____ __ _ _____ _ | ____/ ___|| \/ \ \ / /_ _| |_ _|__ ___ | | | _| \___ \| |\/| |\ \ / / _` | | | |/ _ \ / _ \| | | |___ ___) | | | | \ V / (_| | | | | (_) | (_) | | |_____|____/|_| |_| \_/ \__,_|_| |_|\___/ \___/|_| ______________________________________________________________________ ESMValTool - Earth System Model Evaluation Tool. http://www.esmvaltool.org CORE DEVELOPMENT TEAM AND CONTACTS: Birgit Hassler (Co-PI; DLR, Germany - birgit.hassler@dlr.de) Alistair Sellar (Co-PI; Met Office, UK - alistair.sellar@metoffice.gov.uk) Bouwe Andela (Netherlands eScience Center, The Netherlands - b.andela@esciencecenter.nl) Lee de Mora (PML, UK - ledm@pml.ac.uk) Niels Drost (Netherlands eScience Center, The Netherlands - n.drost@esciencecenter.nl) Veronika Eyring (DLR, Germany - veronika.eyring@dlr.de) Bettina Gier (UBremen, Germany - gier@uni-bremen.de) Remi Kazeroni (DLR, Germany - remi.kazeroni@dlr.de) Nikolay Koldunov (AWI, Germany - nikolay.koldunov@awi.de) Axel Lauer (DLR, Germany - axel.lauer@dlr.de) Saskia Loosveldt-Tomas (BSC, Spain - saskia.loosveldt@bsc.es) Ruth Lorenz (ETH Zurich, Switzerland - ruth.lorenz@env.ethz.ch) Benjamin Mueller (LMU, Germany - b.mueller@iggf.geo.uni-muenchen.de) Valeriu Predoi (URead, UK - valeriu.predoi@ncas.ac.uk) Mattia Righi (DLR, Germany - mattia.righi@dlr.de) Manuel Schlund (DLR, Germany - manuel.schlund@dlr.de) Breixo Solino Fernandez (DLR, Germany - breixo.solinofernandez@dlr.de) Javier Vegas-Regidor (BSC, Spain - javier.vegas@bsc.es) Klaus Zimmermann (SMHI, Sweden - klaus.zimmermann@smhi.se) For further help, please read the documentation at http://docs.esmvaltool.org. Have fun! 2023-12-27 23:41:53,928 UTC [929581] INFO esmvalcore._main:493 Package versions 2023-12-27 23:41:53,928 UTC [929581] INFO esmvalcore._main:494 ---------------- 2023-12-27 23:41:53,928 UTC [929581] INFO esmvalcore._main:495 ESMValCore: 2.10.0 2023-12-27 23:41:53,928 UTC [929581] INFO esmvalcore._main:497 ESMValTool: 2.10.0 2023-12-27 23:41:53,928 UTC [929581] INFO esmvalcore._main:498 ---------------- 2023-12-27 23:41:53,929 UTC [929581] INFO esmvalcore._main:499 Using config file /nird/home/tomast/.esmvaltool/config-user.yml 2023-12-27 23:41:53,929 UTC [929581] INFO esmvalcore._main:500 Writing program log files to: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/run/main_log.txt /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/run/main_log_debug.txt 2023-12-27 23:41:54,109 UTC [929581] INFO esmvalcore.config._esgf_pyclient:116 Using default ESGF configuration, configuration file /nird/home/tomast/.esmvaltool/esgf-pyclient.yml not present. 2023-12-27 23:41:54,109 UTC [929581] WARNING esmvalcore.config._esgf_pyclient:181 ESGF credentials missing, only data that is accessible without logging in will be available. See https://esgf.github.io/esgf-user-support/user_guide.html for instructions on how to create an account if you do not have one yet. Next, configure your system so esmvaltool can use your credentials. This can be done using the keyring package, or you can just enter them in /nird/home/tomast/.esmvaltool/esgf-pyclient.yml. keyring ======= First install the keyring package (requires a supported backend, see https://pypi.org/project/keyring/): $ pip install keyring Next, set your username and password by running the commands: $ keyring set ESGF hostname $ keyring set ESGF username $ keyring set ESGF password To check that you entered your credentials correctly, run: $ keyring get ESGF hostname $ keyring get ESGF username $ keyring get ESGF password configuration file ================== You can store the hostname, username, and password or your OpenID account in a plain text in the file /nird/home/tomast/.esmvaltool/esgf-pyclient.yml like this: logon: hostname: "your-hostname" username: "your-username" password: "your-password" or your can configure an interactive log in: logon: interactive: true Note that storing your password in plain text in the configuration file is less secure. On shared systems, make sure the permissions of the file are set so only you can read it, i.e. $ ls -l /nird/home/tomast/.esmvaltool/esgf-pyclient.yml shows permissions -rw-------. 2023-12-27 23:41:54,704 UTC [929581] INFO esmvalcore._main:91 Starting the Earth System Model Evaluation Tool at time: 2023-12-27 23:41:54 UTC 2023-12-27 23:41:54,705 UTC [929581] INFO esmvalcore._main:95 ---------------------------------------------------------------------- 2023-12-27 23:41:54,705 UTC [929581] INFO esmvalcore._main:96 RECIPE = /projects/NS2980K/home/tomast/ESMValTool/recipes-2.10.0/standard_recipes/atmosphere/recipe_quantilebias.yml 2023-12-27 23:41:54,705 UTC [929581] INFO esmvalcore._main:97 RUNDIR = /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/run 2023-12-27 23:41:54,705 UTC [929581] INFO esmvalcore._main:98 WORKDIR = /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/work 2023-12-27 23:41:54,705 UTC [929581] INFO esmvalcore._main:99 PREPROCDIR = /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/preproc 2023-12-27 23:41:54,705 UTC [929581] INFO esmvalcore._main:100 PLOTDIR = /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/plots 2023-12-27 23:41:54,705 UTC [929581] INFO esmvalcore._main:101 ---------------------------------------------------------------------- 2023-12-27 23:41:54,705 UTC [929581] INFO esmvalcore._main:104 Running tasks using at most 256 processes 2023-12-27 23:41:54,705 UTC [929581] INFO esmvalcore._main:106 If your system hangs during execution, it may not have enough memory for keeping this number of tasks in memory. 2023-12-27 23:41:54,705 UTC [929581] INFO esmvalcore._main:109 If you experience memory problems, try reducing 'max_parallel_tasks' in your user configuration file. 2023-12-27 23:41:54,706 UTC [929581] WARNING esmvalcore.config._dask:18 Using the Dask basic scheduler. This may lead to slow computations and out-of-memory errors. Note that the basic scheduler may still be the best choice for preprocessor functions that are not lazy. In that case, you can safely ignore this warning. See https://docs.esmvaltool.org/projects/ESMValCore/en/latest/quickstart/configure.html#dask-distributed-configuration for more information. 2023-12-27 23:41:54,706 UTC [929581] DEBUG esmvalcore._recipe.check:54 Checking recipe against schema /projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/_recipe/recipe_schema.yml 2023-12-27 23:41:54,716 UTC [929581] DEBUG esmvalcore._recipe.to_datasets:343 Populating list of datasets for variable pr in diagnostic quantilebias 2023-12-27 23:41:54,717 UTC [929581] DEBUG esmvalcore.config._config:45 Loading extra facets from /projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config/extra_facets/cmip5-fx.yml 2023-12-27 23:41:54,717 UTC [929581] DEBUG esmvalcore.config._config:45 Loading extra facets from /projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config/extra_facets/cmip5-institutes.yml 2023-12-27 23:41:54,732 UTC [929581] DEBUG esmvalcore.config._config:45 Loading extra facets from /projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config/extra_facets/cmip5-product.yml 2023-12-27 23:41:54,735 UTC [929581] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/*/CanESM2/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_CanESM2_*_**.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/*/CanESM2/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_CanESM2_*_**.nc /scratch/tomast/ESGF/cmip5/output1/*/CanESM2/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_CanESM2_*_**.nc 2023-12-27 23:41:54,738 UTC [929581] DEBUG esmvalcore._recipe.to_datasets:367 Found Dataset: pr, Amon, CMIP5, CanESM2, historical, r1i1p1, supplementaries: sftlf, fx, r0i0p0 2023-12-27 23:41:54,738 UTC [929581] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9560K-datalake/ESGF/obs4MIPs/Tier1/GPCP-V2.3/sftlf_*.nc /projects/NS9560K-datalake/ESGF/obsdata/Tier1/GPCP-V2.3/sftlf_*.nc /scratch/tomast/ESGF/obs4MIPs/Tier1/GPCP-V2.3/sftlf_*.nc 2023-12-27 23:41:54,739 UTC [929581] DEBUG esmvalcore.esgf._search:116 Searching https://esgf.ceda.ac.uk/esg-search for datasets using facets={'project': 'obs4MIPs', 'source_id': 'GPCP-V2.3', 'time_frequency': 'mon', 'variable': 'sftlf'} 2023-12-27 23:41:54,739 UTC [929581] DEBUG requests_cache.backends:63 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2023-12-27 23:41:54,739 UTC [929581] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2023-12-27 23:41:54,739 UTC [929581] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2023-12-27 23:41:54,826 UTC [929581] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: None 2023-12-27 23:41:54,826 UTC [929581] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2023-12-27 23:41:54,828 UTC [929581] DEBUG requests_cache.policy.actions:91 Cache directives from request headers: CacheDirectives() 2023-12-27 23:41:54,828 UTC [929581] DEBUG requests_cache.policy.actions:344 Pre-read cache checks: Passed 2023-12-27 23:41:54,829 UTC [929581] DEBUG requests_cache.policy.actions:205 Post-read cache actions: CacheActions(expire_after=86400, send_request=True) 2023-12-27 23:41:54,829 UTC [929581] DEBUG urllib3.connectionpool:1052 Starting new HTTPS connection (1): esgf.ceda.ac.uk:443 2023-12-27 23:41:55,077 UTC [929581] DEBUG urllib3.connectionpool:546 https://esgf.ceda.ac.uk:443 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=obs4MIPs&source_id=GPCP-V2.3&time_frequency=mon&variable=sftlf HTTP/1.1" 200 1055 2023-12-27 23:41:55,078 UTC [929581] DEBUG requests_cache.policy.actions:344 Pre-write cache checks: Passed 2023-12-27 23:41:55,189 UTC [929581] DEBUG requests_cache.backends.base:106 Closing backend connections 2023-12-27 23:41:55,192 UTC [929581] DEBUG esmvalcore.esgf._search:154 Found the following files matching facets {'project': 'obs4MIPs', 'source_id': 'GPCP-V2.3', 'time_frequency': 'mon', 'variable': 'sftlf'}: none 2023-12-27 23:41:55,192 UTC [929581] DEBUG esmvalcore.esgf._search:364 Selected files: 2023-12-27 23:41:55,193 UTC [929581] INFO esmvalcore.dataset:315 For Dataset: pr, Amon, obs4MIPs, GPCP-V2.3, supplementaries: sftlf: ignoring supplementary variable 'sftlf', unable to expand wildcards 'institute'. 2023-12-27 23:41:55,193 UTC [929581] DEBUG esmvalcore._recipe.to_datasets:367 Found Dataset: pr, Amon, obs4MIPs, GPCP-V2.3 2023-12-27 23:41:55,193 UTC [929581] DEBUG esmvalcore._recipe.recipe:781 Retrieving diagnostics from recipe 2023-12-27 23:41:55,194 UTC [929581] DEBUG esmvalcore._recipe.recipe:809 Setting script for diagnostic quantilebias 2023-12-27 23:41:55,194 UTC [929581] INFO esmvalcore._recipe.recipe:999 Creating tasks from recipe 2023-12-27 23:41:55,194 UTC [929581] INFO esmvalcore._recipe.recipe:1008 Creating tasks for diagnostic quantilebias 2023-12-27 23:41:55,194 UTC [929581] INFO esmvalcore._recipe.recipe:930 Creating diagnostic task quantilebias/main 2023-12-27 23:41:55,405 UTC [929581] INFO esmvalcore._recipe.recipe:983 Creating preprocessor task quantilebias/pr 2023-12-27 23:41:55,405 UTC [929581] INFO esmvalcore._recipe.recipe:653 Creating preprocessor 'mask_regrid_preproc' task for variable 'pr' 2023-12-27 23:41:55,406 UTC [929581] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/CCCMA/CanESM2/historical/mon/atmos/Amon/r1i1p1/*/pr/pr_Amon_CanESM2_historical_r1i1p1*.nc /projects/NS9034K/CMIP5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/*/pr/pr_Amon_CanESM2_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCMA/CanESM2/historical/mon/atmos/Amon/r1i1p1/*/pr/pr_Amon_CanESM2_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/*/pr/pr_Amon_CanESM2_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/CCCMA/CanESM2/historical/mon/atmos/Amon/r1i1p1/*/pr/pr_Amon_CanESM2_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/*/pr/pr_Amon_CanESM2_historical_r1i1p1*.nc 2023-12-27 23:41:55,864 UTC [929581] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9034K/CMIP5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_CanESM2_historical_r0i0p0*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_CanESM2_historical_r0i0p0*.nc /scratch/tomast/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_CanESM2_historical_r0i0p0*.nc 2023-12-27 23:41:55,865 UTC [929581] DEBUG esmvalcore._recipe.recipe:263 Using input files for variable pr of dataset CMIP5: /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc 2023-12-27 23:41:55,865 UTC [929581] INFO esmvalcore._recipe.recipe:520 Found input files for Dataset: pr, Amon, CMIP5, CanESM2, historical, r1i1p1, v20120718, supplementaries: sftlf, fx, r0i0p0, v20120410 2023-12-27 23:41:55,866 UTC [929581] WARNING esmvalcore._recipe.check:206 Preprocessor function mask_landsea works best when at least one supplementary variable of ['sftlf', 'sftof'] is defined in the recipe for Dataset: {'diagnostic': 'quantilebias', 'variable_group': 'pr', 'dataset': 'GPCP-V2.3', 'project': 'obs4MIPs', 'mip': 'Amon', 'short_name': 'pr', 'alias': 'obs4MIPs', 'frequency': 'mon', 'level': 'L3', 'long_name': 'Precipitation', 'modeling_realm': [], 'original_short_name': 'pr', 'preprocessor': 'mask_regrid_preproc', 'recipe_dataset_index': 1, 'reference_dataset': 'GPCP-V2.3', 'standard_name': 'precipitation_flux', 'tier': 1, 'timerange': '1979/2005', 'units': 'kg m-2 s-1'} session: 'recipe_quantilebias_20231227_234153'. 2023-12-27 23:41:55,867 UTC [929581] DEBUG esmvalcore.local:444 Looking for files matching: /projects/NS9560K-datalake/ESGF/obs4MIPs/Tier1/GPCP-V2.3/pr_*.nc /projects/NS9560K-datalake/ESGF/obsdata/Tier1/GPCP-V2.3/pr_*.nc /scratch/tomast/ESGF/obs4MIPs/Tier1/GPCP-V2.3/pr_*.nc 2023-12-27 23:41:55,867 UTC [929581] DEBUG esmvalcore.esgf._search:116 Searching https://esgf.ceda.ac.uk/esg-search for datasets using facets={'project': 'obs4MIPs', 'source_id': 'GPCP-V2.3', 'time_frequency': 'mon', 'variable': 'pr'} 2023-12-27 23:41:55,867 UTC [929581] DEBUG requests_cache.backends:63 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2023-12-27 23:41:55,867 UTC [929581] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2023-12-27 23:41:55,868 UTC [929581] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2023-12-27 23:41:55,868 UTC [929581] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: None 2023-12-27 23:41:55,868 UTC [929581] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2023-12-27 23:41:55,871 UTC [929581] DEBUG requests_cache.policy.actions:91 Cache directives from request headers: CacheDirectives() 2023-12-27 23:41:55,871 UTC [929581] DEBUG requests_cache.policy.actions:344 Pre-read cache checks: Passed 2023-12-27 23:41:55,872 UTC [929581] DEBUG requests_cache.policy.actions:205 Post-read cache actions: CacheActions(expire_after=86400, send_request=True) 2023-12-27 23:41:55,872 UTC [929581] DEBUG urllib3.connectionpool:1052 Starting new HTTPS connection (1): esgf.ceda.ac.uk:443 2023-12-27 23:43:42,582 UTC [929581] DEBUG esmvalcore.esgf._search:129 Unable to connect to https://esgf.ceda.ac.uk/esg-search due to HTTPSConnectionPool(host='esgf.ceda.ac.uk', port=443): Max retries exceeded with url: /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=obs4MIPs&source_id=GPCP-V2.3&time_frequency=mon&variable=pr (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1006)'))) 2023-12-27 23:43:42,582 UTC [929581] DEBUG esmvalcore.esgf._search:116 Searching https://esgf-node.llnl.gov/esg-search for datasets using facets={'project': 'obs4MIPs', 'source_id': 'GPCP-V2.3', 'time_frequency': 'mon', 'variable': 'pr'} 2023-12-27 23:43:42,582 UTC [929581] DEBUG requests_cache.backends:63 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2023-12-27 23:43:42,582 UTC [929581] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2023-12-27 23:43:42,582 UTC [929581] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2023-12-27 23:43:42,583 UTC [929581] DEBUG requests_cache.backends.base:384 Initialized SQLiteDict with serializer: None 2023-12-27 23:43:42,583 UTC [929581] DEBUG requests_cache.backends.sqlite:234 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2023-12-27 23:43:42,585 UTC [929581] DEBUG requests_cache.policy.actions:91 Cache directives from request headers: CacheDirectives() 2023-12-27 23:43:42,585 UTC [929581] DEBUG requests_cache.policy.actions:344 Pre-read cache checks: Passed 2023-12-27 23:43:42,585 UTC [929581] DEBUG requests_cache.policy.actions:205 Post-read cache actions: CacheActions(expire_after=86400, send_request=True) 2023-12-27 23:43:42,586 UTC [929581] DEBUG urllib3.connectionpool:1052 Starting new HTTPS connection (1): esgf-node.llnl.gov:443 2023-12-27 23:43:43,712 UTC [929581] DEBUG urllib3.connectionpool:546 https://esgf-node.llnl.gov:443 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=obs4MIPs&source_id=GPCP-V2.3&time_frequency=mon&variable=pr HTTP/1.1" 200 2602 2023-12-27 23:43:43,714 UTC [929581] DEBUG requests_cache.policy.actions:344 Pre-write cache checks: Passed 2023-12-27 23:43:47,542 UTC [929581] DEBUG requests_cache.backends.base:106 Closing backend connections 2023-12-27 23:43:47,542 UTC [929581] DEBUG esmvalcore.esgf._download:323 Wrong dataset_id_template_ %(project)s.%(institute)s.%(source_id)s.%(realm)s.%(time_frequency)s or facet values containing '.' for dataset obs4MIPs.NASA-GSFC.GPCP-V2.3.atmos.mon.v20180519 2023-12-27 23:43:47,543 UTC [929581] DEBUG esmvalcore.esgf._download:272 Correcting facet 'version' from 'None' to 'v20180519' for obs4MIPs.GPCP-V2.3.v20180519.pr_GPCP-SG_L3_v2.3_197901-201710.nc 2023-12-27 23:43:47,543 UTC [929581] DEBUG esmvalcore.esgf._search:154 Found the following files matching facets {'project': 'obs4MIPs', 'source_id': 'GPCP-V2.3', 'time_frequency': 'mon', 'variable': 'pr'}: ESGFFile:obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc on hosts ['dpesgf03.nccs.nasa.gov'] 2023-12-27 23:43:47,543 UTC [929581] DEBUG esmvalcore.esgf._search:364 Selected files: ESGFFile:obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc on hosts ['dpesgf03.nccs.nasa.gov'] 2023-12-27 23:43:47,543 UTC [929581] DEBUG esmvalcore._recipe.recipe:263 Using input files for variable pr of dataset obs4MIPs: /scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc (will be downloaded) 2023-12-27 23:43:47,543 UTC [929581] INFO esmvalcore._recipe.recipe:520 Found input files for Dataset: pr, Amon, obs4MIPs, GPCP-V2.3, v20180519 2023-12-27 23:43:47,544 UTC [929581] INFO esmvalcore._recipe.recipe:677 PreprocessingTask quantilebias/pr created. 2023-12-27 23:43:47,544 UTC [929581] DEBUG esmvalcore._recipe.recipe:678 PreprocessingTask quantilebias/pr will create the files: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005.nc /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005.nc 2023-12-27 23:43:47,544 UTC [929581] DEBUG esmvalcore._recipe.recipe:858 Linking tasks for diagnostic quantilebias script main 2023-12-27 23:43:47,544 UTC [929581] DEBUG esmvalcore._recipe.recipe:867 Pattern quantilebias/pr matches ['quantilebias/pr'] 2023-12-27 23:43:47,544 UTC [929581] INFO esmvalcore._recipe.recipe:1048 These tasks will be executed: quantilebias/pr, quantilebias/main 2023-12-27 23:43:47,545 UTC [929581] DEBUG esmvalcore._main:128 Recipe summary: DiagnosticTask: quantilebias/main script: quantilebias/quantilebias.R settings: {'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'log_level': 'info', 'output_file_type': 'png', 'perc_lev': 75, 'plot_dir': '/projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/plots/quantilebias/main', 'profile_diagnostic': False, 'recipe': PosixPath('recipe_quantilebias.yml'), 'run_dir': '/projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/run/quantilebias/main', 'script': 'main', 'version': '2.10.0', 'work_dir': '/projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/work/quantilebias/main'} ancestors: PreprocessingTask: quantilebias/pr order: ['mask_landsea', 'regrid', 'remove_supplementary_variables', 'save'] PreprocessorFile: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005.nc input files: [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] settings: {'mask_landsea': {'mask_out': 'sea'}, 'regrid': {'scheme': 'area_weighted', 'target_grid': '2x2'}, 'remove_supplementary_variables': {}, 'save': {'compress': False, 'filename': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005.nc')}} PreprocessorFile: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005.nc input files: [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc'), LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] settings: {'mask_landsea': {'mask_out': 'sea'}, 'regrid': {'scheme': 'area_weighted', 'target_grid': '2x2'}, 'remove_supplementary_variables': {}, 'save': {'compress': False, 'filename': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005.nc')}} ancestors: None 2023-12-27 23:43:47,549 UTC [929581] INFO esmvalcore._recipe.recipe:1111 Wrote recipe with version numbers and wildcards to: file:///projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/run/recipe_quantilebias_filled.yml 2023-12-27 23:43:47,549 UTC [929581] INFO esmvalcore.esgf._download:540 Will download 19.35 MB Will download the following files: 19.35 MB ESGFFile:obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc on hosts ['dpesgf03.nccs.nasa.gov'] Downloading 19.35 MB.. 2023-12-27 23:43:47,777 UTC [929581] DEBUG esmvalcore.esgf._download:461 Downloading https://dpesgf03.nccs.nasa.gov/thredds/fileServer/obs4MIPs/observations/NASA-GSFC/Obs-GPCP/GPCP/V2.3/atmos/pr/pr_GPCP-SG_L3_v2.3_197901-201710.nc to /scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc.c_tqkjz1 2023-12-27 23:43:47,778 UTC [929581] DEBUG urllib3.connectionpool:1052 Starting new HTTPS connection (1): dpesgf03.nccs.nasa.gov:443 2023-12-27 23:43:48,915 UTC [929581] DEBUG urllib3.connectionpool:546 https://dpesgf03.nccs.nasa.gov:443 "GET /thredds/fileServer/obs4MIPs/observations/NASA-GSFC/Obs-GPCP/GPCP/V2.3/atmos/pr/pr_GPCP-SG_L3_v2.3_197901-201710.nc HTTP/1.1" 200 19348352 2023-12-27 23:43:50,494 UTC [929581] INFO esmvalcore.esgf._download:490 Downloaded /scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc (19.35 MB) in 2.7 seconds (7.16 MB/s) from dpesgf03.nccs.nasa.gov 2023-12-27 23:43:50,496 UTC [929581] INFO esmvalcore.esgf._download:569 Downloaded 19.35 MB in 2.95 seconds (6.57 MB/s) 2023-12-27 23:43:50,496 UTC [929581] INFO esmvalcore.esgf._download:581 Successfully downloaded all requested files. 2023-12-27 23:43:50,496 UTC [929581] INFO esmvalcore.config._dask:70 Using the Dask basic scheduler. 2023-12-27 23:43:50,496 UTC [929581] INFO esmvalcore._task:760 Running 2 tasks using 2 processes 2023-12-27 23:43:50,521 UTC [930142] INFO esmvalcore._task:261 Starting task quantilebias/pr in process [930142] 2023-12-27 23:43:50,618 UTC [929581] INFO esmvalcore._task:793 Progress: 1 tasks running, 1 tasks waiting for ancestors, 0/2 done 2023-12-27 23:43:51,109 UTC [930142] DEBUG esmvalcore.preprocessor:651 Running block ['mask_landsea', 'regrid'] 2023-12-27 23:43:51,109 UTC [930142] DEBUG esmvalcore.preprocessor:658 Applying single-model steps to PreprocessorFile: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005.nc 2023-12-27 23:43:51,110 UTC [930142] DEBUG esmvalcore.esgf._download:535 All required data is available locally, not downloading anything. 2023-12-27 23:43:51,110 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step fix_file 2023-12-27 23:43:51,111 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'fix_file' on the data LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc') loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) output_dir = PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/preproc/fixed_files/pr_Amon_CMIP5_CanESM2_historical_r1i1p1_v20120718_'), add_unique_suffix = True, session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': None, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': True, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': False, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), preprocessor = 'mask_regrid_preproc', reference_dataset = 'GPCP-V2.3', mip = 'Amon', dataset = 'CanESM2', project = 'CMIP5', exp = 'historical', ensemble = 'r1i1p1', short_name = 'pr', timerange = '1979/2005', variable_group = 'pr', diagnostic = 'quantilebias', recipe_dataset_index = 0, alias = 'CMIP5', product = ['output1', 'output2'], institute = ['CCCma', 'CCCMA'], original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = ['atmos'], frequency = 'mon', version = 'v20120718' 2023-12-27 23:43:51,113 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step load 2023-12-27 23:43:51,113 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'load' on the data LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc') loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) ignore_warnings = None 2023-12-27 23:43:51,113 UTC [930142] DEBUG esmvalcore.preprocessor._io:145 Loading: /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc 2023-12-27 23:43:53,736 UTC [930142] DEBUG esmvalcore.preprocessor._io:176 Done with loading /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc 2023-12-27 23:43:53,736 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step fix_metadata 2023-12-27 23:43:53,739 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'fix_metadata' on the data [] loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) check_level = , session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': None, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': True, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': False, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), preprocessor = 'mask_regrid_preproc', reference_dataset = 'GPCP-V2.3', mip = 'Amon', dataset = 'CanESM2', project = 'CMIP5', exp = 'historical', ensemble = 'r1i1p1', short_name = 'pr', timerange = '1979/2005', variable_group = 'pr', diagnostic = 'quantilebias', recipe_dataset_index = 0, alias = 'CMIP5', product = ['output1', 'output2'], institute = ['CCCma', 'CCCMA'], original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = ['atmos'], frequency = 'mon', version = 'v20120718' 2023-12-27 23:43:53,744 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step concatenate 2023-12-27 23:43:53,746 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'concatenate' on the data [] loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) check_level = 2023-12-27 23:43:53,746 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step cmor_check_metadata 2023-12-27 23:43:53,746 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'cmor_check_metadata' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) check_level = , cmor_table = 'CMIP5', mip = 'Amon', frequency = 'mon', short_name = 'pr' 2023-12-27 23:43:53,750 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step clip_timerange 2023-12-27 23:43:53,751 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'clip_timerange' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) timerange = '1979/2005' 2023-12-27 23:43:53,759 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step fix_data 2023-12-27 23:43:53,760 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'fix_data' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) check_level = , session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': None, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': True, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': False, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), preprocessor = 'mask_regrid_preproc', reference_dataset = 'GPCP-V2.3', mip = 'Amon', dataset = 'CanESM2', project = 'CMIP5', exp = 'historical', ensemble = 'r1i1p1', short_name = 'pr', timerange = '1979/2005', variable_group = 'pr', diagnostic = 'quantilebias', recipe_dataset_index = 0, alias = 'CMIP5', product = ['output1', 'output2'], institute = ['CCCma', 'CCCMA'], original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = ['atmos'], frequency = 'mon', version = 'v20120718' 2023-12-27 23:43:53,760 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step cmor_check_data 2023-12-27 23:43:53,761 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'cmor_check_data' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) check_level = , cmor_table = 'CMIP5', mip = 'Amon', frequency = 'mon', short_name = 'pr' 2023-12-27 23:43:53,761 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step fix_file 2023-12-27 23:43:53,761 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'fix_file' on the data LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc') loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) output_dir = PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/preproc/fixed_files/sftlf_fx_CMIP5_CanESM2_historical_r0i0p0_v20120410_'), add_unique_suffix = True, session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': None, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': True, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': False, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), reference_dataset = 'GPCP-V2.3', mip = 'fx', dataset = 'CanESM2', project = 'CMIP5', exp = 'historical', ensemble = 'r0i0p0', short_name = 'sftlf', institute = 'CCCma', product = ['output1', 'output2'], original_short_name = 'sftlf', standard_name = 'land_area_fraction', long_name = 'Land Area Fraction', units = '%', modeling_realm = ['atmos'], frequency = 'fx', version = 'v20120410' 2023-12-27 23:43:53,762 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step load 2023-12-27 23:43:53,762 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'load' on the data LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc') loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) ignore_warnings = None 2023-12-27 23:43:53,762 UTC [930142] DEBUG esmvalcore.preprocessor._io:145 Loading: /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc 2023-12-27 23:43:53,768 UTC [930142] DEBUG esmvalcore.preprocessor._io:176 Done with loading /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc 2023-12-27 23:43:53,768 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step fix_metadata 2023-12-27 23:43:53,769 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'fix_metadata' on the data [] loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) check_level = , session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': None, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': True, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': False, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), reference_dataset = 'GPCP-V2.3', mip = 'fx', dataset = 'CanESM2', project = 'CMIP5', exp = 'historical', ensemble = 'r0i0p0', short_name = 'sftlf', institute = 'CCCma', product = ['output1', 'output2'], original_short_name = 'sftlf', standard_name = 'land_area_fraction', long_name = 'Land Area Fraction', units = '%', modeling_realm = ['atmos'], frequency = 'fx', version = 'v20120410' 2023-12-27 23:43:53,769 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step concatenate 2023-12-27 23:43:53,770 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'concatenate' on the data [] loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) check_level = 2023-12-27 23:43:53,770 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step cmor_check_metadata 2023-12-27 23:43:53,770 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'cmor_check_metadata' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) check_level = , cmor_table = 'CMIP5', mip = 'fx', frequency = 'fx', short_name = 'sftlf' 2023-12-27 23:43:53,770 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step fix_data 2023-12-27 23:43:53,771 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'fix_data' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) check_level = , session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': None, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': True, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': False, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), reference_dataset = 'GPCP-V2.3', mip = 'fx', dataset = 'CanESM2', project = 'CMIP5', exp = 'historical', ensemble = 'r0i0p0', short_name = 'sftlf', institute = 'CCCma', product = ['output1', 'output2'], original_short_name = 'sftlf', standard_name = 'land_area_fraction', long_name = 'Land Area Fraction', units = '%', modeling_realm = ['atmos'], frequency = 'fx', version = 'v20120410' 2023-12-27 23:43:53,772 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step cmor_check_data 2023-12-27 23:43:53,772 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'cmor_check_data' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) check_level = , cmor_table = 'CMIP5', mip = 'fx', frequency = 'fx', short_name = 'sftlf' 2023-12-27 23:43:53,772 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step add_supplementary_variables 2023-12-27 23:43:53,773 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'add_supplementary_variables' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc'), LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) supplementary_cubes = [] 2023-12-27 23:43:53,773 UTC [930142] DEBUG esmvalcore.preprocessor._supplementary_vars:104 Added sftlf as ancillary variable in cube of pr. 2023-12-27 23:43:53,773 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step mask_landsea 2023-12-27 23:43:53,789 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'mask_landsea' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc'), LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) mask_out = 'sea' 2023-12-27 23:43:53,854 UTC [930142] DEBUG esmvalcore.preprocessor._mask:129 Applying land-sea mask: sftlf 2023-12-27 23:43:53,854 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step regrid 2023-12-27 23:43:53,863 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'regrid' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc'), LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) target_grid = '2x2', scheme = 'area_weighted' 2023-12-27 23:43:55,272 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step save 2023-12-27 23:43:55,273 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'save' on the data [] loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc'), LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) compress = False, filename = PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005.nc') 2023-12-27 23:43:55,792 UTC [930142] DEBUG esmvalcore.preprocessor._io:452 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 324; latitude: 90; longitude: 180) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean (interval: 15 minutes) Attributes: CCCma_data_licence '1) GRANT OF LICENCE - The Government of Canada (Environment Canada) is ...' CCCma_parent_runid 'IGA' CCCma_runid 'IGM' Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_CanESM2_historical_r0i0p0.nc ...' branch_time 171915.0 branch_time_YMDH '2321:01:01:00' cmor_version '2.5.4' contact 'cccma_info@ec.gc.ca' experiment 'historical' experiment_id 'historical' forcing 'GHG,Oz,SA,BC,OC,LU,Sl,Vl (GHG includes CO2,CH4,N2O,CFC11,effective CFC ...' frequency 'mon' initialization_method 1 institute_id 'CCCma' institution 'CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, ...' model_id 'CanESM2' modeling_realm 'atmos' original_name 'PCP' parent_experiment 'pre-industrial control' parent_experiment_id 'piControl' parent_experiment_rip 'r1i1p1' physics_version 1 product 'output' project_id 'CMIP5' realization 1 references 'http://www.cccma.ec.gc.ca/models' source 'CanESM2 2010 atmosphere: CanAM4 (AGCM15i, T63L35) ocean: CanOM4 (OGCM4.0, ...' table_id 'Table Amon (31 January 2011) 53b766a395ac41696af40aab76a49ae5' title 'CanESM2 model output prepared for CMIP5 historical' with realized data to /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005.nc 2023-12-27 23:43:57,520 UTC [930142] DEBUG esmvalcore.preprocessor:658 Applying single-model steps to PreprocessorFile: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005.nc 2023-12-27 23:43:57,521 UTC [930142] DEBUG esmvalcore.esgf._download:535 All required data is available locally, not downloading anything. 2023-12-27 23:43:57,521 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step fix_file 2023-12-27 23:43:57,522 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'fix_file' on the data LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc') loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) output_dir = PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/preproc/fixed_files/pr_Amon_obs4MIPs_GPCP-V2.3_v20180519_'), add_unique_suffix = True, session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': None, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': True, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': False, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), preprocessor = 'mask_regrid_preproc', reference_dataset = 'GPCP-V2.3', mip = 'Amon', dataset = 'GPCP-V2.3', project = 'obs4MIPs', level = 'L3', tier = 1, short_name = 'pr', timerange = '1979/2005', variable_group = 'pr', diagnostic = 'quantilebias', recipe_dataset_index = 1, alias = 'obs4MIPs', original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = [], frequency = 'mon', version = 'v20180519' 2023-12-27 23:43:57,523 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step load 2023-12-27 23:43:57,523 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'load' on the data LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc') loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) ignore_warnings = None 2023-12-27 23:43:57,524 UTC [930142] DEBUG esmvalcore.preprocessor._io:145 Loading: /scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc 2023-12-27 23:43:57,542 UTC [930142] DEBUG esmvalcore.preprocessor._io:176 Done with loading /scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc 2023-12-27 23:43:57,543 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step fix_metadata 2023-12-27 23:43:57,544 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'fix_metadata' on the data [] loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) check_level = , session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': None, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': True, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': False, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), preprocessor = 'mask_regrid_preproc', reference_dataset = 'GPCP-V2.3', mip = 'Amon', dataset = 'GPCP-V2.3', project = 'obs4MIPs', level = 'L3', tier = 1, short_name = 'pr', timerange = '1979/2005', variable_group = 'pr', diagnostic = 'quantilebias', recipe_dataset_index = 1, alias = 'obs4MIPs', original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = [], frequency = 'mon', version = 'v20180519' 2023-12-27 23:43:57,547 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step concatenate 2023-12-27 23:43:57,548 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'concatenate' on the data [] loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) check_level = 2023-12-27 23:43:57,548 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step cmor_check_metadata 2023-12-27 23:43:57,549 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'cmor_check_metadata' on the data loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) check_level = , cmor_table = 'obs4MIPs', mip = 'Amon', frequency = 'mon', short_name = 'pr' 2023-12-27 23:43:57,551 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step clip_timerange 2023-12-27 23:43:57,552 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'clip_timerange' on the data loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) timerange = '1979/2005' 2023-12-27 23:43:57,555 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step fix_data 2023-12-27 23:43:57,556 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'fix_data' on the data loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) check_level = , session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': None, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': True, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': False, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), preprocessor = 'mask_regrid_preproc', reference_dataset = 'GPCP-V2.3', mip = 'Amon', dataset = 'GPCP-V2.3', project = 'obs4MIPs', level = 'L3', tier = 1, short_name = 'pr', timerange = '1979/2005', variable_group = 'pr', diagnostic = 'quantilebias', recipe_dataset_index = 1, alias = 'obs4MIPs', original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = [], frequency = 'mon', version = 'v20180519' 2023-12-27 23:43:57,556 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step cmor_check_data 2023-12-27 23:43:57,557 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'cmor_check_data' on the data loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) check_level = , cmor_table = 'obs4MIPs', mip = 'Amon', frequency = 'mon', short_name = 'pr' 2023-12-27 23:43:57,557 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step add_supplementary_variables 2023-12-27 23:43:57,557 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'add_supplementary_variables' on the data loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) supplementary_cubes = [] 2023-12-27 23:43:57,557 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step mask_landsea 2023-12-27 23:43:57,558 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'mask_landsea' on the data loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) mask_out = 'sea' 2023-12-27 23:43:57,558 UTC [930142] DEBUG esmvalcore.preprocessor._mask:121 Ancillary variables land/sea area fraction not found in cube. Check fx_file availability. 2023-12-27 23:43:57,559 UTC [930142] DEBUG fiona._env:315 GDAL_DATA found in environment. 2023-12-27 23:43:57,559 UTC [930142] DEBUG fiona._env:315 PROJ_DATA found in environment. 2023-12-27 23:43:57,924 UTC [930142] DEBUG fiona._env:315 GDAL_DATA found in environment. 2023-12-27 23:43:57,924 UTC [930142] DEBUG fiona._env:315 PROJ_DATA found in environment. 2023-12-27 23:43:57,925 UTC [930142] DEBUG fiona.ogrext:453 OLC_FASTSETNEXTBYINDEX: 1 2023-12-27 23:43:57,925 UTC [930142] DEBUG fiona.ogrext:453 OLC_FASTFEATURECOUNT: 1 2023-12-27 23:43:57,925 UTC [930142] DEBUG fiona.ogrext:453 Next index: 0 2023-12-27 23:43:57,925 UTC [930142] DEBUG fiona.ogrext:210 Next index: 1 2023-12-27 23:43:58,488 UTC [930142] DEBUG fiona.collection:678 Flushed buffer 2023-12-27 23:43:58,488 UTC [930142] DEBUG fiona.collection:680 Stopped session 2023-12-27 23:43:58,535 UTC [930142] DEBUG esmvalcore.preprocessor._mask:135 Applying land-sea mask from Natural Earth shapefile: /projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvalcore/preprocessor/ne_masks/ne_50m_ocean.shp 2023-12-27 23:43:58,535 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step regrid 2023-12-27 23:43:58,536 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'regrid' on the data loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) target_grid = '2x2', scheme = 'area_weighted' 2023-12-27 23:43:59,376 UTC [930142] DEBUG esmvalcore.preprocessor:364 Running preprocessor step save 2023-12-27 23:43:59,377 UTC [930142] DEBUG esmvalcore.preprocessor:318 Running preprocessor function 'save' on the data [] loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) compress = False, filename = PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005.nc') 2023-12-27 23:43:59,377 UTC [930142] DEBUG esmvalcore.preprocessor._io:452 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 324; latitude: 90; longitude: 180) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean Attributes: Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_Obs-GPCP_GPCP_r0i0p0.nc ...' cmor_version '2.9.1' contact 'George Huffman (george.j.huffman@nasa.gov)' frequency 'mon' institute_id 'NASA-GSFC' institution 'NASA Goddard Space Flight Center, Greenbelt MD, USA' mip_specs 'CMIP5' model_id 'Obs-GPCP' modeling_realm 'atmos' original_name 'precip' product 'observations' project_id 'obs4MIPs' realm 'atmos' references 'Huffman et al. 1997, http://dx.doi.org/10.1175/1520-0477(1997)078<0005:TGPCPG>2.0.CO;2; ...' source 'Obs-GPCP (Global Precipitation Climatology Project) v23rB1' source_id 'GPCP' source_type 'satellite_retrieval_and_gauge_analysis' table_id 'Table Amon_ana (10 March 2011) 34230b4cbd7bedf38c827d6e41c1b8ea' title 'Global Precipitation Climatology Project (GPCP) Climate Data Record (CDR), ...' with realized data to /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005.nc 2023-12-27 23:43:59,593 UTC [930142] INFO esmvalcore._task:266 Successfully completed task quantilebias/pr (priority 1) in 0:00:09.072016 2023-12-27 23:43:59,626 UTC [929581] INFO esmvalcore._task:793 Progress: 0 tasks running, 1 tasks waiting for ancestors, 1/2 done 2023-12-27 23:43:59,630 UTC [930143] INFO esmvalcore._task:261 Starting task quantilebias/main in process [930143] 2023-12-27 23:43:59,726 UTC [929581] INFO esmvalcore._task:793 Progress: 1 tasks running, 0 tasks waiting for ancestors, 1/2 done 2023-12-27 23:43:59,819 UTC [930143] INFO esmvalcore._task:502 Running command ['/projects/NS2980K/conda/esmvaltool2.10.0/bin/Rscript', '/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvaltool/diag_scripts/quantilebias/quantilebias.R', '/projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/run/quantilebias/main/settings.yml'] 2023-12-27 23:43:59,819 UTC [930143] DEBUG esmvalcore._task:503 in environment {'diag_scripts': '/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvaltool/diag_scripts'} 2023-12-27 23:43:59,819 UTC [930143] DEBUG esmvalcore._task:505 in current working directory: /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/run/quantilebias/main 2023-12-27 23:43:59,819 UTC [930143] INFO esmvalcore._task:506 Writing output to /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/work/quantilebias/main 2023-12-27 23:43:59,819 UTC [930143] INFO esmvalcore._task:507 Writing plots to /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/plots/quantilebias/main 2023-12-27 23:43:59,820 UTC [930143] INFO esmvalcore._task:508 Writing log to /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/run/quantilebias/main/log.txt 2023-12-27 23:43:59,820 UTC [930143] INFO esmvalcore._task:518 To re-run this diagnostic script, run: cd /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/run/quantilebias/main; diag_scripts="/projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvaltool/diag_scripts" /projects/NS2980K/conda/esmvaltool2.10.0/bin/Rscript /projects/NS2980K/conda/esmvaltool2.10.0/lib/python3.11/site-packages/esmvaltool/diag_scripts/quantilebias/quantilebias.R /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/run/quantilebias/main/settings.yml 2023-12-27 23:44:05,484 UTC [930143] INFO esmvalcore._task:128 Maximum memory used (estimate): 0.1 GB 2023-12-27 23:44:05,485 UTC [930143] INFO esmvalcore._task:130 Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. 2023-12-27 23:44:05,485 UTC [930143] DEBUG esmvalcore._task:591 Script quantilebias/quantilebias.R completed successfully 2023-12-27 23:44:05,486 UTC [930143] DEBUG esmvalcore._task:610 Collecting provenance from /projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/run/quantilebias/main/diagnostic_provenance.yml 2023-12-27 23:44:07,686 UTC [930143] DEBUG PIL.PngImagePlugin:190 STREAM b'IHDR' 16 13 2023-12-27 23:44:07,686 UTC [930143] DEBUG PIL.PngImagePlugin:190 STREAM b'IDAT' 41 8192 2023-12-27 23:44:07,707 UTC [930143] DEBUG esmvalcore._task:685 Collecting provenance of task quantilebias/main took 2.2 seconds 2023-12-27 23:44:07,707 UTC [930143] INFO esmvalcore._task:266 Successfully completed task quantilebias/main (priority 0) in 0:00:08.076758 2023-12-27 23:44:07,753 UTC [929581] INFO esmvalcore._task:793 Progress: 0 tasks running, 0 tasks waiting for ancestors, 2/2 done 2023-12-27 23:44:07,753 UTC [929581] INFO esmvalcore._task:798 Successfully completed all tasks. 2023-12-27 23:44:07,769 UTC [929581] INFO esmvalcore._recipe.recipe:1076 Wrote recipe with version numbers and wildcards to: file:///projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/run/recipe_quantilebias_filled.yml 2023-12-27 23:44:08,309 UTC [929581] INFO esmvalcore.experimental.recipe_output:244 Wrote recipe output to: file:///projects/NS2980K/www/tomast/esmvaltool2.10.0/output/recipe_quantilebias_20231227_234153/index.html 2023-12-27 23:44:08,310 UTC [929581] INFO esmvalcore._main:133 Ending the Earth System Model Evaluation Tool at time: 2023-12-27 23:44:08 UTC 2023-12-27 23:44:08,310 UTC [929581] INFO esmvalcore._main:136 Time for running the recipe was: 0:02:13.605317 2023-12-27 23:44:08,367 UTC [929581] INFO esmvalcore._task:128 Maximum memory used (estimate): 1.6 GB 2023-12-27 23:44:08,368 UTC [929581] INFO esmvalcore._task:130 Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. 2023-12-27 23:44:08,368 UTC [929581] INFO esmvalcore._main:470 Removing `preproc` directory containing preprocessed data 2023-12-27 23:44:08,369 UTC [929581] INFO esmvalcore._main:473 If this data is further needed, then set `remove_preproc_dir` to `false` in your user configuration file 2023-12-27 23:44:09,359 UTC [929581] INFO esmvalcore._main:450 Run was successful