2025-01-31 16:49:47,919 UTC [2299982] INFO esmvalcore._main:507 ______________________________________________________________________ _____ ____ __ ____ __ _ _____ _ | ____/ ___|| \/ \ \ / /_ _| |_ _|__ ___ | | | _| \___ \| |\/| |\ \ / / _` | | | |/ _ \ / _ \| | | |___ ___) | | | | \ V / (_| | | | | (_) | (_) | | |_____|____/|_| |_| \_/ \__,_|_| |_|\___/ \___/|_| ______________________________________________________________________ ESMValTool - Earth System Model Evaluation Tool. http://www.esmvaltool.org CORE DEVELOPMENT TEAM AND CONTACTS: Birgit Hassler (Co-PI; DLR, Germany - birgit.hassler@dlr.de) Alistair Sellar (Co-PI; Met Office, UK - alistair.sellar@metoffice.gov.uk) Bouwe Andela (Netherlands eScience Center, The Netherlands - b.andela@esciencecenter.nl) Lee de Mora (PML, UK - ledm@pml.ac.uk) Niels Drost (Netherlands eScience Center, The Netherlands - n.drost@esciencecenter.nl) Veronika Eyring (DLR, Germany - veronika.eyring@dlr.de) Bettina Gier (UBremen, Germany - gier@uni-bremen.de) Remi Kazeroni (DLR, Germany - remi.kazeroni@dlr.de) Nikolay Koldunov (AWI, Germany - nikolay.koldunov@awi.de) Axel Lauer (DLR, Germany - axel.lauer@dlr.de) Saskia Loosveldt-Tomas (BSC, Spain - saskia.loosveldt@bsc.es) Ruth Lorenz (ETH Zurich, Switzerland - ruth.lorenz@env.ethz.ch) Benjamin Mueller (LMU, Germany - b.mueller@iggf.geo.uni-muenchen.de) Valeriu Predoi (URead, UK - valeriu.predoi@ncas.ac.uk) Mattia Righi (DLR, Germany - mattia.righi@dlr.de) Manuel Schlund (DLR, Germany - manuel.schlund@dlr.de) Breixo Solino Fernandez (DLR, Germany - breixo.solinofernandez@dlr.de) Javier Vegas-Regidor (BSC, Spain - javier.vegas@bsc.es) Klaus Zimmermann (SMHI, Sweden - klaus.zimmermann@smhi.se) For further help, please read the documentation at http://docs.esmvaltool.org. Have fun! 2025-01-31 16:49:47,920 UTC [2299982] INFO esmvalcore._main:508 Package versions 2025-01-31 16:49:47,920 UTC [2299982] INFO esmvalcore._main:509 ---------------- 2025-01-31 16:49:47,920 UTC [2299982] INFO esmvalcore._main:510 ESMValCore: 2.11.1 2025-01-31 16:49:47,920 UTC [2299982] INFO esmvalcore._main:512 ESMValTool: 2.11.0 2025-01-31 16:49:47,920 UTC [2299982] INFO esmvalcore._main:513 ---------------- 2025-01-31 16:49:47,920 UTC [2299982] INFO esmvalcore._main:514 Using config file /nird/home/tomast/.esmvaltool/config-user.yml 2025-01-31 16:49:47,920 UTC [2299982] INFO esmvalcore._main:515 Writing program log files to: /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/run/main_log.txt /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/run/main_log_debug.txt /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/run/cmor_log.txt 2025-01-31 16:49:48,051 UTC [2299982] INFO esmvalcore.config._esgf_pyclient:62 Using default ESGF configuration, configuration file /nird/home/tomast/.esmvaltool/esgf-pyclient.yml not present. 2025-01-31 16:49:48,164 UTC [2299982] INFO esmvalcore._main:91 Starting the Earth System Model Evaluation Tool at time: 2025-01-31 16:49:48 UTC 2025-01-31 16:49:48,165 UTC [2299982] INFO esmvalcore._main:95 ---------------------------------------------------------------------- 2025-01-31 16:49:48,165 UTC [2299982] INFO esmvalcore._main:96 RECIPE = /projects/NS2980K/home/tomast/ESMValTool/recipes-2.11.0/standard_recipes/atmosphere/recipe_quantilebias.yml 2025-01-31 16:49:48,165 UTC [2299982] INFO esmvalcore._main:97 RUNDIR = /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/run 2025-01-31 16:49:48,165 UTC [2299982] INFO esmvalcore._main:98 WORKDIR = /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/work 2025-01-31 16:49:48,165 UTC [2299982] INFO esmvalcore._main:99 PREPROCDIR = /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc 2025-01-31 16:49:48,166 UTC [2299982] INFO esmvalcore._main:100 PLOTDIR = /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/plots 2025-01-31 16:49:48,166 UTC [2299982] INFO esmvalcore._main:101 ---------------------------------------------------------------------- 2025-01-31 16:49:48,166 UTC [2299982] INFO esmvalcore._main:104 Running tasks using at most 16 processes 2025-01-31 16:49:48,167 UTC [2299982] INFO esmvalcore._main:106 If your system hangs during execution, it may not have enough memory for keeping this number of tasks in memory. 2025-01-31 16:49:48,167 UTC [2299982] INFO esmvalcore._main:109 If you experience memory problems, try reducing 'max_parallel_tasks' in your user configuration file. 2025-01-31 16:49:48,167 UTC [2299982] WARNING esmvalcore.config._dask:18 Using the Dask basic scheduler. This may lead to slow computations and out-of-memory errors. Note that the basic scheduler may still be the best choice for preprocessor functions that are not lazy. In that case, you can safely ignore this warning. See https://docs.esmvaltool.org/projects/ESMValCore/en/latest/quickstart/configure.html#dask-distributed-configuration for more information. 2025-01-31 16:49:48,169 UTC [2299982] DEBUG esmvalcore._recipe.check:61 Checking recipe against schema /projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvalcore/_recipe/recipe_schema.yml 2025-01-31 16:49:48,178 UTC [2299982] DEBUG esmvalcore._recipe.to_datasets:347 Populating list of datasets for variable pr in diagnostic quantilebias 2025-01-31 16:49:48,179 UTC [2299982] DEBUG esmvalcore.config._config:45 Loading extra facets from /projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvalcore/config/extra_facets/cmip5-fx.yml 2025-01-31 16:49:48,179 UTC [2299982] DEBUG esmvalcore.config._config:45 Loading extra facets from /projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvalcore/config/extra_facets/cmip5-institutes.yml 2025-01-31 16:49:48,193 UTC [2299982] DEBUG esmvalcore.config._config:45 Loading extra facets from /projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvalcore/config/extra_facets/cmip5-product.yml 2025-01-31 16:49:48,197 UTC [2299982] DEBUG esmvalcore.local:439 Looking for files matching: /projects/NS9034K/CMIP5/output1/*/CanESM2/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_CanESM2_*_**.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/*/CanESM2/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_CanESM2_*_**.nc /scratch/tomast/ESGF/cmip5/output1/*/CanESM2/*/fx/atmos/fx/*/*/sftlf/sftlf_fx_CanESM2_*_**.nc 2025-01-31 16:49:48,433 UTC [2299982] DEBUG esmvalcore._recipe.to_datasets:371 Found Dataset: pr, Amon, CMIP5, CanESM2, historical, r1i1p1, supplementaries: sftlf, fx, r0i0p0 2025-01-31 16:49:48,434 UTC [2299982] DEBUG esmvalcore.local:439 Looking for files matching: /projects/NS9560K-datalake/ESGF/obs4MIPs/Tier1/GPCP-V2.3/sftlf_*.nc /projects/NS9560K-datalake/ESGF/obsdata/Tier1/GPCP-V2.3/sftlf_*.nc /scratch/tomast/ESGF/obs4MIPs/Tier1/GPCP-V2.3/sftlf_*.nc 2025-01-31 16:49:48,447 UTC [2299982] DEBUG esmvalcore.esgf._search:116 Searching https://esgf.ceda.ac.uk/esg-search for datasets using facets={'project': 'obs4MIPs', 'source_id': 'GPCP-V2.3', 'time_frequency': 'mon', 'variable': 'sftlf'} 2025-01-31 16:49:48,448 UTC [2299982] DEBUG requests_cache.backends:66 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2025-01-31 16:49:48,448 UTC [2299982] DEBUG requests_cache.backends.base:322 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2025-01-31 16:49:48,448 UTC [2299982] DEBUG requests_cache.backends.sqlite:252 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2025-01-31 16:49:48,571 UTC [2299982] DEBUG requests_cache.backends.base:322 Initialized SQLiteDict with serializer: None 2025-01-31 16:49:48,571 UTC [2299982] DEBUG requests_cache.backends.sqlite:252 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2025-01-31 16:49:48,574 UTC [2299982] DEBUG requests_cache.policy.actions:103 Cache directives from request headers: CacheDirectives() 2025-01-31 16:49:48,574 UTC [2299982] DEBUG requests_cache.policy.actions:368 Pre-read cache checks: Passed 2025-01-31 16:49:48,617 UTC [2299982] DEBUG requests_cache.policy.actions:217 Post-read cache actions: CacheActions(expire_after=86400, resend_request=True) 2025-01-31 16:49:48,617 UTC [2299982] DEBUG requests_cache.session:284 Stale response; attempting to re-send request 2025-01-31 16:49:48,617 UTC [2299982] DEBUG urllib3.connectionpool:1049 Starting new HTTPS connection (1): esgf.ceda.ac.uk:443 2025-01-31 16:49:49,002 UTC [2299982] DEBUG urllib3.connectionpool:544 https://esgf.ceda.ac.uk:443 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=obs4MIPs&source_id=GPCP-V2.3&time_frequency=mon&variable=sftlf HTTP/1.1" 200 1100 2025-01-31 16:49:49,002 UTC [2299982] DEBUG requests_cache.policy.actions:368 Pre-write cache checks: Passed 2025-01-31 16:49:49,540 UTC [2299982] DEBUG requests_cache.backends.base:115 Closing backend connections 2025-01-31 16:49:49,541 UTC [2299982] DEBUG esmvalcore.esgf._search:154 Found the following files matching facets {'project': 'obs4MIPs', 'source_id': 'GPCP-V2.3', 'time_frequency': 'mon', 'variable': 'sftlf'}: none 2025-01-31 16:49:49,541 UTC [2299982] DEBUG esmvalcore.esgf._search:364 Selected files: 2025-01-31 16:49:49,541 UTC [2299982] INFO esmvalcore.dataset:315 For Dataset: pr, Amon, obs4MIPs, GPCP-V2.3, supplementaries: sftlf: ignoring supplementary variable 'sftlf', unable to expand wildcards 'institute'. 2025-01-31 16:49:49,541 UTC [2299982] DEBUG esmvalcore._recipe.to_datasets:371 Found Dataset: pr, Amon, obs4MIPs, GPCP-V2.3 2025-01-31 16:49:49,542 UTC [2299982] DEBUG esmvalcore._recipe.recipe:801 Retrieving diagnostics from recipe 2025-01-31 16:49:49,542 UTC [2299982] DEBUG esmvalcore._recipe.recipe:829 Setting script for diagnostic quantilebias 2025-01-31 16:49:49,542 UTC [2299982] INFO esmvalcore._recipe.recipe:1019 Creating tasks from recipe 2025-01-31 16:49:49,542 UTC [2299982] INFO esmvalcore._recipe.recipe:1028 Creating tasks for diagnostic quantilebias 2025-01-31 16:49:49,542 UTC [2299982] INFO esmvalcore._recipe.recipe:950 Creating diagnostic task quantilebias/main 2025-01-31 16:49:49,542 UTC [2299982] INFO esmvalcore._recipe.recipe:1003 Creating preprocessor task quantilebias/pr 2025-01-31 16:49:49,543 UTC [2299982] INFO esmvalcore._recipe.recipe:673 Creating preprocessor 'mask_regrid_preproc' task for variable 'pr' 2025-01-31 16:49:49,543 UTC [2299982] DEBUG esmvalcore.local:439 Looking for files matching: /projects/NS9034K/CMIP5/output1/CCCMA/CanESM2/historical/mon/atmos/Amon/r1i1p1/*/pr/pr_Amon_CanESM2_historical_r1i1p1*.nc /projects/NS9034K/CMIP5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/*/pr/pr_Amon_CanESM2_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCMA/CanESM2/historical/mon/atmos/Amon/r1i1p1/*/pr/pr_Amon_CanESM2_historical_r1i1p1*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/*/pr/pr_Amon_CanESM2_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/CCCMA/CanESM2/historical/mon/atmos/Amon/r1i1p1/*/pr/pr_Amon_CanESM2_historical_r1i1p1*.nc /scratch/tomast/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/*/pr/pr_Amon_CanESM2_historical_r1i1p1*.nc 2025-01-31 16:49:49,614 UTC [2299982] DEBUG esmvalcore.local:439 Looking for files matching: /projects/NS9034K/CMIP5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_CanESM2_historical_r0i0p0*.nc /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_CanESM2_historical_r0i0p0*.nc /scratch/tomast/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/*/sftlf/sftlf_fx_CanESM2_historical_r0i0p0*.nc 2025-01-31 16:49:49,615 UTC [2299982] DEBUG esmvalcore._recipe.recipe:270 Using input files for variable pr of dataset CMIP5: /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc with files for supplementary variable sftlf: /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc 2025-01-31 16:49:49,615 UTC [2299982] INFO esmvalcore._recipe.recipe:536 Found input files for Dataset: pr, Amon, CMIP5, CanESM2, historical, r1i1p1, v20120718, supplementaries: sftlf, fx, r0i0p0, v20120410 2025-01-31 16:49:49,615 UTC [2299982] WARNING esmvalcore._recipe.check:217 Preprocessor function mask_landsea works best when at least one supplementary variable of ['sftlf', 'sftof'] is defined in the recipe for Dataset: {'diagnostic': 'quantilebias', 'variable_group': 'pr', 'dataset': 'GPCP-V2.3', 'project': 'obs4MIPs', 'mip': 'Amon', 'short_name': 'pr', 'alias': 'obs4MIPs', 'frequency': 'mon', 'level': 'L3', 'long_name': 'Precipitation', 'modeling_realm': [], 'original_short_name': 'pr', 'preprocessor': 'mask_regrid_preproc', 'recipe_dataset_index': 1, 'reference_dataset': 'GPCP-V2.3', 'standard_name': 'precipitation_flux', 'tier': 1, 'timerange': '1979/2005', 'units': 'kg m-2 s-1'} session: 'recipe_quantilebias_20250131_164947'. 2025-01-31 16:49:49,615 UTC [2299982] DEBUG esmvalcore.local:439 Looking for files matching: /projects/NS9560K-datalake/ESGF/obs4MIPs/Tier1/GPCP-V2.3/pr_*.nc /projects/NS9560K-datalake/ESGF/obsdata/Tier1/GPCP-V2.3/pr_*.nc /scratch/tomast/ESGF/obs4MIPs/Tier1/GPCP-V2.3/pr_*.nc 2025-01-31 16:49:49,616 UTC [2299982] DEBUG esmvalcore.esgf._search:116 Searching https://esgf.ceda.ac.uk/esg-search for datasets using facets={'project': 'obs4MIPs', 'source_id': 'GPCP-V2.3', 'time_frequency': 'mon', 'variable': 'pr'} 2025-01-31 16:49:49,616 UTC [2299982] DEBUG requests_cache.backends:66 Initializing backend: None /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results 2025-01-31 16:49:49,616 UTC [2299982] DEBUG requests_cache.backends.base:322 Initialized SQLiteDict with serializer: SerializerPipeline(name=pickle, n_stages=2) 2025-01-31 16:49:49,616 UTC [2299982] DEBUG requests_cache.backends.sqlite:252 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:responses 2025-01-31 16:49:49,616 UTC [2299982] DEBUG requests_cache.backends.base:322 Initialized SQLiteDict with serializer: None 2025-01-31 16:49:49,617 UTC [2299982] DEBUG requests_cache.backends.sqlite:252 Opening connection to /nird/home/tomast/.esmvaltool/cache/pyesgf-search-results.sqlite:redirects 2025-01-31 16:49:49,618 UTC [2299982] DEBUG requests_cache.policy.actions:103 Cache directives from request headers: CacheDirectives() 2025-01-31 16:49:49,618 UTC [2299982] DEBUG requests_cache.policy.actions:368 Pre-read cache checks: Passed 2025-01-31 16:49:49,640 UTC [2299982] DEBUG requests_cache.policy.actions:217 Post-read cache actions: CacheActions(expire_after=86400, send_request=True) 2025-01-31 16:49:49,641 UTC [2299982] DEBUG urllib3.connectionpool:1049 Starting new HTTPS connection (1): esgf.ceda.ac.uk:443 2025-01-31 16:49:49,929 UTC [2299982] DEBUG urllib3.connectionpool:544 https://esgf.ceda.ac.uk:443 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&project=obs4MIPs&source_id=GPCP-V2.3&time_frequency=mon&variable=pr HTTP/1.1" 200 2848 2025-01-31 16:49:49,929 UTC [2299982] DEBUG requests_cache.policy.actions:368 Pre-write cache checks: Passed 2025-01-31 16:49:50,135 UTC [2299982] DEBUG requests_cache.backends.base:115 Closing backend connections 2025-01-31 16:49:50,135 UTC [2299982] DEBUG esmvalcore.esgf._download:323 Wrong dataset_id_template_ %(project)s.%(institute)s.%(source_id)s.%(realm)s.%(time_frequency)s or facet values containing '.' for dataset obs4MIPs.NASA-GSFC.GPCP-V2.3.atmos.mon.v20180519 2025-01-31 16:49:50,135 UTC [2299982] DEBUG esmvalcore.esgf._download:272 Correcting facet 'version' from 'None' to 'v20180519' for obs4MIPs.GPCP-V2.3.v20180519.pr_GPCP-SG_L3_v2.3_197901-201710.nc 2025-01-31 16:49:50,136 UTC [2299982] DEBUG esmvalcore.esgf._search:154 Found the following files matching facets {'project': 'obs4MIPs', 'source_id': 'GPCP-V2.3', 'time_frequency': 'mon', 'variable': 'pr'}: ESGFFile:obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc on hosts ['dpesgf03.nccs.nasa.gov'] 2025-01-31 16:49:50,136 UTC [2299982] DEBUG esmvalcore.esgf._search:364 Selected files: ESGFFile:obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc on hosts ['dpesgf03.nccs.nasa.gov'] 2025-01-31 16:49:50,136 UTC [2299982] DEBUG esmvalcore._recipe.recipe:270 Using input files for variable pr of dataset obs4MIPs: /scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc (will be downloaded) 2025-01-31 16:49:50,136 UTC [2299982] INFO esmvalcore._recipe.recipe:536 Found input files for Dataset: pr, Amon, obs4MIPs, GPCP-V2.3, v20180519 2025-01-31 16:49:50,137 UTC [2299982] INFO esmvalcore._recipe.recipe:697 PreprocessingTask quantilebias/pr created. 2025-01-31 16:49:50,137 UTC [2299982] DEBUG esmvalcore._recipe.recipe:698 PreprocessingTask quantilebias/pr will create the files: /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005.nc /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005.nc 2025-01-31 16:49:50,137 UTC [2299982] DEBUG esmvalcore._recipe.recipe:878 Linking tasks for diagnostic quantilebias script main 2025-01-31 16:49:50,137 UTC [2299982] DEBUG esmvalcore._recipe.recipe:887 Pattern quantilebias/pr matches ['quantilebias/pr'] 2025-01-31 16:49:50,137 UTC [2299982] INFO esmvalcore._recipe.recipe:1068 These tasks will be executed: quantilebias/pr, quantilebias/main 2025-01-31 16:49:50,138 UTC [2299982] DEBUG esmvalcore._main:128 Recipe summary: DiagnosticTask: quantilebias/main script: quantilebias/quantilebias.R settings: {'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'log_level': 'info', 'output_file_type': 'png', 'perc_lev': 75, 'plot_dir': '/projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/plots/quantilebias/main', 'profile_diagnostic': False, 'recipe': PosixPath('recipe_quantilebias.yml'), 'run_dir': '/projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/run/quantilebias/main', 'script': 'main', 'version': '2.11.1', 'work_dir': '/projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/work/quantilebias/main'} ancestors: PreprocessingTask: quantilebias/pr order: ['mask_landsea', 'regrid', 'remove_supplementary_variables', 'save'] PreprocessorFile: /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005.nc input files: [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc'), LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] settings: {'mask_landsea': {'mask_out': 'sea'}, 'regrid': {'scheme': 'area_weighted', 'target_grid': '2x2'}, 'remove_supplementary_variables': {}, 'save': {'compress': False, 'filename': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005.nc')}} PreprocessorFile: /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005.nc input files: [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] settings: {'mask_landsea': {'mask_out': 'sea'}, 'regrid': {'scheme': 'area_weighted', 'target_grid': '2x2'}, 'remove_supplementary_variables': {}, 'save': {'compress': False, 'filename': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005.nc')}} ancestors: None 2025-01-31 16:49:50,141 UTC [2299982] INFO esmvalcore._recipe.recipe:1131 Wrote recipe with version numbers and wildcards to: file:///projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/run/recipe_quantilebias_filled.yml 2025-01-31 16:49:50,141 UTC [2299982] INFO esmvalcore.esgf._download:543 Will download 19.35 MB Will download the following files: 19.35 MB ESGFFile:obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc on hosts ['dpesgf03.nccs.nasa.gov'] Downloading 19.35 MB.. 2025-01-31 16:49:50,151 UTC [2299982] DEBUG esmvalcore.esgf._download:461 Downloading https://dpesgf03.nccs.nasa.gov/thredds/fileServer/obs4MIPs/observations/NASA-GSFC/Obs-GPCP/GPCP/V2.3/atmos/pr/pr_GPCP-SG_L3_v2.3_197901-201710.nc to /scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc.1t940mhm 2025-01-31 16:49:50,152 UTC [2299982] DEBUG urllib3.connectionpool:1049 Starting new HTTPS connection (1): dpesgf03.nccs.nasa.gov:443 2025-01-31 16:49:51,119 UTC [2299982] DEBUG urllib3.connectionpool:544 https://dpesgf03.nccs.nasa.gov:443 "GET /thredds/fileServer/obs4MIPs/observations/NASA-GSFC/Obs-GPCP/GPCP/V2.3/atmos/pr/pr_GPCP-SG_L3_v2.3_197901-201710.nc HTTP/1.1" 200 19348352 2025-01-31 16:49:52,545 UTC [2299982] INFO esmvalcore.esgf._download:493 Downloaded /scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc (19.35 MB) in 2.26 seconds (8.57 MB/s) from dpesgf03.nccs.nasa.gov 2025-01-31 16:49:52,554 UTC [2299982] INFO esmvalcore.esgf._download:572 Downloaded 19.35 MB in 2.41 seconds (8.02 MB/s) 2025-01-31 16:49:52,554 UTC [2299982] INFO esmvalcore.esgf._download:584 Successfully downloaded all requested files. 2025-01-31 16:49:52,554 UTC [2299982] INFO esmvalcore.config._dask:70 Using the Dask basic scheduler. 2025-01-31 16:49:52,554 UTC [2299982] INFO esmvalcore._task:760 Running 2 tasks using 2 processes 2025-01-31 16:49:52,654 UTC [2300145] INFO esmvalcore._task:261 Starting task quantilebias/pr in process [2300145] 2025-01-31 16:49:52,696 UTC [2299982] INFO esmvalcore._task:793 Progress: 1 tasks running, 1 tasks waiting for ancestors, 0/2 done 2025-01-31 16:49:52,855 UTC [2300145] DEBUG esmvalcore.preprocessor:674 Running block ['mask_landsea', 'regrid'] 2025-01-31 16:49:52,855 UTC [2300145] DEBUG esmvalcore.preprocessor:681 Applying single-model steps to PreprocessorFile: /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005.nc 2025-01-31 16:49:52,856 UTC [2300145] DEBUG esmvalcore.esgf._download:538 All required data is available locally, not downloading anything. 2025-01-31 16:49:52,856 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step fix_file 2025-01-31 16:49:52,857 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'fix_file' on the data LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc') loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) output_dir = PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/fixed_files/pr_Amon_CMIP5_CanESM2_historical_r1i1p1_v20120718_'), add_unique_suffix = True, session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': 16, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.11.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': False, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS2980K-datalake/shared/external/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': True, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), preprocessor = 'mask_regrid_preproc', reference_dataset = 'GPCP-V2.3', mip = 'Amon', dataset = 'CanESM2', project = 'CMIP5', exp = 'historical', ensemble = 'r1i1p1', short_name = 'pr', timerange = '1979/2005', variable_group = 'pr', diagnostic = 'quantilebias', recipe_dataset_index = 0, alias = 'CMIP5', product = ['output1', 'output2'], institute = ['CCCma', 'CCCMA'], original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = ['atmos'], frequency = 'mon', version = 'v20120718' 2025-01-31 16:49:52,859 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] 2025-01-31 16:49:52,859 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step load 2025-01-31 16:49:52,859 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'load' on the data LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc') loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) ignore_warnings = None 2025-01-31 16:49:52,859 UTC [2300145] DEBUG esmvalcore.preprocessor._io:98 Loading: /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc 2025-01-31 16:49:53,361 UTC [2300145] DEBUG esmvalcore.preprocessor._io:134 Done with loading /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc 2025-01-31 16:49:53,361 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:53,365 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 1872; latitude: 64; longitude: 128) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean (interval: 15 minutes) Attributes: CCCma_data_licence '1) GRANT OF LICENCE - The Government of Canada (Environment Canada) is ...' CCCma_parent_runid 'IGA' CCCma_runid 'IGM' Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_CanESM2_historical_r0i0p0.nc ...' branch_time 171915.0 branch_time_YMDH '2321:01:01:00' cmor_version '2.5.4' contact 'cccma_info@ec.gc.ca' experiment 'historical' experiment_id 'historical' forcing 'GHG,Oz,SA,BC,OC,LU,Sl,Vl (GHG includes CO2,CH4,N2O,CFC11,effective CFC ...' frequency 'mon' initialization_method 1 institute_id 'CCCma' institution 'CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, ...' model_id 'CanESM2' modeling_realm 'atmos' original_name 'PCP' parent_experiment 'pre-industrial control' parent_experiment_id 'piControl' parent_experiment_rip 'r1i1p1' physics_version 1 product 'output' project_id 'CMIP5' realization 1 references 'http://www.cccma.ec.gc.ca/models' source 'CanESM2 2010 atmosphere: CanAM4 (AGCM15i, T63L35) ocean: CanOM4 (OGCM4.0, ...' source_file '/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical ...' table_id 'Table Amon (31 January 2011) 53b766a395ac41696af40aab76a49ae5' title 'CanESM2 model output prepared for CMIP5 historical' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005/00_load.nc 2025-01-31 16:49:53,458 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step fix_metadata 2025-01-31 16:49:53,461 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'fix_metadata' on the data [] loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) check_level = , session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': 16, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.11.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': False, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS2980K-datalake/shared/external/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': True, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), preprocessor = 'mask_regrid_preproc', reference_dataset = 'GPCP-V2.3', mip = 'Amon', dataset = 'CanESM2', project = 'CMIP5', exp = 'historical', ensemble = 'r1i1p1', short_name = 'pr', timerange = '1979/2005', variable_group = 'pr', diagnostic = 'quantilebias', recipe_dataset_index = 0, alias = 'CMIP5', product = ['output1', 'output2'], institute = ['CCCma', 'CCCMA'], original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = ['atmos'], frequency = 'mon', version = 'v20120718' 2025-01-31 16:49:53,466 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:53,467 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 1872; latitude: 64; longitude: 128) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean (interval: 15 minutes) Attributes: CCCma_data_licence '1) GRANT OF LICENCE - The Government of Canada (Environment Canada) is ...' CCCma_parent_runid 'IGA' CCCma_runid 'IGM' Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_CanESM2_historical_r0i0p0.nc ...' branch_time 171915.0 branch_time_YMDH '2321:01:01:00' cmor_version '2.5.4' contact 'cccma_info@ec.gc.ca' experiment 'historical' experiment_id 'historical' forcing 'GHG,Oz,SA,BC,OC,LU,Sl,Vl (GHG includes CO2,CH4,N2O,CFC11,effective CFC ...' frequency 'mon' initialization_method 1 institute_id 'CCCma' institution 'CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, ...' model_id 'CanESM2' modeling_realm 'atmos' original_name 'PCP' parent_experiment 'pre-industrial control' parent_experiment_id 'piControl' parent_experiment_rip 'r1i1p1' physics_version 1 product 'output' project_id 'CMIP5' realization 1 references 'http://www.cccma.ec.gc.ca/models' source 'CanESM2 2010 atmosphere: CanAM4 (AGCM15i, T63L35) ocean: CanOM4 (OGCM4.0, ...' table_id 'Table Amon (31 January 2011) 53b766a395ac41696af40aab76a49ae5' title 'CanESM2 model output prepared for CMIP5 historical' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005/01_fix_metadata.nc 2025-01-31 16:49:53,556 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step concatenate 2025-01-31 16:49:53,558 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'concatenate' on the data [] loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) check_level = 2025-01-31 16:49:53,558 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:53,559 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 1872; latitude: 64; longitude: 128) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean (interval: 15 minutes) Attributes: CCCma_data_licence '1) GRANT OF LICENCE - The Government of Canada (Environment Canada) is ...' CCCma_parent_runid 'IGA' CCCma_runid 'IGM' Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_CanESM2_historical_r0i0p0.nc ...' branch_time 171915.0 branch_time_YMDH '2321:01:01:00' cmor_version '2.5.4' contact 'cccma_info@ec.gc.ca' experiment 'historical' experiment_id 'historical' forcing 'GHG,Oz,SA,BC,OC,LU,Sl,Vl (GHG includes CO2,CH4,N2O,CFC11,effective CFC ...' frequency 'mon' initialization_method 1 institute_id 'CCCma' institution 'CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, ...' model_id 'CanESM2' modeling_realm 'atmos' original_name 'PCP' parent_experiment 'pre-industrial control' parent_experiment_id 'piControl' parent_experiment_rip 'r1i1p1' physics_version 1 product 'output' project_id 'CMIP5' realization 1 references 'http://www.cccma.ec.gc.ca/models' source 'CanESM2 2010 atmosphere: CanAM4 (AGCM15i, T63L35) ocean: CanOM4 (OGCM4.0, ...' table_id 'Table Amon (31 January 2011) 53b766a395ac41696af40aab76a49ae5' title 'CanESM2 model output prepared for CMIP5 historical' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005/02_concatenate.nc 2025-01-31 16:49:53,646 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step cmor_check_metadata 2025-01-31 16:49:53,648 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'cmor_check_metadata' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) check_level = , cmor_table = 'CMIP5', mip = 'Amon', frequency = 'mon', short_name = 'pr' 2025-01-31 16:49:53,651 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:53,652 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 1872; latitude: 64; longitude: 128) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean (interval: 15 minutes) Attributes: CCCma_data_licence '1) GRANT OF LICENCE - The Government of Canada (Environment Canada) is ...' CCCma_parent_runid 'IGA' CCCma_runid 'IGM' Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_CanESM2_historical_r0i0p0.nc ...' branch_time 171915.0 branch_time_YMDH '2321:01:01:00' cmor_version '2.5.4' contact 'cccma_info@ec.gc.ca' experiment 'historical' experiment_id 'historical' forcing 'GHG,Oz,SA,BC,OC,LU,Sl,Vl (GHG includes CO2,CH4,N2O,CFC11,effective CFC ...' frequency 'mon' initialization_method 1 institute_id 'CCCma' institution 'CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, ...' model_id 'CanESM2' modeling_realm 'atmos' original_name 'PCP' parent_experiment 'pre-industrial control' parent_experiment_id 'piControl' parent_experiment_rip 'r1i1p1' physics_version 1 product 'output' project_id 'CMIP5' realization 1 references 'http://www.cccma.ec.gc.ca/models' source 'CanESM2 2010 atmosphere: CanAM4 (AGCM15i, T63L35) ocean: CanOM4 (OGCM4.0, ...' table_id 'Table Amon (31 January 2011) 53b766a395ac41696af40aab76a49ae5' title 'CanESM2 model output prepared for CMIP5 historical' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005/03_cmor_check_metadata.nc 2025-01-31 16:49:53,739 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step clip_timerange 2025-01-31 16:49:53,741 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'clip_timerange' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) timerange = '1979/2005' 2025-01-31 16:49:53,749 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:53,753 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 324; latitude: 64; longitude: 128) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean (interval: 15 minutes) Attributes: CCCma_data_licence '1) GRANT OF LICENCE - The Government of Canada (Environment Canada) is ...' CCCma_parent_runid 'IGA' CCCma_runid 'IGM' Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_CanESM2_historical_r0i0p0.nc ...' branch_time 171915.0 branch_time_YMDH '2321:01:01:00' cmor_version '2.5.4' contact 'cccma_info@ec.gc.ca' experiment 'historical' experiment_id 'historical' forcing 'GHG,Oz,SA,BC,OC,LU,Sl,Vl (GHG includes CO2,CH4,N2O,CFC11,effective CFC ...' frequency 'mon' initialization_method 1 institute_id 'CCCma' institution 'CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, ...' model_id 'CanESM2' modeling_realm 'atmos' original_name 'PCP' parent_experiment 'pre-industrial control' parent_experiment_id 'piControl' parent_experiment_rip 'r1i1p1' physics_version 1 product 'output' project_id 'CMIP5' realization 1 references 'http://www.cccma.ec.gc.ca/models' source 'CanESM2 2010 atmosphere: CanAM4 (AGCM15i, T63L35) ocean: CanOM4 (OGCM4.0, ...' table_id 'Table Amon (31 January 2011) 53b766a395ac41696af40aab76a49ae5' title 'CanESM2 model output prepared for CMIP5 historical' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005/04_clip_timerange.nc 2025-01-31 16:49:53,819 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step fix_data 2025-01-31 16:49:53,820 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'fix_data' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) check_level = , session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': 16, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.11.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': False, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS2980K-datalake/shared/external/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': True, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), preprocessor = 'mask_regrid_preproc', reference_dataset = 'GPCP-V2.3', mip = 'Amon', dataset = 'CanESM2', project = 'CMIP5', exp = 'historical', ensemble = 'r1i1p1', short_name = 'pr', timerange = '1979/2005', variable_group = 'pr', diagnostic = 'quantilebias', recipe_dataset_index = 0, alias = 'CMIP5', product = ['output1', 'output2'], institute = ['CCCma', 'CCCMA'], original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = ['atmos'], frequency = 'mon', version = 'v20120718' 2025-01-31 16:49:53,821 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:53,821 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 324; latitude: 64; longitude: 128) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean (interval: 15 minutes) Attributes: CCCma_data_licence '1) GRANT OF LICENCE - The Government of Canada (Environment Canada) is ...' CCCma_parent_runid 'IGA' CCCma_runid 'IGM' Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_CanESM2_historical_r0i0p0.nc ...' branch_time 171915.0 branch_time_YMDH '2321:01:01:00' cmor_version '2.5.4' contact 'cccma_info@ec.gc.ca' experiment 'historical' experiment_id 'historical' forcing 'GHG,Oz,SA,BC,OC,LU,Sl,Vl (GHG includes CO2,CH4,N2O,CFC11,effective CFC ...' frequency 'mon' initialization_method 1 institute_id 'CCCma' institution 'CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, ...' model_id 'CanESM2' modeling_realm 'atmos' original_name 'PCP' parent_experiment 'pre-industrial control' parent_experiment_id 'piControl' parent_experiment_rip 'r1i1p1' physics_version 1 product 'output' project_id 'CMIP5' realization 1 references 'http://www.cccma.ec.gc.ca/models' source 'CanESM2 2010 atmosphere: CanAM4 (AGCM15i, T63L35) ocean: CanOM4 (OGCM4.0, ...' table_id 'Table Amon (31 January 2011) 53b766a395ac41696af40aab76a49ae5' title 'CanESM2 model output prepared for CMIP5 historical' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005/05_fix_data.nc 2025-01-31 16:49:53,886 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step cmor_check_data 2025-01-31 16:49:53,887 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'cmor_check_data' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc')] with function argument(s) check_level = , cmor_table = 'CMIP5', mip = 'Amon', frequency = 'mon', short_name = 'pr' 2025-01-31 16:49:53,887 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:53,888 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 324; latitude: 64; longitude: 128) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean (interval: 15 minutes) Attributes: CCCma_data_licence '1) GRANT OF LICENCE - The Government of Canada (Environment Canada) is ...' CCCma_parent_runid 'IGA' CCCma_runid 'IGM' Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_CanESM2_historical_r0i0p0.nc ...' branch_time 171915.0 branch_time_YMDH '2321:01:01:00' cmor_version '2.5.4' contact 'cccma_info@ec.gc.ca' experiment 'historical' experiment_id 'historical' forcing 'GHG,Oz,SA,BC,OC,LU,Sl,Vl (GHG includes CO2,CH4,N2O,CFC11,effective CFC ...' frequency 'mon' initialization_method 1 institute_id 'CCCma' institution 'CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, ...' model_id 'CanESM2' modeling_realm 'atmos' original_name 'PCP' parent_experiment 'pre-industrial control' parent_experiment_id 'piControl' parent_experiment_rip 'r1i1p1' physics_version 1 product 'output' project_id 'CMIP5' realization 1 references 'http://www.cccma.ec.gc.ca/models' source 'CanESM2 2010 atmosphere: CanAM4 (AGCM15i, T63L35) ocean: CanOM4 (OGCM4.0, ...' table_id 'Table Amon (31 January 2011) 53b766a395ac41696af40aab76a49ae5' title 'CanESM2 model output prepared for CMIP5 historical' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005/06_cmor_check_data.nc 2025-01-31 16:49:53,968 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step fix_file 2025-01-31 16:49:53,969 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'fix_file' on the data LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc') loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) output_dir = PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/fixed_files/sftlf_fx_CMIP5_CanESM2_historical_r0i0p0_v20120410_'), add_unique_suffix = True, session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': 16, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.11.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': False, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS2980K-datalake/shared/external/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': True, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), reference_dataset = 'GPCP-V2.3', mip = 'fx', dataset = 'CanESM2', project = 'CMIP5', exp = 'historical', ensemble = 'r0i0p0', short_name = 'sftlf', institute = 'CCCma', product = ['output1', 'output2'], original_short_name = 'sftlf', standard_name = 'land_area_fraction', long_name = 'Land Area Fraction', units = '%', modeling_realm = ['atmos'], frequency = 'fx', version = 'v20120410' 2025-01-31 16:49:53,969 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] 2025-01-31 16:49:53,969 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step load 2025-01-31 16:49:53,969 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'load' on the data LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc') loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) ignore_warnings = None 2025-01-31 16:49:53,969 UTC [2300145] DEBUG esmvalcore.preprocessor._io:98 Loading: /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc 2025-01-31 16:49:53,976 UTC [2300145] DEBUG esmvalcore.preprocessor._io:134 Done with loading /projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc 2025-01-31 16:49:53,976 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:53,978 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: land_area_fraction / (%) (latitude: 64; longitude: 128) Dimension coordinates: latitude x - longitude - x Attributes: CCCma_data_licence '1) GRANT OF LICENCE - The Government of Canada (Environment Canada) is ...' CCCma_parent_runid 'IGA' CCCma_runid 'IGY' Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_CanESM2_historical_r0i0p0.nc ...' branch_time 171915.0 branch_time_YMDH '2321:01:01:00' cmor_version '2.5.4' contact 'cccma_info@ec.gc.ca' experiment 'historical' experiment_id 'historical' forcing 'GHG,Oz,SA,BC,OC,LU,Sl,Vl (GHG includes CO2,CH4,N2O,CFC11,effective CFC ...' frequency 'fx' initialization_method 0 institute_id 'CCCma' institution 'CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, ...' model_id 'CanESM2' modeling_realm 'atmos' original_name 'DATA_DESCRIPTION' parent_experiment 'pre-industrial control' parent_experiment_id 'piControl' parent_experiment_rip 'r1i1p1' physics_version 0 product 'output' project_id 'CMIP5' realization 0 references 'http://www.cccma.ec.gc.ca/models' source 'CanESM2 2010 atmosphere: CanAM4 (AGCM15i, T63L35) ocean: CanOM4 (OGCM4.0, ...' source_file '/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical ...' table_id 'Table fx (28 March 2011) c046d0887aeb2bb7ed34d63897d7de8f' title 'CanESM2 model output prepared for CMIP5 historical' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/CMIP5_CanESM2_fx_historical_r0i0p0_sftlf/00_load.nc 2025-01-31 16:49:53,994 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step fix_metadata 2025-01-31 16:49:53,996 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'fix_metadata' on the data [] loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) check_level = , session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': 16, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.11.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': False, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS2980K-datalake/shared/external/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': True, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), reference_dataset = 'GPCP-V2.3', mip = 'fx', dataset = 'CanESM2', project = 'CMIP5', exp = 'historical', ensemble = 'r0i0p0', short_name = 'sftlf', institute = 'CCCma', product = ['output1', 'output2'], original_short_name = 'sftlf', standard_name = 'land_area_fraction', long_name = 'Land Area Fraction', units = '%', modeling_realm = ['atmos'], frequency = 'fx', version = 'v20120410' 2025-01-31 16:49:53,996 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:53,997 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: land_area_fraction / (%) (latitude: 64; longitude: 128) Dimension coordinates: latitude x - longitude - x Attributes: CCCma_data_licence '1) GRANT OF LICENCE - The Government of Canada (Environment Canada) is ...' CCCma_parent_runid 'IGA' CCCma_runid 'IGY' Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_CanESM2_historical_r0i0p0.nc ...' branch_time 171915.0 branch_time_YMDH '2321:01:01:00' cmor_version '2.5.4' contact 'cccma_info@ec.gc.ca' experiment 'historical' experiment_id 'historical' forcing 'GHG,Oz,SA,BC,OC,LU,Sl,Vl (GHG includes CO2,CH4,N2O,CFC11,effective CFC ...' frequency 'fx' initialization_method 0 institute_id 'CCCma' institution 'CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, ...' model_id 'CanESM2' modeling_realm 'atmos' original_name 'DATA_DESCRIPTION' parent_experiment 'pre-industrial control' parent_experiment_id 'piControl' parent_experiment_rip 'r1i1p1' physics_version 0 product 'output' project_id 'CMIP5' realization 0 references 'http://www.cccma.ec.gc.ca/models' source 'CanESM2 2010 atmosphere: CanAM4 (AGCM15i, T63L35) ocean: CanOM4 (OGCM4.0, ...' table_id 'Table fx (28 March 2011) c046d0887aeb2bb7ed34d63897d7de8f' title 'CanESM2 model output prepared for CMIP5 historical' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/CMIP5_CanESM2_fx_historical_r0i0p0_sftlf/01_fix_metadata.nc 2025-01-31 16:49:54,010 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step concatenate 2025-01-31 16:49:54,011 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'concatenate' on the data [] loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) check_level = 2025-01-31 16:49:54,011 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:54,011 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: land_area_fraction / (%) (latitude: 64; longitude: 128) Dimension coordinates: latitude x - longitude - x Attributes: CCCma_data_licence '1) GRANT OF LICENCE - The Government of Canada (Environment Canada) is ...' CCCma_parent_runid 'IGA' CCCma_runid 'IGY' Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_CanESM2_historical_r0i0p0.nc ...' branch_time 171915.0 branch_time_YMDH '2321:01:01:00' cmor_version '2.5.4' contact 'cccma_info@ec.gc.ca' experiment 'historical' experiment_id 'historical' forcing 'GHG,Oz,SA,BC,OC,LU,Sl,Vl (GHG includes CO2,CH4,N2O,CFC11,effective CFC ...' frequency 'fx' initialization_method 0 institute_id 'CCCma' institution 'CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, ...' model_id 'CanESM2' modeling_realm 'atmos' original_name 'DATA_DESCRIPTION' parent_experiment 'pre-industrial control' parent_experiment_id 'piControl' parent_experiment_rip 'r1i1p1' physics_version 0 product 'output' project_id 'CMIP5' realization 0 references 'http://www.cccma.ec.gc.ca/models' source 'CanESM2 2010 atmosphere: CanAM4 (AGCM15i, T63L35) ocean: CanOM4 (OGCM4.0, ...' table_id 'Table fx (28 March 2011) c046d0887aeb2bb7ed34d63897d7de8f' title 'CanESM2 model output prepared for CMIP5 historical' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/CMIP5_CanESM2_fx_historical_r0i0p0_sftlf/02_concatenate.nc 2025-01-31 16:49:54,025 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step cmor_check_metadata 2025-01-31 16:49:54,026 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'cmor_check_metadata' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) check_level = , cmor_table = 'CMIP5', mip = 'fx', frequency = 'fx', short_name = 'sftlf' 2025-01-31 16:49:54,026 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:54,027 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: land_area_fraction / (%) (latitude: 64; longitude: 128) Dimension coordinates: latitude x - longitude - x Attributes: CCCma_data_licence '1) GRANT OF LICENCE - The Government of Canada (Environment Canada) is ...' CCCma_parent_runid 'IGA' CCCma_runid 'IGY' Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_CanESM2_historical_r0i0p0.nc ...' branch_time 171915.0 branch_time_YMDH '2321:01:01:00' cmor_version '2.5.4' contact 'cccma_info@ec.gc.ca' experiment 'historical' experiment_id 'historical' forcing 'GHG,Oz,SA,BC,OC,LU,Sl,Vl (GHG includes CO2,CH4,N2O,CFC11,effective CFC ...' frequency 'fx' initialization_method 0 institute_id 'CCCma' institution 'CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, ...' model_id 'CanESM2' modeling_realm 'atmos' original_name 'DATA_DESCRIPTION' parent_experiment 'pre-industrial control' parent_experiment_id 'piControl' parent_experiment_rip 'r1i1p1' physics_version 0 product 'output' project_id 'CMIP5' realization 0 references 'http://www.cccma.ec.gc.ca/models' source 'CanESM2 2010 atmosphere: CanAM4 (AGCM15i, T63L35) ocean: CanOM4 (OGCM4.0, ...' table_id 'Table fx (28 March 2011) c046d0887aeb2bb7ed34d63897d7de8f' title 'CanESM2 model output prepared for CMIP5 historical' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/CMIP5_CanESM2_fx_historical_r0i0p0_sftlf/03_cmor_check_metadata.nc 2025-01-31 16:49:54,056 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step fix_data 2025-01-31 16:49:54,057 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'fix_data' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) check_level = , session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': 16, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.11.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': False, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS2980K-datalake/shared/external/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': True, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), reference_dataset = 'GPCP-V2.3', mip = 'fx', dataset = 'CanESM2', project = 'CMIP5', exp = 'historical', ensemble = 'r0i0p0', short_name = 'sftlf', institute = 'CCCma', product = ['output1', 'output2'], original_short_name = 'sftlf', standard_name = 'land_area_fraction', long_name = 'Land Area Fraction', units = '%', modeling_realm = ['atmos'], frequency = 'fx', version = 'v20120410' 2025-01-31 16:49:54,057 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:54,058 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: land_area_fraction / (%) (latitude: 64; longitude: 128) Dimension coordinates: latitude x - longitude - x Attributes: CCCma_data_licence '1) GRANT OF LICENCE - The Government of Canada (Environment Canada) is ...' CCCma_parent_runid 'IGA' CCCma_runid 'IGY' Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_CanESM2_historical_r0i0p0.nc ...' branch_time 171915.0 branch_time_YMDH '2321:01:01:00' cmor_version '2.5.4' contact 'cccma_info@ec.gc.ca' experiment 'historical' experiment_id 'historical' forcing 'GHG,Oz,SA,BC,OC,LU,Sl,Vl (GHG includes CO2,CH4,N2O,CFC11,effective CFC ...' frequency 'fx' initialization_method 0 institute_id 'CCCma' institution 'CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, ...' model_id 'CanESM2' modeling_realm 'atmos' original_name 'DATA_DESCRIPTION' parent_experiment 'pre-industrial control' parent_experiment_id 'piControl' parent_experiment_rip 'r1i1p1' physics_version 0 product 'output' project_id 'CMIP5' realization 0 references 'http://www.cccma.ec.gc.ca/models' source 'CanESM2 2010 atmosphere: CanAM4 (AGCM15i, T63L35) ocean: CanOM4 (OGCM4.0, ...' table_id 'Table fx (28 March 2011) c046d0887aeb2bb7ed34d63897d7de8f' title 'CanESM2 model output prepared for CMIP5 historical' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/CMIP5_CanESM2_fx_historical_r0i0p0_sftlf/04_fix_data.nc 2025-01-31 16:49:54,072 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step cmor_check_data 2025-01-31 16:49:54,073 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'cmor_check_data' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) check_level = , cmor_table = 'CMIP5', mip = 'fx', frequency = 'fx', short_name = 'sftlf' 2025-01-31 16:49:54,073 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:54,073 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: land_area_fraction / (%) (latitude: 64; longitude: 128) Dimension coordinates: latitude x - longitude - x Attributes: CCCma_data_licence '1) GRANT OF LICENCE - The Government of Canada (Environment Canada) is ...' CCCma_parent_runid 'IGA' CCCma_runid 'IGY' Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_CanESM2_historical_r0i0p0.nc ...' branch_time 171915.0 branch_time_YMDH '2321:01:01:00' cmor_version '2.5.4' contact 'cccma_info@ec.gc.ca' experiment 'historical' experiment_id 'historical' forcing 'GHG,Oz,SA,BC,OC,LU,Sl,Vl (GHG includes CO2,CH4,N2O,CFC11,effective CFC ...' frequency 'fx' initialization_method 0 institute_id 'CCCma' institution 'CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, ...' model_id 'CanESM2' modeling_realm 'atmos' original_name 'DATA_DESCRIPTION' parent_experiment 'pre-industrial control' parent_experiment_id 'piControl' parent_experiment_rip 'r1i1p1' physics_version 0 product 'output' project_id 'CMIP5' realization 0 references 'http://www.cccma.ec.gc.ca/models' source 'CanESM2 2010 atmosphere: CanAM4 (AGCM15i, T63L35) ocean: CanOM4 (OGCM4.0, ...' table_id 'Table fx (28 March 2011) c046d0887aeb2bb7ed34d63897d7de8f' title 'CanESM2 model output prepared for CMIP5 historical' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/CMIP5_CanESM2_fx_historical_r0i0p0_sftlf/05_cmor_check_data.nc 2025-01-31 16:49:54,088 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step add_supplementary_variables 2025-01-31 16:49:54,089 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'add_supplementary_variables' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc'), LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) supplementary_cubes = [] 2025-01-31 16:49:54,089 UTC [2300145] DEBUG esmvalcore.preprocessor._supplementary_vars:104 Added sftlf as ancillary variable in cube of pr. 2025-01-31 16:49:54,089 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:54,090 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 324; latitude: 64; longitude: 128) Dimension coordinates: time x - - latitude - x - longitude - - x Ancillary variables: land_area_fraction - x x Cell methods: 0 time: mean (interval: 15 minutes) Attributes: CCCma_data_licence '1) GRANT OF LICENCE - The Government of Canada (Environment Canada) is ...' CCCma_parent_runid 'IGA' CCCma_runid 'IGM' Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_CanESM2_historical_r0i0p0.nc ...' branch_time 171915.0 branch_time_YMDH '2321:01:01:00' cmor_version '2.5.4' contact 'cccma_info@ec.gc.ca' experiment 'historical' experiment_id 'historical' forcing 'GHG,Oz,SA,BC,OC,LU,Sl,Vl (GHG includes CO2,CH4,N2O,CFC11,effective CFC ...' frequency 'mon' initialization_method 1 institute_id 'CCCma' institution 'CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, ...' model_id 'CanESM2' modeling_realm 'atmos' original_name 'PCP' parent_experiment 'pre-industrial control' parent_experiment_id 'piControl' parent_experiment_rip 'r1i1p1' physics_version 1 product 'output' project_id 'CMIP5' realization 1 references 'http://www.cccma.ec.gc.ca/models' source 'CanESM2 2010 atmosphere: CanAM4 (AGCM15i, T63L35) ocean: CanOM4 (OGCM4.0, ...' table_id 'Table Amon (31 January 2011) 53b766a395ac41696af40aab76a49ae5' title 'CanESM2 model output prepared for CMIP5 historical' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005/07_add_supplementary_variables.nc 2025-01-31 16:49:54,181 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step mask_landsea 2025-01-31 16:49:54,182 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'mask_landsea' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc'), LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) mask_out = 'sea' 2025-01-31 16:49:54,189 UTC [2300145] DEBUG esmvalcore.preprocessor._mask:151 Applying land-sea mask: sftlf 2025-01-31 16:49:54,189 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:54,189 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 324; latitude: 64; longitude: 128) Dimension coordinates: time x - - latitude - x - longitude - - x Ancillary variables: land_area_fraction - x x Cell methods: 0 time: mean (interval: 15 minutes) Attributes: CCCma_data_licence '1) GRANT OF LICENCE - The Government of Canada (Environment Canada) is ...' CCCma_parent_runid 'IGA' CCCma_runid 'IGM' Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_CanESM2_historical_r0i0p0.nc ...' branch_time 171915.0 branch_time_YMDH '2321:01:01:00' cmor_version '2.5.4' contact 'cccma_info@ec.gc.ca' experiment 'historical' experiment_id 'historical' forcing 'GHG,Oz,SA,BC,OC,LU,Sl,Vl (GHG includes CO2,CH4,N2O,CFC11,effective CFC ...' frequency 'mon' initialization_method 1 institute_id 'CCCma' institution 'CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, ...' model_id 'CanESM2' modeling_realm 'atmos' original_name 'PCP' parent_experiment 'pre-industrial control' parent_experiment_id 'piControl' parent_experiment_rip 'r1i1p1' physics_version 1 product 'output' project_id 'CMIP5' realization 1 references 'http://www.cccma.ec.gc.ca/models' source 'CanESM2 2010 atmosphere: CanAM4 (AGCM15i, T63L35) ocean: CanOM4 (OGCM4.0, ...' table_id 'Table Amon (31 January 2011) 53b766a395ac41696af40aab76a49ae5' title 'CanESM2 model output prepared for CMIP5 historical' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005/08_mask_landsea.nc 2025-01-31 16:49:54,285 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step regrid 2025-01-31 16:49:54,286 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'regrid' on the data loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc'), LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) target_grid = '2x2', scheme = 'area_weighted' 2025-01-31 16:49:54,287 UTC [2300145] DEBUG esmvalcore.preprocessor._regrid:607 Loaded regridding scheme AreaWeighted(mdtol=1) 2025-01-31 16:49:54,293 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:54,293 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 324; latitude: 90; longitude: 180) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean (interval: 15 minutes) Attributes: CCCma_data_licence '1) GRANT OF LICENCE - The Government of Canada (Environment Canada) is ...' CCCma_parent_runid 'IGA' CCCma_runid 'IGM' Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_CanESM2_historical_r0i0p0.nc ...' branch_time 171915.0 branch_time_YMDH '2321:01:01:00' cmor_version '2.5.4' contact 'cccma_info@ec.gc.ca' experiment 'historical' experiment_id 'historical' forcing 'GHG,Oz,SA,BC,OC,LU,Sl,Vl (GHG includes CO2,CH4,N2O,CFC11,effective CFC ...' frequency 'mon' initialization_method 1 institute_id 'CCCma' institution 'CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, ...' model_id 'CanESM2' modeling_realm 'atmos' original_name 'PCP' parent_experiment 'pre-industrial control' parent_experiment_id 'piControl' parent_experiment_rip 'r1i1p1' physics_version 1 product 'output' project_id 'CMIP5' realization 1 references 'http://www.cccma.ec.gc.ca/models' source 'CanESM2 2010 atmosphere: CanAM4 (AGCM15i, T63L35) ocean: CanOM4 (OGCM4.0, ...' table_id 'Table Amon (31 January 2011) 53b766a395ac41696af40aab76a49ae5' title 'CanESM2 model output prepared for CMIP5 historical' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005/09_regrid.nc 2025-01-31 16:49:54,645 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step save 2025-01-31 16:49:54,647 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'save' on the data [] loaded from original input file(s) [LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/mon/atmos/Amon/r1i1p1/v20120718/pr/pr_Amon_CanESM2_historical_r1i1p1_185001-200512.nc'), LocalFile('/projects/NS9560K-datalake/ESGF/cmip5/output1/CCCma/CanESM2/historical/fx/atmos/fx/r0i0p0/v20120410/sftlf/sftlf_fx_CanESM2_historical_r0i0p0.nc')] with function argument(s) compress = False, filename = PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005.nc') 2025-01-31 16:49:54,647 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 324; latitude: 90; longitude: 180) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean (interval: 15 minutes) Attributes: CCCma_data_licence '1) GRANT OF LICENCE - The Government of Canada (Environment Canada) is ...' CCCma_parent_runid 'IGA' CCCma_runid 'IGM' Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_CanESM2_historical_r0i0p0.nc ...' branch_time 171915.0 branch_time_YMDH '2321:01:01:00' cmor_version '2.5.4' contact 'cccma_info@ec.gc.ca' experiment 'historical' experiment_id 'historical' forcing 'GHG,Oz,SA,BC,OC,LU,Sl,Vl (GHG includes CO2,CH4,N2O,CFC11,effective CFC ...' frequency 'mon' initialization_method 1 institute_id 'CCCma' institution 'CCCma (Canadian Centre for Climate Modelling and Analysis, Victoria, BC, ...' model_id 'CanESM2' modeling_realm 'atmos' original_name 'PCP' parent_experiment 'pre-industrial control' parent_experiment_id 'piControl' parent_experiment_rip 'r1i1p1' physics_version 1 product 'output' project_id 'CMIP5' realization 1 references 'http://www.cccma.ec.gc.ca/models' source 'CanESM2 2010 atmosphere: CanAM4 (AGCM15i, T63L35) ocean: CanOM4 (OGCM4.0, ...' table_id 'Table Amon (31 January 2011) 53b766a395ac41696af40aab76a49ae5' title 'CanESM2 model output prepared for CMIP5 historical' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/CMIP5_CanESM2_Amon_historical_r1i1p1_pr_1979-2005.nc 2025-01-31 16:49:54,991 UTC [2300145] DEBUG esmvalcore.preprocessor:681 Applying single-model steps to PreprocessorFile: /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005.nc 2025-01-31 16:49:54,991 UTC [2300145] DEBUG esmvalcore.esgf._download:538 All required data is available locally, not downloading anything. 2025-01-31 16:49:54,991 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step fix_file 2025-01-31 16:49:54,992 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'fix_file' on the data LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc') loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) output_dir = PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/fixed_files/pr_Amon_obs4MIPs_GPCP-V2.3_v20180519_'), add_unique_suffix = True, session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': 16, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.11.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': False, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS2980K-datalake/shared/external/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': True, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), preprocessor = 'mask_regrid_preproc', reference_dataset = 'GPCP-V2.3', mip = 'Amon', dataset = 'GPCP-V2.3', project = 'obs4MIPs', level = 'L3', tier = 1, short_name = 'pr', timerange = '1979/2005', variable_group = 'pr', diagnostic = 'quantilebias', recipe_dataset_index = 1, alias = 'obs4MIPs', original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = [], frequency = 'mon', version = 'v20180519' 2025-01-31 16:49:54,992 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] 2025-01-31 16:49:54,993 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step load 2025-01-31 16:49:54,993 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'load' on the data LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc') loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) ignore_warnings = None 2025-01-31 16:49:54,993 UTC [2300145] DEBUG esmvalcore.preprocessor._io:98 Loading: /scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc 2025-01-31 16:49:55,008 UTC [2300145] DEBUG esmvalcore.preprocessor._io:134 Done with loading /scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc 2025-01-31 16:49:55,008 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:55,009 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 466; latitude: 72; longitude: 144) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean Attributes: Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_Obs-GPCP_GPCP_r0i0p0.nc ...' cmor_version '2.9.1' contact 'George Huffman (george.j.huffman@nasa.gov)' frequency 'mon' institute_id 'NASA-GSFC' institution 'NASA Goddard Space Flight Center, Greenbelt MD, USA' mip_specs 'CMIP5' model_id 'Obs-GPCP' modeling_realm 'atmos' original_name 'precip' product 'observations' project_id 'obs4MIPs' realm 'atmos' references 'Huffman et al. 1997, http://dx.doi.org/10.1175/1520-0477(1997)078<0005:TGPCPG>2.0.CO;2; ...' source 'Obs-GPCP (Global Precipitation Climatology Project) v23rB1' source_file '/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_1 ...' source_id 'GPCP' source_type 'satellite_retrieval_and_gauge_analysis' table_id 'Table Amon_ana (10 March 2011) 34230b4cbd7bedf38c827d6e41c1b8ea' title 'Global Precipitation Climatology Project (GPCP) Climate Data Record (CDR), ...' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005/00_load.nc 2025-01-31 16:49:55,059 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step fix_metadata 2025-01-31 16:49:55,060 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'fix_metadata' on the data [] loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) check_level = , session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': 16, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.11.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': False, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS2980K-datalake/shared/external/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': True, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), preprocessor = 'mask_regrid_preproc', reference_dataset = 'GPCP-V2.3', mip = 'Amon', dataset = 'GPCP-V2.3', project = 'obs4MIPs', level = 'L3', tier = 1, short_name = 'pr', timerange = '1979/2005', variable_group = 'pr', diagnostic = 'quantilebias', recipe_dataset_index = 1, alias = 'obs4MIPs', original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = [], frequency = 'mon', version = 'v20180519' 2025-01-31 16:49:55,063 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:55,063 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 466; latitude: 72; longitude: 144) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean Attributes: Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_Obs-GPCP_GPCP_r0i0p0.nc ...' cmor_version '2.9.1' contact 'George Huffman (george.j.huffman@nasa.gov)' frequency 'mon' institute_id 'NASA-GSFC' institution 'NASA Goddard Space Flight Center, Greenbelt MD, USA' mip_specs 'CMIP5' model_id 'Obs-GPCP' modeling_realm 'atmos' original_name 'precip' product 'observations' project_id 'obs4MIPs' realm 'atmos' references 'Huffman et al. 1997, http://dx.doi.org/10.1175/1520-0477(1997)078<0005:TGPCPG>2.0.CO;2; ...' source 'Obs-GPCP (Global Precipitation Climatology Project) v23rB1' source_id 'GPCP' source_type 'satellite_retrieval_and_gauge_analysis' table_id 'Table Amon_ana (10 March 2011) 34230b4cbd7bedf38c827d6e41c1b8ea' title 'Global Precipitation Climatology Project (GPCP) Climate Data Record (CDR), ...' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005/01_fix_metadata.nc 2025-01-31 16:49:55,111 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step concatenate 2025-01-31 16:49:55,112 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'concatenate' on the data [] loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) check_level = 2025-01-31 16:49:55,112 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:55,112 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 466; latitude: 72; longitude: 144) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean Attributes: Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_Obs-GPCP_GPCP_r0i0p0.nc ...' cmor_version '2.9.1' contact 'George Huffman (george.j.huffman@nasa.gov)' frequency 'mon' institute_id 'NASA-GSFC' institution 'NASA Goddard Space Flight Center, Greenbelt MD, USA' mip_specs 'CMIP5' model_id 'Obs-GPCP' modeling_realm 'atmos' original_name 'precip' product 'observations' project_id 'obs4MIPs' realm 'atmos' references 'Huffman et al. 1997, http://dx.doi.org/10.1175/1520-0477(1997)078<0005:TGPCPG>2.0.CO;2; ...' source 'Obs-GPCP (Global Precipitation Climatology Project) v23rB1' source_id 'GPCP' source_type 'satellite_retrieval_and_gauge_analysis' table_id 'Table Amon_ana (10 March 2011) 34230b4cbd7bedf38c827d6e41c1b8ea' title 'Global Precipitation Climatology Project (GPCP) Climate Data Record (CDR), ...' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005/02_concatenate.nc 2025-01-31 16:49:55,161 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step cmor_check_metadata 2025-01-31 16:49:55,162 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'cmor_check_metadata' on the data loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) check_level = , cmor_table = 'obs4MIPs', mip = 'Amon', frequency = 'mon', short_name = 'pr' 2025-01-31 16:49:55,163 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:55,164 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 466; latitude: 72; longitude: 144) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean Attributes: Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_Obs-GPCP_GPCP_r0i0p0.nc ...' cmor_version '2.9.1' contact 'George Huffman (george.j.huffman@nasa.gov)' frequency 'mon' institute_id 'NASA-GSFC' institution 'NASA Goddard Space Flight Center, Greenbelt MD, USA' mip_specs 'CMIP5' model_id 'Obs-GPCP' modeling_realm 'atmos' original_name 'precip' product 'observations' project_id 'obs4MIPs' realm 'atmos' references 'Huffman et al. 1997, http://dx.doi.org/10.1175/1520-0477(1997)078<0005:TGPCPG>2.0.CO;2; ...' source 'Obs-GPCP (Global Precipitation Climatology Project) v23rB1' source_id 'GPCP' source_type 'satellite_retrieval_and_gauge_analysis' table_id 'Table Amon_ana (10 March 2011) 34230b4cbd7bedf38c827d6e41c1b8ea' title 'Global Precipitation Climatology Project (GPCP) Climate Data Record (CDR), ...' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005/03_cmor_check_metadata.nc 2025-01-31 16:49:55,211 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step clip_timerange 2025-01-31 16:49:55,212 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'clip_timerange' on the data loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) timerange = '1979/2005' 2025-01-31 16:49:55,216 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:55,216 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 324; latitude: 72; longitude: 144) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean Attributes: Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_Obs-GPCP_GPCP_r0i0p0.nc ...' cmor_version '2.9.1' contact 'George Huffman (george.j.huffman@nasa.gov)' frequency 'mon' institute_id 'NASA-GSFC' institution 'NASA Goddard Space Flight Center, Greenbelt MD, USA' mip_specs 'CMIP5' model_id 'Obs-GPCP' modeling_realm 'atmos' original_name 'precip' product 'observations' project_id 'obs4MIPs' realm 'atmos' references 'Huffman et al. 1997, http://dx.doi.org/10.1175/1520-0477(1997)078<0005:TGPCPG>2.0.CO;2; ...' source 'Obs-GPCP (Global Precipitation Climatology Project) v23rB1' source_id 'GPCP' source_type 'satellite_retrieval_and_gauge_analysis' table_id 'Table Amon_ana (10 March 2011) 34230b4cbd7bedf38c827d6e41c1b8ea' title 'Global Precipitation Climatology Project (GPCP) Climate Data Record (CDR), ...' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005/04_clip_timerange.nc 2025-01-31 16:49:55,246 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step fix_data 2025-01-31 16:49:55,248 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'fix_data' on the data loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) check_level = , session = Session({'auxiliary_data_dir': PosixPath('/projects/NS9560K-datalake/ESGF/auxiliary_data'), 'check_level': , 'compress_netcdf': False, 'config_developer_file': PosixPath('/projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvalcore/config-developer.yml'), 'config_file': PosixPath('/nird/home/tomast/.esmvaltool/config-user.yml'), 'diagnostics': None, 'download_dir': PosixPath('/scratch/tomast/ESGF'), 'drs': {'CMIP3': 'DKRZ', 'CMIP5': 'DKRZ', 'CMIP6': 'DKRZ', 'CORDEX': 'DKRZ', 'OBS': 'default', 'OBS6': 'default', 'ana4mips': 'default', 'native6': 'default', 'obs4MIPs': 'default'}, 'exit_on_warning': False, 'extra_facets_dir': (), 'log_level': 'info', 'max_datasets': None, 'max_parallel_tasks': 16, 'max_years': None, 'output_dir': PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.11.0/output'), 'output_file_type': 'png', 'profile_diagnostic': False, 'remove_preproc_dir': False, 'resume_from': [], 'rootpath': {'CMIP3': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip3')], 'CMIP5': [PosixPath('/projects/NS9560K-datalake/ESGF/cmip5/output1'), PosixPath('/projects/NS9034K/CMIP5/output1'), PosixPath('/scratch/tomast/ESGF/cmip5/output1')], 'CMIP6': [PosixPath('/projects/NS9560K-datalake/ESGF/CMIP6'), PosixPath('/projects/NS9034K/CMIP6'), PosixPath('/projects/NS2980K-datalake/shared/external/CMIP6'), PosixPath('/projects/NS9252K/KeyCLIM_CMOR'), PosixPath('/scratch/tomast/ESGF/CMIP6')], 'OBS': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'OBS6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obsdata')], 'RAWOBS': [PosixPath('/projects/NS9560K-datalake/ESGF/rawdata/obs'), PosixPath('/scratch/tomast/ESGF/rawdata/obs')], 'ana4mips': [PosixPath('/projects/NS9560K-datalake/ESGF/ana4mips'), PosixPath('/scratch/tomast/ESGF/ana4mips')], 'default': [PosixPath('/scratch/tomast/ESGF/obsdata')], 'native6': [PosixPath('/projects/NS9560K-datalake/ESGF/obsdata/native6'), PosixPath('/scratch/tomast/ESGF/obsdata/native6')], 'obs4MIPs': [PosixPath('/projects/NS9560K-datalake/ESGF/obs4MIPs'), PosixPath('/projects/NS9560K-datalake/ESGF/obsdata'), PosixPath('/scratch/tomast/ESGF/obs4MIPs')]}, 'run_diagnostic': True, 'save_intermediary_cubes': True, 'search_esgf': 'when_missing', 'skip_nonexistent': False, 'write_ncl_interface': False}), preprocessor = 'mask_regrid_preproc', reference_dataset = 'GPCP-V2.3', mip = 'Amon', dataset = 'GPCP-V2.3', project = 'obs4MIPs', level = 'L3', tier = 1, short_name = 'pr', timerange = '1979/2005', variable_group = 'pr', diagnostic = 'quantilebias', recipe_dataset_index = 1, alias = 'obs4MIPs', original_short_name = 'pr', standard_name = 'precipitation_flux', long_name = 'Precipitation', units = 'kg m-2 s-1', modeling_realm = [], frequency = 'mon', version = 'v20180519' 2025-01-31 16:49:55,248 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:55,248 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 324; latitude: 72; longitude: 144) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean Attributes: Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_Obs-GPCP_GPCP_r0i0p0.nc ...' cmor_version '2.9.1' contact 'George Huffman (george.j.huffman@nasa.gov)' frequency 'mon' institute_id 'NASA-GSFC' institution 'NASA Goddard Space Flight Center, Greenbelt MD, USA' mip_specs 'CMIP5' model_id 'Obs-GPCP' modeling_realm 'atmos' original_name 'precip' product 'observations' project_id 'obs4MIPs' realm 'atmos' references 'Huffman et al. 1997, http://dx.doi.org/10.1175/1520-0477(1997)078<0005:TGPCPG>2.0.CO;2; ...' source 'Obs-GPCP (Global Precipitation Climatology Project) v23rB1' source_id 'GPCP' source_type 'satellite_retrieval_and_gauge_analysis' table_id 'Table Amon_ana (10 March 2011) 34230b4cbd7bedf38c827d6e41c1b8ea' title 'Global Precipitation Climatology Project (GPCP) Climate Data Record (CDR), ...' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005/05_fix_data.nc 2025-01-31 16:49:55,286 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step cmor_check_data 2025-01-31 16:49:55,287 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'cmor_check_data' on the data loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) check_level = , cmor_table = 'obs4MIPs', mip = 'Amon', frequency = 'mon', short_name = 'pr' 2025-01-31 16:49:55,288 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:55,288 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 324; latitude: 72; longitude: 144) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean Attributes: Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_Obs-GPCP_GPCP_r0i0p0.nc ...' cmor_version '2.9.1' contact 'George Huffman (george.j.huffman@nasa.gov)' frequency 'mon' institute_id 'NASA-GSFC' institution 'NASA Goddard Space Flight Center, Greenbelt MD, USA' mip_specs 'CMIP5' model_id 'Obs-GPCP' modeling_realm 'atmos' original_name 'precip' product 'observations' project_id 'obs4MIPs' realm 'atmos' references 'Huffman et al. 1997, http://dx.doi.org/10.1175/1520-0477(1997)078<0005:TGPCPG>2.0.CO;2; ...' source 'Obs-GPCP (Global Precipitation Climatology Project) v23rB1' source_id 'GPCP' source_type 'satellite_retrieval_and_gauge_analysis' table_id 'Table Amon_ana (10 March 2011) 34230b4cbd7bedf38c827d6e41c1b8ea' title 'Global Precipitation Climatology Project (GPCP) Climate Data Record (CDR), ...' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005/06_cmor_check_data.nc 2025-01-31 16:49:55,324 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step add_supplementary_variables 2025-01-31 16:49:55,324 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'add_supplementary_variables' on the data loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) supplementary_cubes = [] 2025-01-31 16:49:55,324 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:55,325 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 324; latitude: 72; longitude: 144) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean Attributes: Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_Obs-GPCP_GPCP_r0i0p0.nc ...' cmor_version '2.9.1' contact 'George Huffman (george.j.huffman@nasa.gov)' frequency 'mon' institute_id 'NASA-GSFC' institution 'NASA Goddard Space Flight Center, Greenbelt MD, USA' mip_specs 'CMIP5' model_id 'Obs-GPCP' modeling_realm 'atmos' original_name 'precip' product 'observations' project_id 'obs4MIPs' realm 'atmos' references 'Huffman et al. 1997, http://dx.doi.org/10.1175/1520-0477(1997)078<0005:TGPCPG>2.0.CO;2; ...' source 'Obs-GPCP (Global Precipitation Climatology Project) v23rB1' source_id 'GPCP' source_type 'satellite_retrieval_and_gauge_analysis' table_id 'Table Amon_ana (10 March 2011) 34230b4cbd7bedf38c827d6e41c1b8ea' title 'Global Precipitation Climatology Project (GPCP) Climate Data Record (CDR), ...' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005/07_add_supplementary_variables.nc 2025-01-31 16:49:55,361 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step mask_landsea 2025-01-31 16:49:55,362 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'mask_landsea' on the data loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) mask_out = 'sea' 2025-01-31 16:49:55,362 UTC [2300145] DEBUG esmvalcore.preprocessor._mask:137 Ancillary variables land/sea area fraction not found in cube. Check fx_file availability. 2025-01-31 16:49:55,363 UTC [2300145] DEBUG fiona._env:315 GDAL_DATA found in environment. 2025-01-31 16:49:55,364 UTC [2300145] DEBUG fiona._env:315 PROJ_DATA found in environment. 2025-01-31 16:49:55,383 UTC [2300145] DEBUG fiona._env:315 GDAL_DATA found in environment. 2025-01-31 16:49:55,383 UTC [2300145] DEBUG fiona._env:315 PROJ_DATA found in environment. 2025-01-31 16:49:55,384 UTC [2300145] DEBUG fiona.ogrext:436 OLC_FASTSETNEXTBYINDEX: 1 2025-01-31 16:49:55,384 UTC [2300145] DEBUG fiona.ogrext:436 OLC_FASTFEATURECOUNT: 1 2025-01-31 16:49:55,384 UTC [2300145] DEBUG fiona.ogrext:436 Next index: 0 2025-01-31 16:49:55,384 UTC [2300145] DEBUG fiona.ogrext:217 Next index: 1 2025-01-31 16:49:55,495 UTC [2300145] DEBUG fiona.collection:660 Flushed buffer 2025-01-31 16:49:55,495 UTC [2300145] DEBUG fiona.collection:662 Stopped session 2025-01-31 16:49:55,507 UTC [2300145] DEBUG esmvalcore.preprocessor._mask:155 Applying land-sea mask from Natural Earth shapefile: /projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvalcore/preprocessor/ne_masks/ne_50m_ocean.shp 2025-01-31 16:49:55,507 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:55,508 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 324; latitude: 72; longitude: 144) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean Attributes: Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_Obs-GPCP_GPCP_r0i0p0.nc ...' cmor_version '2.9.1' contact 'George Huffman (george.j.huffman@nasa.gov)' frequency 'mon' institute_id 'NASA-GSFC' institution 'NASA Goddard Space Flight Center, Greenbelt MD, USA' mip_specs 'CMIP5' model_id 'Obs-GPCP' modeling_realm 'atmos' original_name 'precip' product 'observations' project_id 'obs4MIPs' realm 'atmos' references 'Huffman et al. 1997, http://dx.doi.org/10.1175/1520-0477(1997)078<0005:TGPCPG>2.0.CO;2; ...' source 'Obs-GPCP (Global Precipitation Climatology Project) v23rB1' source_id 'GPCP' source_type 'satellite_retrieval_and_gauge_analysis' table_id 'Table Amon_ana (10 March 2011) 34230b4cbd7bedf38c827d6e41c1b8ea' title 'Global Precipitation Climatology Project (GPCP) Climate Data Record (CDR), ...' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005/08_mask_landsea.nc 2025-01-31 16:49:55,583 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step regrid 2025-01-31 16:49:55,584 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'regrid' on the data loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) target_grid = '2x2', scheme = 'area_weighted' 2025-01-31 16:49:55,584 UTC [2300145] DEBUG esmvalcore.preprocessor._regrid:607 Loaded regridding scheme AreaWeighted(mdtol=1) 2025-01-31 16:49:55,588 UTC [2300145] DEBUG esmvalcore.preprocessor:412 Result [] 2025-01-31 16:49:55,589 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 324; latitude: 90; longitude: 180) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean Attributes: Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_Obs-GPCP_GPCP_r0i0p0.nc ...' cmor_version '2.9.1' contact 'George Huffman (george.j.huffman@nasa.gov)' frequency 'mon' institute_id 'NASA-GSFC' institution 'NASA Goddard Space Flight Center, Greenbelt MD, USA' mip_specs 'CMIP5' model_id 'Obs-GPCP' modeling_realm 'atmos' original_name 'precip' product 'observations' project_id 'obs4MIPs' realm 'atmos' references 'Huffman et al. 1997, http://dx.doi.org/10.1175/1520-0477(1997)078<0005:TGPCPG>2.0.CO;2; ...' source 'Obs-GPCP (Global Precipitation Climatology Project) v23rB1' source_id 'GPCP' source_type 'satellite_retrieval_and_gauge_analysis' table_id 'Table Amon_ana (10 March 2011) 34230b4cbd7bedf38c827d6e41c1b8ea' title 'Global Precipitation Climatology Project (GPCP) Climate Data Record (CDR), ...' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005/09_regrid.nc 2025-01-31 16:49:55,821 UTC [2300145] DEBUG esmvalcore.preprocessor:387 Running preprocessor step save 2025-01-31 16:49:55,823 UTC [2300145] DEBUG esmvalcore.preprocessor:341 Running preprocessor function 'save' on the data [] loaded from original input file(s) [LocalFile('/scratch/tomast/ESGF/obs4MIPs/GPCP-V2.3/v20180519/pr_GPCP-SG_L3_v2.3_197901-201710.nc')] with function argument(s) compress = False, filename = PosixPath('/projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005.nc') 2025-01-31 16:49:55,823 UTC [2300145] DEBUG esmvalcore.preprocessor._io:408 Saving cube: precipitation_flux / (kg m-2 s-1) (time: 324; latitude: 90; longitude: 180) Dimension coordinates: time x - - latitude - x - longitude - - x Cell methods: 0 time: mean Attributes: Conventions 'CF-1.4' associated_files 'baseURL: http://cmip-pcmdi.llnl.gov/CMIP5/dataLocation gridspecFile: gridspec_atmos_fx_Obs-GPCP_GPCP_r0i0p0.nc ...' cmor_version '2.9.1' contact 'George Huffman (george.j.huffman@nasa.gov)' frequency 'mon' institute_id 'NASA-GSFC' institution 'NASA Goddard Space Flight Center, Greenbelt MD, USA' mip_specs 'CMIP5' model_id 'Obs-GPCP' modeling_realm 'atmos' original_name 'precip' product 'observations' project_id 'obs4MIPs' realm 'atmos' references 'Huffman et al. 1997, http://dx.doi.org/10.1175/1520-0477(1997)078<0005:TGPCPG>2.0.CO;2; ...' source 'Obs-GPCP (Global Precipitation Climatology Project) v23rB1' source_id 'GPCP' source_type 'satellite_retrieval_and_gauge_analysis' table_id 'Table Amon_ana (10 March 2011) 34230b4cbd7bedf38c827d6e41c1b8ea' title 'Global Precipitation Climatology Project (GPCP) Climate Data Record (CDR), ...' with lazy data to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/preproc/quantilebias/pr/obs4MIPs_GPCP-V2.3_pr_1979-2005.nc 2025-01-31 16:49:56,040 UTC [2300145] INFO esmvalcore._task:266 Successfully completed task quantilebias/pr (priority 1) in 0:00:03.350954 2025-01-31 16:49:56,099 UTC [2299982] INFO esmvalcore._task:793 Progress: 0 tasks running, 1 tasks waiting for ancestors, 1/2 done 2025-01-31 16:49:56,103 UTC [2300146] INFO esmvalcore._task:261 Starting task quantilebias/main in process [2300146] 2025-01-31 16:49:56,106 UTC [2300146] INFO esmvalcore._task:502 Running command ['/projects/NS2980K/conda/esmvaltool2.11.0/bin/Rscript', '/projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvaltool/diag_scripts/quantilebias/quantilebias.R', '/projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/run/quantilebias/main/settings.yml'] 2025-01-31 16:49:56,106 UTC [2300146] DEBUG esmvalcore._task:503 in environment {'diag_scripts': '/projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvaltool/diag_scripts'} 2025-01-31 16:49:56,106 UTC [2300146] DEBUG esmvalcore._task:505 in current working directory: /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/run/quantilebias/main 2025-01-31 16:49:56,106 UTC [2300146] INFO esmvalcore._task:506 Writing output to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/work/quantilebias/main 2025-01-31 16:49:56,107 UTC [2300146] INFO esmvalcore._task:507 Writing plots to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/plots/quantilebias/main 2025-01-31 16:49:56,107 UTC [2300146] INFO esmvalcore._task:508 Writing log to /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/run/quantilebias/main/log.txt 2025-01-31 16:49:56,107 UTC [2300146] INFO esmvalcore._task:518 To re-run this diagnostic script, run: cd /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/run/quantilebias/main; diag_scripts="/projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvaltool/diag_scripts" /projects/NS2980K/conda/esmvaltool2.11.0/bin/Rscript /projects/NS2980K/conda/esmvaltool2.11.0/lib/python3.11/site-packages/esmvaltool/diag_scripts/quantilebias/quantilebias.R /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/run/quantilebias/main/settings.yml 2025-01-31 16:49:56,199 UTC [2299982] INFO esmvalcore._task:793 Progress: 1 tasks running, 0 tasks waiting for ancestors, 1/2 done 2025-01-31 16:49:59,114 UTC [2300146] INFO esmvalcore._task:128 Maximum memory used (estimate): 0.1 GB 2025-01-31 16:49:59,116 UTC [2300146] INFO esmvalcore._task:130 Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. 2025-01-31 16:49:59,116 UTC [2300146] DEBUG esmvalcore._task:591 Script quantilebias/quantilebias.R completed successfully 2025-01-31 16:49:59,116 UTC [2300146] DEBUG esmvalcore._task:610 Collecting provenance from /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/run/quantilebias/main/diagnostic_provenance.yml 2025-01-31 16:49:59,274 UTC [2300146] DEBUG PIL.PngImagePlugin:198 STREAM b'IHDR' 16 13 2025-01-31 16:49:59,274 UTC [2300146] DEBUG PIL.PngImagePlugin:198 STREAM b'IDAT' 41 8192 2025-01-31 16:49:59,294 UTC [2300146] DEBUG esmvalcore._task:685 Collecting provenance of task quantilebias/main took 0.2 seconds 2025-01-31 16:49:59,294 UTC [2300146] INFO esmvalcore._task:266 Successfully completed task quantilebias/main (priority 0) in 0:00:03.190481 2025-01-31 16:49:59,301 UTC [2299982] INFO esmvalcore._task:793 Progress: 0 tasks running, 0 tasks waiting for ancestors, 2/2 done 2025-01-31 16:49:59,302 UTC [2299982] INFO esmvalcore._task:798 Successfully completed all tasks. 2025-01-31 16:49:59,325 UTC [2299982] INFO esmvalcore._recipe.recipe:1096 Wrote recipe with version numbers and wildcards to: file:///projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/run/recipe_quantilebias_filled.yml 2025-01-31 16:49:59,474 UTC [2299982] INFO esmvalcore.experimental.recipe_output:278 Wrote recipe output to: file:///projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947/index.html 2025-01-31 16:49:59,474 UTC [2299982] INFO esmvalcore.experimental.recipe_output:248 It looks like you are connected to a remote machine via SSH. To show the output html file, you can try the following command on your local machine: server=10.11.120.250 && port=31415 && ssh -t -L ${port}:localhost:${port} tomast@${server} /projects/NS2980K/conda/esmvaltool2.11.0/bin/python -m http.server ${port} -d /projects/NS2980K/www/tomast/esmvaltool2.11.0/output/recipe_quantilebias_20250131_164947 Then visit http://localhost:31415 in your browser 2025-01-31 16:49:59,474 UTC [2299982] INFO esmvalcore.experimental.recipe_output:256 If the port 31415 is already in use, you can replace it with any other free one (e.g., 12789). If you are connected through a jump host, replace the server IP address 10.11.120.250 with your SSH server name 2025-01-31 16:49:59,474 UTC [2299982] INFO esmvalcore._main:133 Ending the Earth System Model Evaluation Tool at time: 2025-01-31 16:49:59 UTC 2025-01-31 16:49:59,474 UTC [2299982] INFO esmvalcore._main:136 Time for running the recipe was: 0:00:11.310057 2025-01-31 16:50:00,088 UTC [2299982] INFO esmvalcore._task:128 Maximum memory used (estimate): 1.3 GB 2025-01-31 16:50:00,089 UTC [2299982] INFO esmvalcore._task:130 Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. 2025-01-31 16:50:00,090 UTC [2299982] INFO esmvalcore._main:465 Run was successful