INFO [16205] ______________________________________________________________________ _____ ____ __ ____ __ _ _____ _ | ____/ ___|| \/ \ \ / /_ _| |_ _|__ ___ | | | _| \___ \| |\/| |\ \ / / _` | | | |/ _ \ / _ \| | | |___ ___) | | | | \ V / (_| | | | | (_) | (_) | | |_____|____/|_| |_| \_/ \__,_|_| |_|\___/ \___/|_| ______________________________________________________________________ ESMValTool - Earth System Model Evaluation Tool. http://www.esmvaltool.org CORE DEVELOPMENT TEAM AND CONTACTS: Birgit Hassler (Co-PI; DLR, Germany - birgit.hassler@dlr.de) Alistair Sellar (Co-PI; Met Office, UK - alistair.sellar@metoffice.gov.uk) Bouwe Andela (Netherlands eScience Center, The Netherlands - b.andela@esciencecenter.nl) Lee de Mora (PML, UK - ledm@pml.ac.uk) Niels Drost (Netherlands eScience Center, The Netherlands - n.drost@esciencecenter.nl) Veronika Eyring (DLR, Germany - veronika.eyring@dlr.de) Bettina Gier (UBremen, Germany - gier@uni-bremen.de) Remi Kazeroni (DLR, Germany - remi.kazeroni@dlr.de) Nikolay Koldunov (AWI, Germany - nikolay.koldunov@awi.de) Axel Lauer (DLR, Germany - axel.lauer@dlr.de) Saskia Loosveldt-Tomas (BSC, Spain - saskia.loosveldt@bsc.es) Ruth Lorenz (ETH Zurich, Switzerland - ruth.lorenz@env.ethz.ch) Benjamin Mueller (LMU, Germany - b.mueller@iggf.geo.uni-muenchen.de) Valeriu Predoi (URead, UK - valeriu.predoi@ncas.ac.uk) Mattia Righi (DLR, Germany - mattia.righi@dlr.de) Manuel Schlund (DLR, Germany - manuel.schlund@dlr.de) Breixo Solino Fernandez (DLR, Germany - breixo.solinofernandez@dlr.de) Javier Vegas-Regidor (BSC, Spain - javier.vegas@bsc.es) Klaus Zimmermann (SMHI, Sweden - klaus.zimmermann@smhi.se) For further help, please read the documentation at http://docs.esmvaltool.org. Have fun! INFO [16205] Package versions INFO [16205] ---------------- INFO [16205] ESMValCore: 2.7.0 INFO [16205] ESMValTool: 2.7.0 INFO [16205] ---------------- INFO [16205] Using config file /projects/NS2980K/home/tomast/ESMValTool/rundir_ipcc_node/config-ipcc_node-270.yml INFO [16205] Writing program log files to: /projects/NS2980K/www/tomast/esmvaltool2.7.0/recipe_arctic_ocean_20230305_195604/run/main_log.txt /projects/NS2980K/www/tomast/esmvaltool2.7.0/recipe_arctic_ocean_20230305_195604/run/main_log_debug.txt INFO [16205] Starting the Earth System Model Evaluation Tool at time: 2023-03-05 19:56:06 UTC INFO [16205] ---------------------------------------------------------------------- INFO [16205] RECIPE = /projects/NS2980K/home/tomast/ESMValTool/rundir_ipcc_node/recipe_arctic_ocean.yml INFO [16205] RUNDIR = /projects/NS2980K/www/tomast/esmvaltool2.7.0/recipe_arctic_ocean_20230305_195604/run INFO [16205] WORKDIR = /projects/NS2980K/www/tomast/esmvaltool2.7.0/recipe_arctic_ocean_20230305_195604/work INFO [16205] PREPROCDIR = /projects/NS2980K/www/tomast/esmvaltool2.7.0/recipe_arctic_ocean_20230305_195604/preproc INFO [16205] PLOTDIR = /projects/NS2980K/www/tomast/esmvaltool2.7.0/recipe_arctic_ocean_20230305_195604/plots INFO [16205] ---------------------------------------------------------------------- INFO [16205] Running tasks using at most 8 processes INFO [16205] If your system hangs during execution, it may not have enough memory for keeping this number of tasks in memory. INFO [16205] If you experience memory problems, try reducing 'max_parallel_tasks' in your user configuration file. INFO [16205] Creating tasks from recipe INFO [16205] Creating tasks for diagnostic arctic_ocean INFO [16205] Creating diagnostic task arctic_ocean/arctic_ocean_main INFO [16205] Creating preprocessor task arctic_ocean/areacello INFO [16205] Creating preprocessor 'default' task for variable 'areacello' INFO [16205] Found input files for CMIP5 MPI-ESM-LR r0i0p0 INFO [16205] Found input files for CMIP5 MPI-ESM-MR r0i0p0 INFO [16205] Found input files for CMIP5 CCSM4 r0i0p0 INFO [16205] Found input files for CMIP5 CanESM2 r0i0p0 INFO [16205] Found input files for CMIP5 CMCC-CM r0i0p0 INFO [16205] Found input files for CMIP5 ACCESS1-3 r0i0p0 INFO [16205] Found input files for CMIP5 ACCESS1-0 r0i0p0 INFO [16205] Found input files for CMIP5 CSIRO-Mk3-6-0 r0i0p0 INFO [16205] Found input files for CMIP5 IPSL-CM5A-LR r0i0p0 INFO [16205] Found input files for CMIP5 IPSL-CM5A-MR r0i0p0 INFO [16205] Found input files for CMIP5 HadCM3 r0i0p0 INFO [16205] Found input files for CMIP5 HadGEM2-ES r0i0p0 INFO [16205] Found input files for OBS6 r0i0p0 INFO [16205] PreprocessingTask arctic_ocean/areacello created. INFO [16205] Creating preprocessor task arctic_ocean/thetao INFO [16205] Creating preprocessor 'default' task for variable 'thetao' INFO [16205] Found input files for CMIP5 MPI-ESM-LR r1i1p1 INFO [16205] Found input files for CMIP5 MPI-ESM-MR r1i1p1 INFO [16205] Found input files for CMIP5 CCSM4 r1i1p1 INFO [16205] Found input files for CMIP5 CanESM2 r1i1p1 INFO [16205] Found input files for CMIP5 CMCC-CM r1i1p1 INFO [16205] Found input files for CMIP5 ACCESS1-3 r1i1p1 INFO [16205] Found input files for CMIP5 ACCESS1-0 r1i1p1 INFO [16205] Found input files for CMIP5 CSIRO-Mk3-6-0 r1i1p1 INFO [16205] Found input files for CMIP5 IPSL-CM5A-LR r1i1p1 INFO [16205] Found input files for CMIP5 IPSL-CM5A-MR r1i1p1 INFO [16205] Found input files for CMIP5 HadCM3 r1i1p1 INFO [16205] Found input files for CMIP5 HadGEM2-ES r1i1p1 INFO [16205] Found input files for OBS6 r1i1p1 INFO [16205] PreprocessingTask arctic_ocean/thetao created. INFO [16205] Creating preprocessor task arctic_ocean/so INFO [16205] Creating preprocessor 'default' task for variable 'so' INFO [16205] Found input files for CMIP5 MPI-ESM-LR r1i1p1 INFO [16205] Found input files for CMIP5 MPI-ESM-MR r1i1p1 INFO [16205] Found input files for CMIP5 CCSM4 r1i1p1 INFO [16205] Found input files for CMIP5 CanESM2 r1i1p1 INFO [16205] Found input files for CMIP5 CMCC-CM r1i1p1 INFO [16205] Found input files for CMIP5 ACCESS1-3 r1i1p1 INFO [16205] Found input files for CMIP5 ACCESS1-0 r1i1p1 INFO [16205] Found input files for CMIP5 CSIRO-Mk3-6-0 r1i1p1 INFO [16205] Found input files for CMIP5 IPSL-CM5A-LR r1i1p1 INFO [16205] Found input files for CMIP5 IPSL-CM5A-MR r1i1p1 INFO [16205] Found input files for CMIP5 HadCM3 r1i1p1 INFO [16205] Found input files for CMIP5 HadGEM2-ES r1i1p1 INFO [16205] Found input files for OBS6 r1i1p1 INFO [16205] PreprocessingTask arctic_ocean/so created. INFO [16205] These tasks will be executed: arctic_ocean/thetao, arctic_ocean/arctic_ocean_main, arctic_ocean/areacello, arctic_ocean/so INFO [16205] Running 4 tasks using 4 processes INFO [16361] Starting task arctic_ocean/areacello in process [16361] INFO [16362] Starting task arctic_ocean/thetao in process [16362] INFO [16363] Starting task arctic_ocean/so in process [16363] INFO [16205] Progress: 3 tasks running, 1 tasks waiting for ancestors, 0/4 done WARNING [16362] There were warnings in variable thetao: Added guessed bounds to coordinate lon from var lon Coordinate lev units meters converted to m Added guessed bounds to coordinate time from var thetao loaded from file /projects/NS9252K/ESGF_betzy/obsdata/Tier2/PHC/OBS6_PHC_clim_3_Omon_thetao_195001-195012.nc WARNING [16362] There were warnings in variable thetao: Added guessed bounds to coordinate lon from var lon Coordinate lev units meters converted to m Added guessed bounds to coordinate time from var thetao loaded from file /projects/NS9252K/obsdata/Tier2/PHC/OBS6_PHC_clim_3_Omon_thetao_195001-195012.nc WARNING [16361] There were warnings in variable areacello: Added guessed bounds to coordinate lon from var lon loaded from file /projects/NS9252K/ESGF_betzy/obsdata/Tier2/PHC/OBS6_PHC_clim_3_fx_areacello.nc WARNING [16361] There were warnings in variable areacello: Added guessed bounds to coordinate lon from var lon loaded from file /projects/NS9252K/obsdata/Tier2/PHC/OBS6_PHC_clim_3_fx_areacello.nc ERROR [16361] Failed to run preprocessor function 'concatenate' on the data [, ] loaded from original input file(s) ['/projects/NS9252K/ESGF_betzy/obsdata/Tier2/PHC/OBS6_PHC_clim_3_fx_areacello.nc', '/projects/NS9252K/obsdata/Tier2/PHC/OBS6_PHC_clim_3_fx_areacello.nc'] with function argument(s) INFO [16205] Maximum memory used (estimate): 1.2 GB INFO [16205] Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. ERROR [16205] Program terminated abnormally, see stack trace below for more information: multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/preprocessor/_io.py", line 233, in concatenate cubes = sorted(cubes, key=lambda c: c.coord("time").cell(0).point) File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/preprocessor/_io.py", line 233, in cubes = sorted(cubes, key=lambda c: c.coord("time").cell(0).point) File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/iris/cube.py", line 1919, in coord raise iris.exceptions.CoordinateNotFoundError(emsg) iris.exceptions.CoordinateNotFoundError: "Expected to find exactly 1 'time' coordinate, but found none." During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/_task.py", line 796, in _run_task output_files = task.run() File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/_task.py", line 263, in run self.output_files = self._run(input_files) File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/preprocessor/__init__.py", line 610, in _run product.apply(step, self.debug) File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/preprocessor/__init__.py", line 421, in apply self.cubes = preprocess(self.cubes, step, File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/preprocessor/__init__.py", line 353, in preprocess result.append(_run_preproc_function(function, items, settings, File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/preprocessor/__init__.py", line 313, in _run_preproc_function return function(items, **kwargs) File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/preprocessor/_io.py", line 237, in concatenate raise ValueError(msg) ValueError: One or more cubes [, ] are missing time coordinate: "Expected to find exactly 1 'time' coordinate, but found none." """ The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/_main.py", line 499, in run fire.Fire(ESMValTool()) File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/fire/core.py", line 141, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/fire/core.py", line 466, in _Fire component, remaining_args = _CallAndUpdateTrace( File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/fire/core.py", line 681, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/_main.py", line 443, in run process_recipe(recipe_file=recipe, config_user=cfg) File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/_main.py", line 124, in process_recipe recipe.run() File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/_recipe.py", line 1921, in run self.tasks.run(max_parallel_tasks=self._cfg['max_parallel_tasks']) File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/_task.py", line 725, in run self._run_parallel(max_parallel_tasks) File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/_task.py", line 768, in _run_parallel _copy_results(task, running[task]) File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/_task.py", line 791, in _copy_results task.output_files, task.products = future.get() File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/multiprocessing/pool.py", line 774, in get raise self._value File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/multiprocessing/pool.py", line 125, in worker result = (True, func(*args, **kwds)) File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/_task.py", line 796, in _run_task output_files = task.run() File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/_task.py", line 263, in run self.output_files = self._run(input_files) File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/preprocessor/__init__.py", line 610, in _run product.apply(step, self.debug) File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/preprocessor/__init__.py", line 421, in apply self.cubes = preprocess(self.cubes, step, File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/preprocessor/__init__.py", line 353, in preprocess result.append(_run_preproc_function(function, items, settings, File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/preprocessor/__init__.py", line 313, in _run_preproc_function return function(items, **kwargs) File "/projects/NS2980K/conda/esmvaltool2.7.0/lib/python3.10/site-packages/esmvalcore/preprocessor/_io.py", line 237, in concatenate raise ValueError(msg) ValueError: One or more cubes [, ] are missing time coordinate: "Expected to find exactly 1 'time' coordinate, but found none." INFO [16205] If you have a question or need help, please start a new discussion on https://github.com/ESMValGroup/ESMValTool/discussions If you suspect this is a bug, please open an issue on https://github.com/ESMValGroup/ESMValTool/issues To make it easier to find out what the problem is, please consider attaching the files run/recipe_*.yml and run/main_log_debug.txt from the output directory.