Determine whether all AutoAssess areas will work with the ESMValTool community environment

[ehogan]

The basic requirements for AutoAssess are Iris and it's associated libraries. A minimum viable conda environment that can run AutoAssess and it's testing environment is set up with the following and is equivalent to the 02/09/2024 release of SciTools. AutoAssess continues to work with all subsequent scitools/default-current releases.

channels:
    - conda-forge
dependencies:
# Python version and packaging
    - python=3.12.5
    - pip=25.0.1
# Iris + Dependencies (to match scitools/default 02/09/2024)
    - iris=3.10.0
    - iris-esmf-regrid=0.11.0
    - mo_pack=0.3.0
    - image-meta-tag=0.8.2
    - nc-time-axis=1.4.1
    - cftime=1.6.4
    - matplotlib=3.9.2
    - cartopy=0.23.0
    - dask=2024.8.2
    - cf-units=3.2.0
    - numpy=1.26.4
    - scipy=1.14.1
    - geos=3.12.2
    - esmf=8.6.1
    - pillow=10.4.0
# Documentation
    - sphinx=7.4.7
    - sphinx_rtd_theme=2.0.0
# Profiling
    - memory_profiler=0.57.0
# Testing
    - click=8.1.7
    - pylint=3.2.7
    - pytest=8.3.2
    - pip:
        - pytest-cov==6.0.0
        - pytest-xdist==3.6.1

[NParsonsMO]

It's probably worth noting that I don't actually understand environments, or conda, or packaging, but simply comparing this list to the lock file for esmvaltool 2.12.0: most things are mentioned there, many have higher version numbers.

A few are not there:

Missing:

• mo_pack
• image-meta-tag
• memory_profiler

Lower version:

• esmf

Not from pip:

• pytest-cov
• pytest-xdist

Here is a copy of the relevant lines (reordered to match the issue description above), in case I've completely missed the point:

https://metoffice.jfrog.io/metoffice/api/conda/conda-forge/

...linux-64/python-3.12.0-hab00c5b_0_cpython.conda
...noarch/pip-25.1.1-pyh8b19718_0.conda

...noarch/iris-3.12.2-pyha770c72_0.conda
...noarch/iris-esmf-regrid-0.12.1-pyhd8ed1ab_0.conda
NOT mo_pack=0.3.0
NOT image-meta-tag=0.8.2
...noarch/nc-time-axis-1.4.1-pyhd8ed1ab_1.conda
...linux-64/cftime-1.6.4-py312hc0a28a1_1.conda
...linux-64/matplotlib-base-3.10.3-py312hd3ec401_0.conda
...linux-64/cartopy-0.24.0-py312hf9745cd_0.conda
...noarch/dask-core-2025.5.1-pyhd8ed1ab_0.conda
...linux-64/cf-units-3.3.0-py312hc0a28a1_0.conda
...linux-64/numpy-2.2.6-py312h72c5963_0.conda
...linux-64/scipy-1.15.2-py312ha707e6e_0.conda
...linux-64/geos-3.12.2-he02047a_1.conda
...linux-64/esmf-8.4.2-nompi_h9e768e6_3.conda
...linux-64/pillow-11.2.1-py312h80c1187_0.conda

...noarch/sphinx-8.1.3-pyhd8ed1ab_1.conda
...noarch/sphinx_rtd_theme-3.0.1-pyha770c72_0.conda

NOT memory_profiler=0.57.0

...noarch/click-8.2.1-pyh707e725_0.conda
...noarch/pylint-3.3.7-pyhe01879c_0.conda
...noarch/pytest-8.4.0-pyhd8ed1ab_0.conda
...noarch/pytest-cov-6.2.1-pyhd8ed1ab_0.conda
...noarch/pytest-xdist-3.7.0-pyhd8ed1ab_0.conda

[pdearnshaw]

Thanks for this Naomi. I do not think specific versions will likely be a problem as AutoAssess is working with a wider array of versions than those specified above across multiple scitools/default releases. As for the missing modules:

• mo_pack = Required for reading packed PP files, so required for AutoAssess at present, but not a requirement if restricted to reading NetCDF files as in ESMValTool.
• image-meta-tag = Only used in validation note, and not necessarily a definitive requirement there anyway. It can cope without it and it is not being used to it's best effect.
• memory_profiler = Not actually used anywhere so no issue with it being missing. I only used it when trying to track down issues with validation note move to python/matplotlib.

I should be able to move on with attempting to use the esmvaltool package but concerned that the lack of the mo_pack module will probably present a problem. Will try, but could use scitools/default-current in the interim.