- Coherence estimation results can have missing pixels(or coherence 0 values) on north and east direction on coastlines when compared to SNAP. Currently it is unresolved what is the correct way. See examples A and B. See posts about the issues/bugs - tile size affecting results and no-data value interpretation.
- Due to the nature of floating point arithmetic there are some discrepancies when compared to SNAP, see slideshow
- Basic S1 metadata in the produced results for
alus-coh,alus-calandalus-coht- PR item - Polynomial estimation for subtract flat earth phase for each burst in coherence opeation - PR item
- Jupyter notebook and related dependencies updates - PR item
- GRD support for
alus-cal- PR item
- All the references and build automation functionality linked to Bitbucket is removed. Everything is now migrated to Github, hence Github actions are supported only.
- Coherence estimation results can have missing pixels(or coherence 0 values) on north and east direction on coastlines when compared to SNAP. Currently it is unresolved what is the correct way. See examples A and B. See posts about the issues/bugs - tile size affecting results and no-data value interpretation.
- Due to the nature of floating point arithmetic there are some discrepancies when compared to SNAP, see slideshow
- Processing scenes with Copernicus DEM 30m COG has not well defined behavior for some specific cases. See workaround in ALUs - https://github.com/cgi-estonia-space/ALUs/wiki/DEM-handling-functionality-for-Sentinel-1-processors#copernicus-dem-cog-30m SNAP processing discrepancies - https://forum.step.esa.int/t/copernicus-dem-complications-when-coregistering-s1/38659/3
- Shapefiles(.shp) are supported for specifying AOI - PR item
- Copernicus DEM 30m COG support - PR1 item PR2 item
- 'Secret' log format can be invoked with
--log_format_creodias. This enables JSON log output with 3 levels - DEBUG, INFO, ERROR - PR item - 'alus-coh' will process scenes without orbit files now, before supplying orbit files was compulsory - PR item
- Some scenes had incorrect merge overlap for IW2 and IW3 for all S1 routines, this is now fixed and made faster - PR item
- Coherence estimation results can have missing pixels(or coherence 0 values) on north and east direction on coastlines when compared to SNAP. Currently it is unresolved what is the correct way. See examples A and B. See posts about the issues/bugs - tile size affecting results and no-data value interpretation.
- Due to the nature of floating point arithmetic there are some discrepancies when compared to SNAP, see slideshow
- There are two code paths to calculate terrain corrected outputs' dimensions and geocoordinates (see description). Since ALUs implements the SNAP UI way the end results probably differ when compared to SNAP GPT outputs.
- ALOS PALSAR Zero-Doppler L0 focuser implemented
- Subswath merge support for calibration routine
- Tile access error in coherence operator fixed
- Coherence estimation results can have missing pixels(or coherence 0 values) on north and east direction on coastlines when compared to SNAP. Currently it is unresolved what is the correct way. See examples A and B. See posts about the issues/bugs - tile size affecting results and no-data value interpretation.
- Due to the nature of floating point arithmetic there are some discrepancies when compared to SNAP, see slideshow
- There are two code paths to calculate terrain corrected outputs' dimensions and geocoordinates (see description). Since ALUs implements the SNAP UI way the end results probably differ when compared to SNAP GPT outputs.
- Gabor feature extraction processor added
- Preliminary ALOS PALSAR level 0 focusser added
- Coherence estimation results can have missing pixels(or coherence 0 values) on north and east direction on coastlines when compared to SNAP. Currently it is unresolved what is the correct way. See examples A and B. See posts about the issues/bugs - tile size affecting results and no-data value interpretation.
- Due to the nature of floating point arithmetic there are some discrepancies when compared to SNAP, see slideshow
- There are two code paths to calculate terrain corrected outputs' dimensions and geocoordinates (see description). Since ALUs implements the SNAP UI way the end results probably differ when compared to SNAP GPT outputs.
- Thermal Noise Removal step added to calibration routine
2 additional processors:
alus-resa- resampling raster dataalus-coht- coherence timeline generation
- Coherence estimation results can have missing pixels(or coherence 0 values) on north and east direction on coastlines when compared to SNAP. Currently it is unresolved what is the correct way. See examples A and B. See posts about the issues/bugs - tile size affecting results and no-data value interpretation.
- Due to the nature of floating point arithmetic there are some discrepancies when compared to SNAP, see slideshow
--no_mask_corimplemented to disable masking during the coregister step for coherence estimation. Disabling avoids bug which creates incorrect pixels on coastline. Using similar functionality in SNAP would make results match on coastlines 100%.- Merge operation supported for coherence estimation. It is triggered by specifying AOI and no subswaths. This differs from SNAP where merge could be generated by manually specifying subswaths. Subswath specification functions only for single subswath in ALUs. Also overlap areas in range direction between subswaths are more optimized where overlapping areas are minimized as much as possible.
- Orbit file directories without structure (all orbit files are stored with no hierarchy at a single level) are supported for coherence estimation now.
- Resampling processor implemented. Specific support implemented for Sentinel-2 L1C and L2A datasets. Other rasters are resampled in a 'generic' way. Additional tiling and reprojection can be specified.
- Memory leak bug fixed for coregistration operator.
This is the first release with refactored architecture where each processor is a different executable.
- Old commands do not work since the
alussingle executable is broken down into multiple separate executables
- Coherence estimation results can have missing pixels(or coherence 0 values) on north and east direction on coastlines when compared to SNAP. Currently it is unresolved what is the correct way. See examples A and B. See posts about the issues/bugs - tile size affecting results and no-data value interpretation.
- Due to the nature of floating point arithmetic there are some discrepancies when compared to SNAP, see slideshow
- There is a Jupyter notebook, which enables to more easily assign processing parameters, inputs and it automatically downloads needed auxiliary files background. This should not be used for speed comparisons, because data transfers and unpacking ZIP archives would consume most of the total processing time.
- GPU device initialization and property component implemented - based on this functionality resources and parameters could be assigned better
- Officially upgraded to Ubuntu 20.04 LTS
- Clang 10 compiler support added - refactored minor issues and CI pipeline checks added
- Boost library versions downgraded to 1.71 to match the distro default ones
This will be the last release with the current architecture (algorithms as shared libraries) and Ubuntu 18.04 as the officially supported platform.
- Auxiliary files must be separately downloaded and supplied via CLI arguments.
- Coherence estimation results can have missing pixels(or coherence 0 values) on north and east direction on coastlines. Currently investigation is ongoing to find out the exact reason. See examples A and B.
- All the files and directories are formatted and named according to rules.
- License headers added where missing.
- Scripts added for automated code style and content conformity. This is checked for every pull request now.
- Auxiliary files must be separately downloaded and supplied via CLI arguments
- Coherence estimation results can have missing pixels(or coherence 0 values) on north and east direction on coastlines. Currently investigation is ongoing to find out the exact reason. See examples A and B.
- ZIP archive support
- Sentinel 1 SLC scenes
- SRTM3 DEM files
- POEORB files
- Git LFS references removed
- Replaced by custom
.alus-lfsscript in the root of the repository along withalusresourcesfile - CI pipelines are now 2x and more faster
- Replaced by custom
- ZIP files are not supported, any input datasets and auxiliary files must be extracted
- Auxiliary files must be separately downloaded and supplied via CLI arguments
- Improvements for calibration routine:
- Split arguments and operation supported and integrated, also support for burst index and AOI WKT polygon splitting
- SRTM3 DEM loading is asynchronous
- Improvements for coherence routine:
- Support for split operation by burst index or AOI WKT polygon
- Performance optimizations for backgeocoding and coherence operations
- Sin and Cos computations with single CUDA API call for backgeocoding
- INT16 is integral type for raster data in backgeocoding now
- Pinned memory and streams utilisation for coherence operator
- Threading refactoring. Coregistration output now supports 4 datasets with 1 band or 1 dataset with 4 bands
- Compute extended amount no longer reallocates metadata for each call and uses atomics for index calculations
- Backgeocoding operator
- Kernel block and grid parameters fixed which caused processing to fail on graphic cards with CC 7.5
- Coherence operator
- Metadata mishandling caused wrong results for scenes with near range on left
- Different tile sizes altering results is fixed
- Unnecessary shared libraries removed from release binary package
- More nightly test validation scenes
- Also refactored the setup so that the scripts could be run on individual development environment
- SNAP-ENGINE and Sentinel 1 toolbox port:
- relocated ported structures to separate folders in the root of the repo
Slides about the speed improvements and accuracy
- Coherence operator and estimation routine:
- Completely removed any TensorFlow implementation code
- From scratch implementation using only core Nvidia native APIs(CUDA, cuBLAS).
- Any included Tensorflow libraries/headers removed from repository
- No AOI or burst range based processing, only whole subswaths
-
Optimizations for Coherence estimation and Calibration routines:
- All the operators use memory more efficiently (CUDA pinned memory, unified memory areas)
- Speedup from 2x-10x achieved depending on the operator
- Datasets of intermediate results are not written to disk by default
-
Unified logging system implemented using Boost's Log.
-
SNAP-ENGINE port:
- Some of the cyclic references that caused memory leaks were removed
- Some still remain
- Some of the cyclic references that caused memory leaks were removed
-
Coherence estimation routine
- Polarization and subswath parameters' hardcoded values removed
-
Backgeocoding
- Out of bounds bug fixed
- Triangular interpolation optimizations
-
Calibration
- Sigma0 and dn bands results' correction