Hi. I'm trying to install arosics on Ubuntu 18.04. I have downloaded 0.9.3 archive from gitlab repository https://gitext.gfz-potsdam.de/danschef/arosics/tree/v0.9.3, extracted it and in the source directory execute make install
. But despite advertised support for Python3, this command failed with the following output
$ make install
rm -fr build/
rm -fr dist/
rm -fr .eggs/
find . -name '*.egg-info' -exec rm -fr {} +
find . -name '*.egg' -exec rm -f {} +
find . -name '*.pyc' -exec rm -f {} +
find . -name '*.pyo' -exec rm -f {} +
find . -name '*~' -exec rm -f {} +
find . -name '__pycache__' -exec rm -fr {} +
## don't include 'coverage erase' lib here because clean-test is also executed during package setup and coverage is
## only a test requirement
rm -fr .tox/
rm -f .coverage
rm -fr htmlcov/
rm -fr nosetests.html
rm -fr nosetests.xml
pip install -r requirements.txt
make: pip: Command not found
Makefile:101: recipe for target 'install' failed
make: *** [install] Error 127
Ok, I looked in the Makefile
and saw that it uses pip instead of pip3 and python instead if python3, chnaged all pip occurences to pip3, python to python3 and tried again. No luck
$ make install
rm -fr build/
rm -fr dist/
rm -fr .eggs/
find . -name '*.egg-info' -exec rm -fr {} +
find . -name '*.egg' -exec rm -f {} +
find . -name '*.pyc' -exec rm -f {} +
find . -name '*.pyo' -exec rm -f {} +
find . -name '*~' -exec rm -f {} +
find . -name '__pycache__' -exec rm -fr {} +
## don't include 'coverage erase' lib here because clean-test is also executed during package setup and coverage is
## only a test requirement
rm -fr .tox/
rm -f .coverage
rm -fr htmlcov/
rm -fr nosetests.html
rm -fr nosetests.xml
pip3 install -r requirements.txt
Collecting geoarray>=0.8.17 (from -r requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/eb/b5/b87143c311c90bfe3bf24048af0fbbc5706902202135ed4c57f14832115f/geoarray-0.8.25.tar.gz (58kB)
100% |████████████████████████████████| 61kB 793kB/s
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/geoarray.egg-info
writing pip-egg-info/geoarray.egg-info/PKG-INFO
writing dependency_links to pip-egg-info/geoarray.egg-info/dependency_links.txt
writing requirements to pip-egg-info/geoarray.egg-info/requires.txt
writing top-level names to pip-egg-info/geoarray.egg-info/top_level.txt
writing manifest file 'pip-egg-info/geoarray.egg-info/SOURCES.txt'
reading manifest file 'pip-egg-info/geoarray.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '__pycache__' found under directory '*'
warning: no previously-included files matching '*.py[co]' found under directory '*'
warning: no files found matching '*.jpg' under directory 'docs'
writing manifest file 'pip-egg-info/geoarray.egg-info/SOURCES.txt'
/tmp/pip-build-qhds088s/geoarray/setup.py:87: UserWarning: You need to install holoviews manually (see www.holoviews.org) if you want to use interactive plotting. It is not automatically installed.
warnings.warn('You need to install holoviews manually (see www.holoviews.org) if you want to use interactive '
Traceback (most recent call last):
File "/usr/lib/python3.6/pkgutil.py", line 490, in find_loader
spec = importlib.util.find_spec(fullname)
File "/usr/lib/python3.6/importlib/util.py", line 88, in find_spec
parent = __import__(parent_name, fromlist=['__path__'])
ModuleNotFoundError: No module named 'mpl_toolkits'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-qhds088s/geoarray/setup.py", line 91, in <module>
if not find_loader('mpl_toolkits.basemap'):
File "/usr/lib/python3.6/pkgutil.py", line 496, in find_loader
raise ImportError(msg.format(fullname, type(ex), ex)) from ex
ImportError: Error while finding loader for 'mpl_toolkits.b
then iI tried to install it from PyPI using following command
pip3 install arosics
and again got error
$ pip3 install arosics
Collecting arosics
Using cached https://files.pythonhosted.org/packages/2b/da/755b4088ee1ab1d661a7780683515d24166a2e08f250803e1594823508cd/arosics-0.9.3.tar.gz
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/arosics.egg-info
writing pip-egg-info/arosics.egg-info/PKG-INFO
writing dependency_links to pip-egg-info/arosics.egg-info/dependency_links.txt
writing requirements to pip-egg-info/arosics.egg-info/requires.txt
writing top-level names to pip-egg-info/arosics.egg-info/top_level.txt
writing manifest file 'pip-egg-info/arosics.egg-info/SOURCES.txt'
reading manifest file 'pip-egg-info/arosics.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '__pycache__' found under directory '*'
warning: no previously-included files matching '*.py[co]' found under directory '*'
warning: no files found matching '*.jpg' under directory 'docs'
writing manifest file 'pip-egg-info/arosics.egg-info/SOURCES.txt'
/tmp/pip-build-n4nsbt19/arosics/setup.py:93: UserWarning: You need to install pyfftw manually (see https://pypi.python.org/pypi/pyFFTW) for speeding up the computation. It is not automatically installed.
warnings.warn('You need to install pyfftw manually (see https://pypi.python.org/pypi/pyFFTW) for speeding up '
Traceback (most recent call last):
File "/usr/lib/python3.6/pkgutil.py", line 490, in find_loader
spec = importlib.util.find_spec(fullname)
File "/usr/lib/python3.6/importlib/util.py", line 88, in find_spec
parent = __import__(parent_name, fromlist=['__path__'])
ModuleNotFoundError: No module named 'mpl_toolkits'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-n4nsbt19/arosics/setup.py", line 97, in <module>
if not find_loader('mpl_toolkits.basemap'):
File "/usr/lib/python3.6/pkgutil.py", line 496, in find_loader
raise ImportError(msg.format(fullname, type(ex), ex)) from ex
ImportError: Error while finding loader for 'mpl_toolkits.basemap' (<class 'ModuleNotFoundError'>: No module named 'mpl_toolkits')
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-n4nsbt19/arosics/
The installation instructions of AROSICS are documented here: http://danschef.gitext.gfz-potsdam.de/arosics/doc/installation.html.
Please note, that AROSICS depends on some packages that cause problems when installed via pip. That´s why it is strongly recommended to install AROSICS within a separate Anaconda environment using the Anaconda package manager. This way it should not be a problem to use the API and CLI on an Ubuntu system.
I'm just getting started with AROSICS -- this is great. Great paper and documentation. I'm filing a bug report here instead of gitlab since I'm not sure how with that enterprise instance.
The latest version of shapely (v1.7.0) is incompatible with arosics (at least when running with the CLI in global mode). I needed to downgrade to an earlier version (conda install -c conda-forge shapely=1.6.4
did the trick). Consequently running through the install commands at http://danschef.gitext.gfz-potsdam.de/arosics/doc/_sources/installation.rst.txt (without pinning the shapely version) will error out.
Hi Daniel, I've been experimenting with Arosics and I'm wondering if it's possible to create a mosaic using this software. What I like about this compared to GeFolki is that your software shows the X/Y shift in the console so I can manually do it if necessary.
I'm trying to create a mosaic using raw ungeoreferenced satellite imagery. I define an arbitrary projection (GCS WGS 1984) and I set the coordinates to 0,0 degrees for all the test images so that Arosics can function. I haven't been having much success using the global coregistration. Very few or none of the images in some cases can be coregistered. In some cases the shift is only a few pixels. Do you think what I'm trying to do is possible using Arosics?
Here is the sample of images I'm trying to coregister and create a mosaic. The image is roughly only 500x500 big and I can only use a window size < 128.
https://giphy.com/gifs/IevI6OC7dMsa6X4nFA
Hi Daniel , Great work. AROSICS works great out of the box on my RGB orthophotos, but I'm having trouble with a single band reflectance map (dtype Float32). Is that expected to work?
I keep getting the following error
First attempt to check if functionality of co-registration failed. Check your input data and parameters. The following error occurred:
Traceback (most recent call last):
File "/Users/adamjson/PycharmProjects/arosics3/test.py", line 69, in <module>
CRL = COREG_LOCAL(geoArr_reference, geoArr_target, **kwargs)
File "/Users/adamjson/anaconda3/envs/arosics4/lib/python3.7/site-packages/arosics/CoReg_local.py", line 256, in __init__
ignore_errors=False)
File "/Users/adamjson/anaconda3/envs/arosics4/lib/python3.7/site-packages/arosics/CoReg.py", line 310, in __init__
self._get_image_params()
File "/Users/adamjson/anaconda3/envs/arosics4/lib/python3.7/site-packages/arosics/CoReg.py", line 442, in _get_image_params
self.ref = GeoArray_CoReg(self.params, 'ref')
File "/Users/adamjson/anaconda3/envs/arosics4/lib/python3.7/site-packages/arosics/CoReg.py", line 122, in __init__
if not self.footprint_poly.is_valid:
File "/Users/adamjson/anaconda3/envs/arosics4/lib/python3.7/site-packages/geoarray/baseclasses.py", line 503, in footprint_poly
self._footprint_poly = fill_holes_within_poly(multipolygon)
File "/Users/adamjson/anaconda3/envs/arosics4/lib/python3.7/site-packages/py_tools_ds/geo/vector/topology.py", line 183, in fill_holes_within_poly
largest_poly_filled = gdf.loc[gdf['area_filled'].idxmax()]['geometry']
File "/Users/adamjson/anaconda3/envs/arosics4/lib/python3.7/site-packages/pandas/core/series.py", line 2110, in idxmax
i = nanops.nanargmax(com.values_from_object(self), skipna=skipna)
File "/Users/adamjson/anaconda3/envs/arosics4/lib/python3.7/site-packages/pandas/core/nanops.py", line 65, in _f
f"reduction operation '{f_name}' not allowed for this dtype"
TypeError: reduction operation 'argmax' not allowed for this dtype
If you could point me in the right direction I'd be happy to try and fix this if it turns out to be a bug.
Hi Daniel, I'm trying to use global georeferencing with 3m resolution reference images and 0.75m target images. I often get a bug where the shift validation repeats the same shifts several times, but doesn't consider it a match:
No clear match found yet. Jumping to iteration 27...
input shifts: -44 38
No clear match found yet. Jumping to iteration 28...
input shifts: -44 38
No clear match found yet. Jumping to iteration 29...
input shifts: -44 38
No clear match found yet. Jumping to iteration 30...
input shifts: -44 38
No match found in the given window, global coregistration failed
Do you have any suggestions on how to fix this?
Hi Daniel, I'm trying to global coregistration.
I got Type Error in calculating actual data corner coordinates.
Input data are ALOS2(PALSAR2) image(geo-coeded and have nodata around observed data).
Calculating actual data corner coordinates for reference image...
Automatically detected nodata value for GeoArray_CoReg 'IN_MEM': 0.0
Polygonize progress |==================================================| 100.0% Complete => 0:00:08
Traceback (most recent call last):
File "****", line 158, in <module>
createRGBCompositeImage(args.before, args.after, args.fpath_out)
File "****", line 129, in createRGBCompositeImage
CR = COREG(geoArr_after, geoArr_before, max_shift=20, q=False)
File "/export_afenfs_high-speed/product/mlt_test/morita/anaconda3/envs/arosics3/lib/python3.7/site-packages/arosics/CoReg.py", line 310, in __init__
self._get_image_params()
File "/export_afenfs_high-speed/product/mlt_test/morita/anaconda3/envs/arosics3/lib/python3.7/site-packages/arosics/CoReg.py", line 442, in _get_image_params
self.ref = GeoArray_CoReg(self.params, 'ref')
File "/export_afenfs_high-speed/product/mlt_test/morita/anaconda3/envs/arosics3/lib/python3.7/site-packages/arosics/CoReg.py", line 122, in __init__
if not self.footprint_poly.is_valid:
File "/export_afenfs_high-speed/product/mlt_test/morita/anaconda3/envs/arosics3/lib/python3.7/site-packages/geoarray/baseclasses.py", line 501, in footprint_poly
self._footprint_poly = fill_holes_within_poly(multipolygon)
File "/export_afenfs_high-speed/product/mlt_test/morita/anaconda3/envs/arosics3/lib/python3.7/site-packages/py_tools_ds/geo/vector/topology.py", line 184, in fill_holes_within_poly
largest_poly_filled = gdf.loc[gdf['area_filled'].idxmax()]['geometry']
File "/export_afenfs_high-speed/product/mlt_test/morita/anaconda3/envs/arosics3/lib/python3.7/site-packages/pandas/core/series.py", line 2168, in idxmax
i = nanops.nanargmax(self._values, skipna=skipna)
File "/export_afenfs_high-speed/product/mlt_test/morita/anaconda3/envs/arosics3/lib/python3.7/site-packages/pandas/core/nanops.py", line 67, in _f
f"reduction operation '{f_name}' not allowed for this dtype"
TypeError: reduction operation 'argmax' not allowed for this dtype
gdf['geometry'] has no element,this caused this error.
So,
gdf['geometry'] = poly
to
gdf['geometry'] = [ p for p in poly]
it seems OK.Do you have any suggestions on how to fix this?
Hi Daniel, I'm trying to fuse spatio-temporal data from Landsat-8 and Modis and I use Arosics for co-registration. The pre-processing steps that I follow before using Arosics are the following. At first I crop the images in the same extent and georeference Modis image in the same projection as Landsat. After that, I resample Modis image from 500 pixel size to 30 pixel size. In this point I use the command:
python arosics_cli.py global modis_image.tif landsat_image.tif
but it raises the following error:
Calculating actual data corner coordinates for reference image...
Bounding box of calculated footprint for reference image:
(280335.0, 3862485.0, 358545.0, 3925665.0)
Calculating actual data corner coordinates for image to be shifted...
/home/server/anaconda3/envs/fusion/lib/python3.7/site-packages/geoarray/baseclasses.py:377: UserWarning: Nodata value could not be clearly identified. It has been set to None.
warnings.warn('Nodata value could not be clearly identified. It has been set to None.')
Bounding box of calculated footprint for image to be shifted:
(280316.1072693282, 3862498.4168893998, 358526.1072693282, 3925678.4168893998)
Matching window position (X,Y): 319430.5536346641/3894081.7084447
No clear match found yet. Jumping to iteration 2...
input shifts: -18 37
No clear match found yet. Jumping to iteration 3...
input shifts: 35 -1
No clear match found yet. Jumping to iteration 4...
input shifts: 18 0
No clear match found yet. Jumping to iteration 5...
input shifts: -57 -21
Traceback (most recent call last):
File "/home/server/anaconda3/envs/fusion/bin/arosics_cli.py", line 362, in <module>
parsed_args.func(parsed_args)
File "/home/server/anaconda3/envs/fusion/bin/arosics_cli.py", line 68, in run_global_coreg
COREG_obj.correct_shifts()
File "/home/server/anaconda3/envs/fusion/lib/python3.7/site-packages/arosics/CoReg.py", line 1426, in correct_shifts
DS = DESHIFTER(self.shift, self.coreg_info,
File "/home/server/anaconda3/envs/fusion/lib/python3.7/site-packages/arosics/CoReg.py", line 1383, in coreg_info
self.calculate_spatial_shifts()
File "/home/server/anaconda3/envs/fusion/lib/python3.7/site-packages/arosics/CoReg.py", line 1296, in calculate_spatial_shifts
self._handle_error(RuntimeError('No match found in the given window.'))
File "/home/server/anaconda3/envs/fusion/lib/python3.7/site-packages/arosics/CoReg.py", line 359, in _handle_error
raise error
RuntimeError: No match found in the given window.
When I try to run the above command on the raw data, it runs successfully.
Could you help me with that issue ?
Thank you in advance !
Hello, I think I discovered a bug using AROSICS v1.0.3. When I use COREG_LOCAL
for geometric correction with q=True
or v=False
it still prints statements related to tie point filtering. I think the problem is due to instantiation of the Tie_Point_Grid
object in CoReg_local.py
in line 339.
I see that there is an input for q
but it is not being followed because Tie_Point_Refiner
in Tie_Point_Grid.py
line 416 only uses keyword arguments from outlDetect_settings
from CoReg_local.py
line 339 to instantiate the object but it does not include the q
parameter.
I managed to suppress the print statement properly by adding the q
parameter inside outlDetect_settings
dict parameter in line 339 Tie_Point_Grid
instantiation.
Hi all - Just stumbled over Arosics and am keen to see if it can help with my reqs - so to start off; should I be able to co-register two PNG files from an aerial survey? No geospatial info available but the docs do say this should be possible. I tried the CLI first with;
python3 bin_arosics_cli.py local 20201030-154404024.tif_despeckle.png 20201030-154404665.tif_despeckle.png 50
but I get;
Calculating actual data corner coordinates for reference image...
Shell is not a LinearRing
bin_arosics_cli.py:362: UserWarning:
First attempt to check if functionality of co-registration failed. Check your input data and parameters. The following error occurred:
parsed_args.func(parsed_args)
Traceback (most recent call last):
File "bin_arosics_cli.py", line 362, in <module>
parsed_args.func(parsed_args)
File "bin_arosics_cli.py", line 73, in run_local_coreg
CRL = COREG_LOCAL(args.path_ref,
File "/usr/local/lib/python3.8/site-packages/arosics/CoReg_local.py", line 233, in __init__
self.COREG_obj = COREG(self.imref, self.im2shift,
File "/usr/local/lib/python3.8/site-packages/arosics/CoReg.py", line 314, in __init__
self._get_image_params()
File "/usr/local/lib/python3.8/site-packages/arosics/CoReg.py", line 449, in _get_image_params
self.ref = GeoArray_CoReg(self.params, 'ref')
File "/usr/local/lib/python3.8/site-packages/arosics/CoReg.py", line 117, in __init__
_ = self.footprint_poly # execute getter
File "/usr/local/lib/python3.8/site-packages/geoarray/baseclasses.py", line 482, in footprint_poly
self._footprint_poly = self.box.mapPoly
File "/usr/local/lib/python3.8/site-packages/geoarray/baseclasses.py", line 340, in box
mapPoly = get_footprint_polygon(get_corner_coordinates(gt=self.geotransform, cols=self.columns, rows=self.rows))
File "/usr/local/lib/python3.8/site-packages/py_tools_ds/geo/vector/topology.py", line 89, in get_footprint_polygon
assert outpoly.is_valid, 'The given coordinates result in an invalid polygon. Check coordinate order.' \
File "/usr/local/lib/python3.8/site-packages/shapely/geometry/base.py", line 731, in is_valid
return bool(self.impl['is_valid'](self))
File "/usr/local/lib/python3.8/site-packages/shapely/predicates.py", line 24, in __call__
self._validate(this)
File "/usr/local/lib/python3.8/site-packages/shapely/topology.py", line 18, in _validate
raise ValueError("Null geometry supports no operations")
ValueError: Null geometry supports no operations
Shell is not a LinearRing
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.8/site-packages/arosics/CoReg.py", line 314, in __init__
self._get_image_params()
File "/usr/local/lib/python3.8/site-packages/arosics/CoReg.py", line 449, in _get_image_params
self.ref = GeoArray_CoReg(self.params, 'ref')
File "/usr/local/lib/python3.8/site-packages/arosics/CoReg.py", line 125, in __init__
if not self.footprint_poly.is_valid:
File "/usr/local/lib/python3.8/site-packages/shapely/geometry/base.py", line 731, in is_valid
return bool(self.impl['is_valid'](self))
File "/usr/local/lib/python3.8/site-packages/shapely/predicates.py", line 24, in __call__
self._validate(this)
File "/usr/local/lib/python3.8/site-packages/shapely/topology.py", line 18, in _validate
raise ValueError("Null geometry supports no operations")
ValueError: Null geometry supports no operations
Hi. I had to rebuild my docker container and I ended up upgrading and I'm getting the following warning printing continuously in the log:
/usr/local/lib/python3.6/dist-packages/arosics/CoReg.py:1383: UserWarning: SSIM input array shapes are not equal! This issue seemed to be already fixed..
My job then fails with:
File "process.py", line 125, in process
CRL.correct_shifts()
File "/usr/local/lib/python3.6/dist-packages/arosics/CoReg_local.py", line 754, in correct_shifts
self.calculate_spatial_shifts()
File "/usr/local/lib/python3.6/dist-packages/arosics/CoReg_local.py", line 437, in calculate_spatial_shifts
self._tiepoint_grid.get_CoRegPoints_table()
File "/usr/local/lib/python3.6/dist-packages/arosics/Tie_Point_Grid.py", line 400, in get_CoRegPoints_table
results = results.get()
File "/usr/lib/python3.6/multiprocessing/pool.py", line 644, in get
raise self._value
ValueError: Input images must have the same dimensions.
Any ideas on why this is happening? I'm having a hard time downgrading. Thanks!
Hello Daniel!
I'm starting to experiment with AROSICS. I've tried to co-registrate two images taken from a DataCube and as you say on your paper about AROSICS I'm applying that with a loop. I want to append the co-registered band to the same cube, and I got this Error:
IndexError: dimension coordinate 'lat' conflicts between indexed and indexing objects:
<xarray.DataArray 'lat' (lat: 224)>
array([13.78654 , 13.78663 , 13.78672 , ..., 13.806448, 13.806538, 13.806628])
Coordinates:
* lat (lat) float64 13.79 13.79 13.79 13.79 ... 13.81 13.81 13.81 13.81
Attributes:
standard_name: latitude
long_name: latitude
units: degrees_north
vs.
<xarray.IndexVariable 'lat' (lat: 224)>
array([13.78654 , 13.78663 , 13.78672 , ..., 13.806448, 13.806538, 13.806628])
Attributes:
units: degrees north
long_name: latitude
standard_name: latitude
To overcome that i set only values (with .values method) to my new co-registered band. Doing this means that I keep the lat/lon from original band.
My question about that is: to set values only is a good form to overcome it?
I decided to keep original lat/lon because if I compare it I got difference 0 (for example: original_lat - new_lat returns 0)
Thanks!
Hi Daniel,
I am trying to register two WorldView-2 images. Both images have cloud, so I am trying to use mask_baddata_ref and mask_baddata_tgt. I couldn't find a clear instruction for me on how to do this. So, I assign mask_baddata_ref and mask_baddata_tgt variables to the mask files in a specific folder. The mask files contain 0 and 1 value, with 1 as the mask (e.g, clouds) and 0 as other obejcts.
But in the results, some tie points are in the cloud.
Any help would be very much appreciated.
Thanks.
Firman.