ASFHyP3/community

Alaska Satellite Facility's Hybrid Pluggable Processing Pipeline

Andres Alonso PhD

@andalonrd

Hello, I tried to do the analyses, but the server got clogged, I submitted close to 1k scenes for processing but nothing happened, except that my quota was consumed.

could you help me with that?

1 reply

Jiageng

@Arctic-Ambrun

Hello everyone, I want to ask a simple question. I want to develop the hyp3_sdk to avoid the temporal and spatial baseline limitation in the vertex

Will it be possible that only use metadata to calculate the spatial baseline?

Joseph H Kennedy

@jhkennedy:matrix.org

[m]

@Arctic-Ambrun the asf_search Python package has a baseline_search which calculates the spatial baseline for a stack (based on the reference scene). A terse example of using that function and pandas to build an SBAS stack is here:
https://gist.github.com/jhkennedy/e9e7ec353b2a05419e50413368ff0505

All (well, most) the metadata is available via asf_search and would let you build pairs/filter however you like. asf_search and Vertex is developed by ASF's Search/Discovery team, and you can also ask for features or chat with them here:
https://gitter.im/ASFDiscovery/community

2 replies

Jiageng

@Arctic-Ambrun

Genius work, great! Thanks

Joseph H Kennedy

@jhkennedy:matrix.org

[m]

@Arctic-Ambrun: sorry, I mixed up the baselines. Yes, there's currently no good way to get at the spatial baselines with asf_search; that should be coming soon but I don't know what the timeline is.

Jiageng

@Arctic-Ambrun

@Arctic-Ambrun: sorry, I mixed up the baselines. Yes, there's currently no good way to get at the spatial baselines with asf_search; that should be coming soon but I don't know what the timeline is.

yeah i checked the code, and seems we can only calculate it via the Iw.xml. So is it possible to download these xmls without data? I`d like to help coding

Jiageng

@Arctic-Ambrun

@Arctic-Ambrun: sorry, I mixed up the baselines. Yes, there's currently no good way to get at the spatial baselines with asf_search; that should be coming soon but I don't know what the timeline is.

yeah i checked the code, and seems we can only calculate it via the Iw.xml. So is it possible to download these xmls without data? I`d like to help coding

And I want to ask a question also. How did ASF-VERTEX returns the spatial baseline data? and the spatial baseline is different with a pair when changing the reference, do you mind telling me the reason? THanks in advance.!

Jiageng

@Arctic-Ambrun

hello guys! I checked the isce baseline tool, seems the info are stored in the iw.xml file. So I want to ask, is there any method to access the annotation folder only? which will be helpful to processing data. Thanks in advance!

Sarah Brennand

@SarahBrennand

I've got an AOI that covers 3 Sentinel-1 frames. I'm looking at creating a long InSAR time-series (may2015-dec2021) for these frames but also want to use a longer temporal threshold to maximise all possible reliable interferograms for my SBAS network. I've calculated that the number of interferogram pairs to process are 4,500. As this is significantly higher than then 250 quota, can I get a temporary increase of 2,250 per month for Jan-Feb? My username is sarbee26.

2 replies

Jiageng

@Arctic-Ambrun

Hello Mr @jhkennedy I`m creating a stack with 563 Pairs. Can I get a temporary increase of 600 this month? My username is chance_zhang . Thanks in advance

2 replies

GabiZuccolotto

@GabiZuccolotto

Hello! Has anyone used ASF HyP3 products for a MintPy analysis and successfully converted the .h5 result files into something compatible with QGIS or Google Earth? I'm trying to figure out how I could work in the ut.to_latlon() function into MintPy's save_kmz.py or save_qgis.py scripts rather than attempting to convert the original HyP3 files into lat/long. However, I am newer to python and am struggling a bit. Thanks in advance!

5 replies

Rodrigo Gomez Fell

@RGFell

Hi @jhkennedy, I got short by 130 on the quota of the month. Is there any chance to get those extra this month? Thanks in advance (my user is rgomezfell)

1 reply

GabiZuccolotto

@GabiZuccolotto

Hi @jhkennedy! I downloaded some hyp3 gamma products a few weeks ago and the downloaded items were accidentally deleted from my computer. I tried to re-download from the project that I created in hyp3, but it now says that the products are expired. Is there anyway to get those same files back? Thanks!

2 replies

Heidi Kristenson

@hjkristenson

Sarah Brennand

@SarahBrennand

Hi, I've had some really good results processing a two large stacks of AFSF HyP3 products in MintPy and it was really easy to use in comparison with SNAP input products. I've now been asked to do the same thing for another study that compliments my main study. I had asked for a temporary increase this month to 2,250 jobs, and I've used most of these jobs, but I need to run another 1,750 jobs. Can I get 1,750 added to this month's quota? My username is sarbee26

5 replies

Jiageng

@Arctic-Ambrun

Hello! I'm using the GAMMA On Demand to study volcano eruption. During the Descending and Ascending, it is calculated to be 240 + 307 Scenes, So the number of Pairs is estimated to be 2211 + 2469 jobs. now I have 1000 jobs left, Can I get extra 4000 jobs added this month? My username is ambrun. Thanks in advance!

2 replies

Sai Deepak

@SaiDeep01058696_twitter

Hi @jhkennedy:matrix.org Hello everyone. I am using ASF Hyp3 products for my area of interest to generate a time series. But my problem is my area of interest is covered by two acquisitions of sentinel-1. So i dont know how to get a single image in order to get the time series for the area of interest alone. It would be really helpful, if someone can tell me how to do it or provide some resources which mentions the answer to the problem. Thank you.

4 replies

Sai Deepak

@SaiDeep01058696_twitter

Sai Deepak

@SaiDeep01058696_twitter

Hello everyone, after using smallbaselineApp.py in MintPy, i am getting around 20 output files but is there any file which explains what does those outputs actually means and my kmz file is not opening. Can some one tell a solution for that. Thank you.

1 reply

rbrb

@rbrb:matrix.org

[m]

Hi,
I have a question to the ASF frame and path naming. I have downloaded S1 data from ASF GUI for two different years. It is a while ago and now I realized, that I have a different amount of data (~100scenes) for both years. My guess is that I had downloaded different path/frames for these years. I like to identify the path and frame of a downloaded S1 SLC scence from its name or its metadata. However, since the frame and path is ASF-specific I have not figured out how to do that. Can you please give me a hint how I can figure out the path and frame of a ASF downloaded S1 SLC scene. after download (from its name or metadata). Thanks a lot!

1 reply

Jiageng

@Arctic-Ambrun

What`s the function to transfer the Path, frame and Absolute Orbit number?

1 reply

Jiageng

@Arctic-Ambrun

Is it possible to merge two DInSAR product together without merge the SLCs and process manually?

3 replies

Jiageng

@Arctic-Ambrun

Hello everyone, may I ask two question here.I`m using the SBAS tools to analyse a eruption of volcano, Unfortunatly, Some of my process failed, Which troubles me to make next step with Mintpy.

Here I want to ask two question:
First, why some process failed, Could anyone please tell me the reason?
Eg: Reference scene: S1A_IW_SLC1SSV_20160804T114946_20160804T115022_012450_01372F_C4E2
Secondary Scene: S1A_IW_SLC1SSV_20160909T114948_20160909T115023_012975_014884_E67D
Second, what kind of parameter in GAMMA should I use, to achieve the same result as that does in ASF?

9 replies

nochwat

@nochwat

Hello, I ran my "normal" workflow for processing S1 velocities using the baseline and Autorift feature on Vertex. All the image pairs showed up as "failed" so I tried again the next day and they also failed. Does anyone know what the problem is? It has been working fine for several months now until this week. Thanks!

11 replies

Yitian Cheng

@ayanori

Hello everyone! May I ask is it possible to use asf_search python package to add custom pairs into a generated stack? (just like what we do in the vertex interface with mouse clicks)

2 replies

Gregory Short

@glshort

Hi @ayanori! Currently, asf_search only provides basic stacking functionality, similar to what Vertex does in the Baseline search. SBAS is effectively an extension of that (looping over the products in the stack and creating a list of pairs), so while asf_search doesn't yet provide that pair creation functionality, it should be possible to put together a loop to create the desired pairs.

3 replies

It's worth noting that the ASFSearchResults object returned by the various stacking methods can be treated as a list, which makes iterative processes like SBAS pairing fairly painless.

That being said, I'll get some SBAS pairing functionality on our backlog, we have some team members who are looking to pick up more asf_search features!

Jiageng

@Arctic-Ambrun

@jhkennedy Hello Mr.Kennedy, I`m planning to do a search about the WOLF volcano erpution, east galapagos islands (another part). May I apply for 5000 extra jobs? (3000ASCENDING + 3000 DESCENDING), approcimate. Thank you very much

5 replies

Yitian Cheng

@ayanori

Hi Specialists! I have a technical question here: in the InSAR processing script you're using, does the unwrapping happen before or after the water mask?

2 replies

Yitian Cheng

@ayanori

Thank you! I have another question about the water mask: how can it be reduced to be much closer to the shore? (right now it's a several km wide swath away from the shore)

2 replies

Ursha

@ursha

Hello colleagues. I wonder if there are any developments/News on downloading Sentinel 1 SLC data by burst. Sentinel 1 SLC data is quite big and wonder if there are any capabilities to download data of only specific bursts?

Lori Schultz

@lasschultz

Hello! Question about the new release of asf-tools on Github - is this automatically updated on OSL in the environments folder, or does this need to be done by the user?

Eric Lundell

@emlundell

@nasasport In general (like on the DAAC OSL), users are responsible for updating their conda environments. If we updated all the time for users, we would eventually break someone's work.

To update, you could rerun the Create_OSL_Conda_Environments.ipynb notebook or do more manual changes to your environments. The environments are usually stored within your home directory so that any customizations carry over from one server session to another.

We try to keep the current environments reflected in GitHub at https://github.com/ASFOpenSARlab/opensarlab-envs. Searching for asf_search (https://github.com/ASFOpenSARlab/opensarlab-envs/search?q=asf_search) shows that it is not pinned so the latest on conda forge build will be pulled in on build or update.

Eric Lundell

@emlundell

As as to asf_tools specifically, you can do conda install -c conda-forge asf_tools on the command line or !conda install -c conda-forge asf_tools (don't forget the exclamation mark) within a notebook (within the right kernel/environment). Again, any changes with the conda environments will be semi-permanent and carry over your server sessions.

Yitian Cheng

@ayanori

Hello! May I ask what is the algorithm we are using in the phase unwrapping part? After unwrapping mintpy seems to show me some weird signal that can not be found in the original 2-path interferograms. Maybe we can do some tests for it?

2 replies

Sayyed Mohammad Javad Mirzadeh

@SMJMirza

Hello! I will mention an issue and will be glad to know your solution. As you know, the HyP3 products are in the UTM coordinate system. When we use the products for the MintPy package and try to generate the KMZ files from the UTM-based MintPy velocity maps, we cannot open them on Google Earth. We thought this was due to the UTM coordinate system. So, we tried to convert the four corners of files from the UTM to lat/long and generate the KMZ files. When we open the new files on Google Earth, we see the distortions and offsets in the files. I can see the KMZ format files of HyP3 products in the downloaded jobs. Could you please let me know how you generate these KMZ files from the UTM-based HyP3 products?

Joseph H Kennedy

@jhkennedy

@SMJMirza we use hyp3lib.resample_geotiff.resample_geotiff to create the KML files. We typically call it like:

  resample_geotiff(geotiff, browse_width, 'KML', geotiff.replace('.tif', ''), use_nn=True)

There's a lot of code there, it's largely untested and it looks like that file hasn't been modified in two years, but it seems to be working. Let me know if there's an issue with it!

mmanthey1994

@mmanthey1994

Hi all, I have a 'soft question' concerning the general use of Sentinel 1 scenes, courtesy of ASF as processed with HyP3. I am doing some research on landslide prediction. I am hoping to use the vertical displacement maps provided for a timeseries analysis - i.e. look at how a given location's displacement tracks over time and flag a red alert if it seems to be steadily (or exponentially) dropping for instance. I have read the product guide and various other information which outlines the many error sources inherent in this technology, many of which are hard to quantify. I would be really interested to hear more about the suitability of such data for my purpose. Would anyone here have expertise in this and be willing to chat with me, or know of some other platform/forum/uni-academic which may be helpful?

1 reply

Yitian Cheng

@ayanori

Hi all, I have a question that might not be directly related with ASFHyp3: I have a wrapped interferogram (a geotiff file) generated by ASF Hyp3 and I'd like to do the phase unwrapping locally. What might be the best tool to use here?

5 replies

guitambe

@guitambe

Hello all,
First of all, thank you very much in advance for your help!
I'm a standard Linux-Debian user, not a programmer nor a Python knowledgest, so I have some general doubts and confusions whn trying to work with MintPy.
I'm struggling to perform a time series analysis with ASF-Vertex and MintPy.
I installed conda, MintPy and HyP3 in my linux computer. I downloaded all the interferogram pairs from Vertex and cropped them to my small AOI with cutGeotiffs.py in the linux console as indicated here:
https://mintpy.readthedocs.io/en/latest/dir_structure/
I also consulted this example, which as far as I can understand, follows the same procedure I would like to apply:
https://zenodo.org/record/5502403#.YpyFNzVByWs
I also found this notebooks which I tried to use as a reference:
https://nbviewer.org/github/ASFHyP3/hyp3-docs/blob/main/docs/tutorials/hyp3_insar_stack_for_ts_analysis.ipynb
https://nbviewer.org/github/insarlab/MintPy-tutorial/blob/main/smallbaselineApp_hyp3.ipynb

But I'm unable to figure out how to start the process to get the time series results. I can’t understand the following aspects:
1- how to get most of the configuration file parameters (RidgecrestSenDT71.txt in the example).
2 - in which “environment” do I must run the scripts. Shall I do work in the linux shell? Or must I get into some programming environment or specific shell like Conda, Python, HyP3 or something else? How do I get into the specific environment? Do I must be sited in a specific path? How do I call to the MintPy scripts?
3 - In short, I can’t figure out how to run smallbaselineApp.py
By other side, following the jupyter notebooks, and if I’m not wrong (highly possible), I realize that every time that I follow the “0 Initial setup” step, I’m installing a sort of a virtual machine in Conda (or every Conda environment is itself a sort of a virtual machine), which multiplies the installation of common linux packages in my computer such as gdal, alsa, hdf5, sql, etc. Is that so?
Well, sorry for throwing such general questions in the forum, and excuse me if my English is not quite clear. I will really thank you for any help you can give me.
Best regards!

8 replies

Leo German

@Leogerman_gitlab

Hi! I´m trying to run the cutGeotiffs.py script, but i don't now how to enter the parameters. I really will apreciate any help that you can give me about that. Thanks!!

8 replies

guitambe

@guitambe

Really disappointing, finally I got MintPy working and after two days downloading the ERA5 files, now the service is down for around the last 24 hours :(

"CDS is currently having some technical issues and data downloads might fail or take longer than usual. Sorry about inconveniences."

Any idea to avoid this problem?

3 replies

Krish Matta

@krishxmatta

How do I get access to HyP3 watermap? I keep getting told that I have a job quota of 0. I am trying to run this notebook from asf-tools: https://github.com/ASFHyP3/asf-tools/blob/develop/prototype/water-extent-map-on-demand.ipynb

1 reply

rohanrama

@rohanrama

Upon trying to open my openSARlab server, I get a message displayed saying [Normal] FailedPostStartHook. While I have been able to access the server before, this new error doesn't seem to go away if I restart my browser or switch computers. How could I get past this error message and access my asf server? I am working for NASA ACCESS19-stream-flow and I was told someone could attempt to reset my server?

1 reply

shayana-venukanthan

@shayana-venukanthan

Hi all, I'm having issues logging into my Earthdata account through ASF. I am running into issues running the cell under 2.1 Use the HyP3 SDK to connect to the HyP3-watermap deployment in this notebook from asf-tools:
https://github.com/ASFHyP3/asf-tools/blob/develop/prototype/water-extent-map-on-demand.ipynb
I've verified that I'm entering the correct username and password and have tried restarting the kernel and editing my Earthdata profile. Any suggestions?

This is the full error message:
HTTPError Traceback (most recent call last)
File ~/.local/envs/asf-tools/lib/python3.9/site-packages/hyp3_sdk/util.py:77, in get_authenticated_session(username, password)
76 try:
---> 77 response.raise_for_status()
78 except requests.HTTPError:

File ~/.local/envs/asf-tools/lib/python3.9/site-packages/requests/models.py:1021, in Response.raise_for_status(self)
1020 if http_error_msg:
-> 1021 raise HTTPError(http_error_msg, response=self)

HTTPError: 401 Client Error: Unauthorized for url: https://auth.asf.alaska.edu/login?error=access_denied&error_msg=Pre%20authorization%20required%20for%20this%20application,%20please%20authorize%20by%20visiting%20the%20resolution%20url&resolution_url=https://urs.earthdata.nasa.gov/approve_app?client_id=BO_n7nTIlMljdvU6kRRB3g

During handling of the above exception, another exception occurred:

AuthenticationError Traceback (most recent call last)
Input In [4], in <cell line: 3>()
1 import hyp3_sdk
----> 3 hyp3_watermap = hyp3_sdk.HyP3('https://hyp3-watermap.asf.alaska.edu', prompt=True)

File ~/.local/envs/asf-tools/lib/python3.9/site-packages/hyp3_sdk/hyp3.py:41, in HyP3.init(self, api_url, username, password, prompt)
38 if password is None and prompt:
39 password = getpass('NASA Earthdata Login password: ')
---> 41 self.session = get_authenticated_session(username, password)
42 self.session.headers.update({'User-Agent': f'{hyp3_sdk.name}/{hyp3_sdk.version}'})

File ~/.local/envs/asf-tools/lib/python3.9/site-packages/hyp3_sdk/util.py:79, in get_authenticated_session(username, password)
77 response.raise_for_status()
78 except requests.HTTPError:
---> 79 raise AuthenticationError('Was not able to authenticate with credentials provided\n'
80 'This could be due to invalid credentials or a connection error.')
81 else:
82 response = s.get(AUTH_URL)

AuthenticationError: Was not able to authenticate with credentials provided
This could be due to invalid credentials or a connection error.

1 reply

Magdalena88

@Magdalena88

Hi all! I was successfully processing HyP3 time series data using Mintpy, but in one case where my area is located at UTM 25S the final result in Mintpy located the area in the wrong geographical location. Any similar experiences?

3 replies

Where communities thrive

People

Repo info

Activity