Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    BigGrainEnergy
    @BigGrainEnergy
    I can send some through in a bit, thanks for that!
    BigGrainEnergy
    @BigGrainEnergy
    For example, when looking at the 2015 Taan Fiord landslide in Alaska
    2 replies
    image.png
    The area of interest is marked by the red dot (indicated by red arrow)
    I have selected the SBAS tool in the above screenshot^
    Joseph H Kennedy
    @jhkennedy
    @BigGrainEnergy perfect! Thanks! I'll look at it tomorrow morning and get back to you soon
    BigGrainEnergy
    @BigGrainEnergy
    Thanks @jhkennedy , it's much appreciated!
    Joseph H Kennedy
    @jhkennedy
    :thumbsup:
    GabiZuccolotto
    @GabiZuccolotto
    Hello! I'm currently working on a groundwater withdrawal and land subsidence project using InSAR. At first, I was attempting to download the Sentinel-1 IW SLC images from ASF Vertex and then processing them using the Sentinel-1 SNAP program. On my MAC desktop this process takes a very long time and then I recently discovered HyP3! Yesterday I was able to send over some SLC pairs using the SBAS tools and was very impressed by the speed of processing, the interferograms and vertical displacement images that I got back. However, I am still just a little unclear on what steps are being taken to process the interferograms (i.e. topographic phase removal, filtering, etc.). I'd like to compare the steps that I am taking using SNAP to the steps that ASF takes. Is there somewhere I could find a detailed explanation on how the interferograms and deformation maps are being generated?
    James Rine
    @Jlrine2

    HI @GabiZuccolotto we have a product guide that you are more then welcome to check out, it's located at https://hyp3-docs.asf.alaska.edu/guides/insar_product_guide/

    If it doesn't have the answers you are looking for then please let us know

    Ryan Avery
    @rbavery

    Hi all, I'm curious if it is advisable/possible in the future to request RTC Sentinel-1 with a Terrain Correction Pixel Spacing (m) of 10 meters instead of 30 meters (even if the Copernicus 30 meter DEM is used)? I'm thinking that in this case, the 30 meter DEM would be resampled to 10 meter for the correction.

    My thinking is this would bring in some assumptions of slope at that resolution that may or may not be true, but could lead to higher res/higher quality data in areas without substantial topography differences, so maybe having the option would be valuable?

    James Rine
    @Jlrine2
    @rbavery thanks for the feedback, how long is this to run 10 m products is on our backlog, unfortunately I don't have an estimate for when it might actually be done. Do you have a specific use case in which you would like to use 10 m or RTC?
    Argh... Typos...
    We have it in our backlog to allow users to run 10m products, but unfortunately I don't have an estimate for when it might be done
    Joseph H Kennedy
    @jhkennedy
    @rbavery as James is saying, we do have 10m RTC in our backlog to provide (a popular user request!) but we don't have a timeline (mostly due to trying to secure stable funding to provide them as they'll cost significantly more to process and store). It's possible we could provide them in the near team at cost, but we'd need to chat more about that (we've mostly supported grants currently with custom processing/deployments)
    Ryan Avery
    @rbavery
    Got it, that's helpful! My specific use case right now is comparing the ASF product to an existing 10 meter product. Longer term we're interested in using a 10 meter product for near-real time deforestation detection and possibly near-real time oil slick detection
    Joseph H Kennedy
    @jhkennedy
    @rbavery if it's just a handful of products, I could generate the 10m ones and give you the s3 links. We'll def. let you know when we start making 10m available generally
    Ryan Avery
    @rbavery
    oh that would be so awesome Joseph! These are the granules I'm looking at
    S1B_IW_GRDH_1SDV_20190102T141245_20190102T141316_014316_01AA2B_6D78
    S1B_IW_GRDH_1SDV_20200109T141251_20200109T141322_019741_02552C_F6F8
    S1B_IW_GRDH_1SDV_20210103T141258_20210103T141328_024991_02F96E_EFDC
    S1A_IW_GRDH_1SDV_20190111T161259_20190111T161324_025432_02D14C_2B50
    S1A_IW_GRDH_1SDV_20200118T161305_20200118T161330_030857_038A73_D4FC
    S1A_IW_GRDH_1SDV_20210112T161311_20210112T161336_036107_043B8E_E752
    S1B_IW_GRDH_1SDV_20190102T084921_20190102T084950_014313_01AA16_5582
    S1B_IW_GRDH_1SDV_20200109T084927_20200109T084956_019738_025516_724C
    S1B_IW_GRDH_1SDV_20210103T084933_20210103T085002_024988_02F958_E58F
    S1B_IW_GRDH_1SDV_20190112T070657_20190112T070722_014457_01AEBA_CE7B
    S1B_IW_GRDH_1SDV_20200107T070703_20200107T070728_019707_025425_2339
    S1B_IW_GRDH_1SDV_20210101T070709_20210101T070734_024957_02F867_1112
    S1B_IW_GRDH_1SDV_20190104T231244_20190104T231309_014351_01AB40_D0FA
    S1B_IW_GRDH_1SDV_20200111T231250_20200111T231315_019776_025643_788C
    S1B_IW_GRDH_1SDV_20210105T231256_20210105T231321_025026_02FA86_628D
    S1B_IW_GRDH_1SDV_20190127T174852_20190127T174917_014683_01B5F0_5A7F
    S1B_IW_GRDH_1SDV_20200110T174859_20200110T174924_019758_0255B5_5B9F
    S1B_IW_GRDH_1SDV_20210104T174905_20210104T174930_025008_02F9F4_DF69
    Does that count as a handful?
    Joseph H Kennedy
    @jhkennedy
    Ryan Avery
    @rbavery
    sweet. Can we use these options?
    speckle_filter=True,  include_rgb=True, radiometry="gamma0", scale = "power", dem_matching=True, resolution=10
    @jhkennedy and I added just now that the output res should be 10
    Joseph H Kennedy
    @jhkennedy

    @rbavery submitted! You can watch for them to finish with this script:

    import hyp3_sdk as sdk
    
    job_ids = [
        '0f850ede-7cf2-45b7-bcd7-0c907f20d932',
        '604ed554-368d-4119-a7bf-9d607ace4ce2',
        '1b3243f5-fe28-4ec1-94f6-fb46d2388b8e',
        '52645c9e-dc7a-48d9-9736-acd2c630410a',
        '0c0e0b11-ae1e-459e-a30e-b0cbba961f0a',
        '6dac1e03-cd3a-4a3a-886c-4d7e58d5d9ab',
        '77063ba9-f539-47a1-a954-9a12540e2535',
        'a0491555-e7f5-4d1d-95a1-3e7183bc036e',
        '17c68c67-8cbc-4f6c-aa73-fb9baa20faa3',
        '12ee1451-b5f2-481e-8cbe-79a2e13cb244',
        'a5f2f57b-615e-4b92-9a3a-ffe78fe6e2c0',
        '83ef4b6e-32c4-4e23-9ee8-b1432e8706ba',
        '7c889f37-0f08-4802-8cd4-2d5bb440ed0a',
        '32f7393b-d7f0-4b24-b15e-8a01074b9722',
        '39cd0e25-94f1-4ebd-83d0-85b35ac42085',
        '0d6bdf48-e7fb-47cd-bdd0-cf24b4e7f7b0',
        'c6997015-9b2b-40d9-80df-3e787900fb09',
        'da4df95a-4b79-4185-a9b8-b1ef78c11f4f',
    ]
    
    hyp3 = sdk.HyP3('https://hyp3-watermap.asf.alaska.edu', prompt=True)
    
    jobs = sdk.Batch()
    for ii in job_ids:
        jobs += hyp3.get_job_by_id(ii)
    
    jobs = hyp3.watch(jobs)
    
    # download
    jobs.download_files('data/')
    # print urls
    for job in jobs:
        print(job.files[0]['url'])

    of if you prefer, I can just post the URLs when they are done

    Ryan Avery
    @rbavery
    awesome! running this now, I can download these. Thanks for the help @jhkennedy
    Joseph H Kennedy
    @jhkennedy
    :thumbsup:
    Ryan Avery
    @rbavery
    I noticed these jobs failed, they're all co-located in Columbia for the same granule bounds. All the other jobs passed. I think we can get by without using this area for comparison but just wanted to flag this.
    Job.from_dict({'job_id': '7c889f37-0f08-4802-8cd4-2d5bb440ed0a', 'job_type': 'RTC_GAMMA', 'request_time': '2021-09-07T22:57:58+00:00', 'status_code': 'FAILED', 'user_id': 'jhkennedy', 'name': 'rbavery-10m-rtc', 'job_parameters': {'dem_matching': True, 'granules': ['S1B_IW_GRDH_1SDV_20190104T231244_20190104T231309_014351_01AB40_D0FA'], 'radiometry': 'gamma0', 'resolution': 10, 'scale': 'power', 'speckle_filter': True}, 'logs': ['https://hyp3-watermap-contentbucket-1rg5gnumqc8vw.s3.us-west-2.amazonaws.com/7c889f37-0f08-4802-8cd4-2d5bb440ed0a/7c889f37-0f08-4802-8cd4-2d5bb440ed0a.log'], 'expiration_time': '2021-09-23T00:00:00+00:00'}),
     Job.from_dict({'job_id': '32f7393b-d7f0-4b24-b15e-8a01074b9722', 'job_type': 'RTC_GAMMA', 'request_time': '2021-09-07T22:57:58+00:00', 'status_code': 'FAILED', 'user_id': 'jhkennedy', 'name': 'rbavery-10m-rtc', 'job_parameters': {'dem_matching': True, 'granules': ['S1B_IW_GRDH_1SDV_20200111T231250_20200111T231315_019776_025643_788C'], 'radiometry': 'gamma0', 'resolution': 10, 'scale': 'power', 'speckle_filter': True}, 'logs': ['https://hyp3-watermap-contentbucket-1rg5gnumqc8vw.s3.us-west-2.amazonaws.com/32f7393b-d7f0-4b24-b15e-8a01074b9722/32f7393b-d7f0-4b24-b15e-8a01074b9722.log'], 'expiration_time': '2021-09-23T00:00:00+00:00'}),
     Job.from_dict({'job_id': '39cd0e25-94f1-4ebd-83d0-85b35ac42085', 'job_type': 'RTC_GAMMA', 'request_time': '2021-09-07T22:57:58+00:00', 'status_code': 'FAILED', 'user_id': 'jhkennedy', 'name': 'rbavery-10m-rtc', 'job_parameters': {'dem_matching': True, 'granules': ['S1B_IW_GRDH_1SDV_20210105T231256_20210105T231321_025026_02FA86_628D'], 'radiometry': 'gamma0', 'resolution': 10, 'scale': 'power', 'speckle_filter': True}, 'logs': ['https://hyp3-watermap-contentbucket-1rg5gnumqc8vw.s3.us-west-2.amazonaws.com/39cd0e25-94f1-4ebd-83d0-85b35ac42085/39cd0e25-94f1-4ebd-83d0-85b35ac42085.log'], 'expiration_time': '2021-09-23T00:00:00+00:00'}),
    Joseph H Kennedy
    @jhkennedy
    @rbavery Thanks! Looks like they all hit the same segfault and crashed out 🤔. I'm re-running them at 30m just to see if they work at that resolution.
    And @rbavery we'd love any feedback you have on the 10m products! We've focused mostly on 30m products so any critical eye on them is apreciated.
    Ryan Avery
    @rbavery
    @jhkennedy for sure, I'll definitely send feedback
    thanks for checking out the 30m!
    Joseph H Kennedy
    @jhkennedy
    @rbavery They did work at 30m, do you want the links to the 30m products?
    Ryan Avery
    @rbavery
    ah interesting. yes those links would be great, thanks!
    Joseph H Kennedy
    @jhkennedy
    @rbavery here's the job ids and product links for the 30m ones!
    84e073d2-d8cc-4e2d-b804-6839c90fc1f2:https://hyp3-watermap-contentbucket-1rg5gnumqc8vw.s3.us-west-2.amazonaws.com/84e073d2-d8cc-4e2d-b804-6839c90fc1f2/S1B_IW_20210105T231256_DVP_RTC30_G_gpufem_7A91.zip
    6dfd43e7-e535-4213-8fe6-7813a00d1560:https://hyp3-watermap-contentbucket-1rg5gnumqc8vw.s3.us-west-2.amazonaws.com/6dfd43e7-e535-4213-8fe6-7813a00d1560/S1B_IW_20190104T231244_DVP_RTC30_G_gpufem_0BA6.zip
    8cc4b01a-879b-4f9f-ae8d-a52e2300b931:https://hyp3-watermap-contentbucket-1rg5gnumqc8vw.s3.us-west-2.amazonaws.com/8cc4b01a-879b-4f9f-ae8d-a52e2300b931/S1B_IW_20200111T231250_DVP_RTC30_G_gpufem_A7DE.zip
    Ryan Avery
    @rbavery
    Hi Joseph! Would you be able to run one more 10 meter job without dem_matching if you get a moment this week? just for this granule S1B_IW_GRDH_1SDV_20190102T141245_20190102T141316_014316_01AA2B_6D78
    and with these parameters:
    speckle_filter=True,  include_rgb=True, radiometry="gamma0", scale = "power", dem_matching=False, resolution=10
    We've noticed that the other product we are comparing to has a roughly 40 meter offset compared to the ASF 10 meter product and wanted to check if this is because of dem_matching or not
    Joseph H Kennedy
    @jhkennedy
    @rbavery Done! Here's the job id:
    8bc42004-5125-40f9-8ee3-f837a9c30bb0
    @rbavery In general, we get better results if dem_matching=False because the ESA precices orbits are such high quality. Caveat: we haven't looked seriously at that comarison in a year-ish
    Ryan Avery
    @rbavery
    awesome, that's cool to hear. do you have a sense of when one would want to use DEM_matching at all then?
    Heidi Kristenson
    @hjkristenson

    @rbavery We haven't done a deep dive on the impacts of DEM matching on products generated with the Copernicus DEM, but when using the legacy DEMs we do not recommend using the DEM matching for products that would be used in a time series analysis. DEM matching can shift the geolocation of the products, and the shifts are not always consistent from one RTC product to the next. This results in features not aligning well when RTC images are stacked through time.

    In some cases, RTC results of individual products are improved when using the DEM matching, because it ensures that the slopes in the image align well with the DEM. If you don't match the DEM and there are geolocation offsets between the image and the DEM, the slopes may be a bit offset between the image and the DEM, so the normalization of the backscatter and the alignment of the ridges might not be as good as if DEM matching is applied.

    For most time series analysis (with the legacy DEMs), we found that the geolocation shifts introduced by DEM matching were more of a detriment to analysis than the reduction in RTC quality that results from not matching. We left the option available for those who are less interested in time series analysis, and are looking at potentially maximizing the RTC quality when looking at a particular image.

    We thought that perhaps with the Copernicus DEM there wouldn't be as much of an offset, but a 40m offset is quite significant. Was the issue resolved by having the DEM matching option disabled? With the legacy DEMs, we found that offsets were more likely to occur in areas that were lacking distinctive topographic features. Without clear features to match to the DEM, features can be matched to the DEM incorrectly; that may still be the case with the Copernicus DEM in areas of low relief.

    For older missions, DEM matching often improved the geolocation of the output product, but because the orbit information for Sentinel-1 is so precise, the geolocation of the S1 data is often better than the legacy DEMs. It seems like the Copernicus DEM has better geolocation, which suggests that the DEM matching should work better, but perhaps that's also a reason not to bother with DEM matching; if both the image and the DEM have good geolocation, the features should align well even without matching.

    Matt Parno
    @mparno

    Hi Everyone! I'm new to using HyP3 and have a question about what types of granules are supported. I'd like to process a GRD_MD file over the Lincoln sea (the specific granule is S1B_EW_GRDM_1SDH_20190909T141720_20190909T141825_017962_021D12_8083). However, it doesn't look like it is possible to process that granule on Vertex or using the SDK. When I try to submit the granule using the SDK I get a 400 Client Error message. On vertex, it looks like all of the GRD_HD files can be processed, but not the GRD_MD files.
    Is that right?

    Are there specifics types of granules that are not supported by Hyp3? Thanks so much for your help. This looks like a fantastic tool and I'm excited to become more familiar with it.

    James Rine
    @Jlrine2
    Thank you for the feedback. Currently we only support SLC files for InSAR processing and we support SLC and GRD_HD files for RTC processing. Would you mind expanding on your usecase for GRD_MD products?
    4 replies
    Oh, and I see you are looking at an EW scene, currently we only support beammode IW.
    Rodrigo Gomez Fell
    @RGFell
    Hi everyone, I just discover HYP3 and start adding it to my workflow. I previously used gamma for speckle tracking and InSAR. I was wondering if the Autorift processing (or post-precessing) has any filtering (low pass)? My other question is if is on the roadmap to give more flexibility to the autorift processing. It would be awesome to be able to try different window sizes for example. Cheers and thanks.
    3 replies
    dstesser
    @dstesser

    I have been using HYP3 over the past several years and update order inventories every few months for a forest degradation project, ROI the lowland Choco of Ecuador. I no longer see Sentinel-1 10 m product availability for RTC GAMMA processing. This is a topographically complex region so the higher resolution data is preferred as I temporal filter the time series data. Is the 10 meter product feature available for this project?

    I have used HYP3 to obtain ~550 scenes at 30 m and would like to reprocess ~300 of these scenes at 10 m for comparison. Thank you for your guidance . HYP3 is a godsend

    James Rine
    @Jlrine2
    Thank you for the feedback. Unfortunately we do not have 10m availability at this time, It is something we would like to do in the future. Feel free to follow us on twitter as we will announce it there once it is available.
    Joseph H Kennedy
    @jhkennedy
    Twitter is here: https://twitter.com/ASFHyP3
    Or you can check our newsfeed: https://hyp3-docs.asf.alaska.edu/whats_new/
    GabiZuccolotto
    @GabiZuccolotto
    Screen Shot 2021-10-07 at 5.16.04 PM.png
    Hello! I'm currently trying to do a time series ground deformation analysis using HYP3 and MintPy. I was able to download my image pairs using the SBAS HYP3 tool, but I am struggling to clip the files to a smaller geographic location. MintPy suggests clipping the images before processing and demonstrates this in their example data directory as shown below. Does anyone know how best to do this? I'm not sure if it's something that can be done through HYP3 or if it has to be done through Python somehow. Thanks!
    cirrusasf
    @cirrusasf
    We have the notebook about how to do MintPy analysis with HyP3 product. I hope you enjoy it. https://github.com/cirrusasf/hyp3-mintpy/blob/notebook/docs/prep_TS_hyp3.ipynb