Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Marc Pfister
    @drwelby
    The other hitch for me is that I would like to also summarize assets, so a client can know ahead of time about the assets that are stored by the collection's items.
    Chris Holmes
    @cholmes
    I think for that we have an extension https://github.com/radiantearth/stac-spec/tree/master/extensions/item-assets that could work
    Marc Pfister
    @drwelby
    Oh perfect! I feel like I saw that but then got it crossed up with the collection-assets.
    RobinF
    @rfergason
    @matthewhanson Hey. I would like to be added to the monthly STAC call invite. Is that call still open to anyone? Thanks!
    Matthew Hanson
    @matthewhanson
    Yes, it is! @rfergason , please just let me know your email address
    RobinF
    @rfergason
    Thanks @matthewhanson ! My email address is rfergason@usgs.gov.
    RobinF
    @rfergason
    Got in invite. Thank you!
    Chris Holmes
    @cholmes
    Hey, I'm just leaving now to drop the kid off at a new daycare. Probably won't be able to join a call for a half hour from now at the earliest. Feel free to meet without me, or push back a half hour (may still be a bit late). Sorry for late notice on this.
    Marc Pfister
    @drwelby
    @m-mohr what I'm trying to describe is also the output of a process, is there any overlap here with the openeo work?
    James Santucci
    @jisantuc
    is anyone free for a second review on radiantearth/stac-spec#875 ?
    Rob Emanuele
    @lossyrob
    done!
    James Santucci
    @jisantuc
    :+1: thanks!
    Matthias Mohr
    @m-mohr
    @drwelby Maybe, but likely not yet for Collections as we use them only for Input Data so far and Output is always an Item. But we have process descriptions in JSON Schema. Anyway, we should talk at some point (currently not at work until end of the week). Maybe at the data sprint? Are you participating?
    Marc Pfister
    @drwelby
    When is the data sprint? I'll try to make it.
    Tony Gill
    @gillt_gitlab

    Hi folks,

    I appreciate you are all busy with the current workshops. So there is no rush on my part for a response. Many months ago (it could have been a year ago, what a year!) I posted a question regarding using STAC to access offline images. In our case these are rasters that reside in a tape archive, and must be 'activated' (to borrow Planet terminology) before they're available for download.

    I am after some feedback on the following scheme for users to be able to order offline data. The vision we have is shown at the video on this page: https://medium.com/@mojodna/a-stac-browser-348a60674061. If we get this right, then:

    • a user will use a STAC browser to navigate a catalogue until they get to a StacItem
    • the browser renders the Assets for the StacItem as hyperlinks
    • the user can select an 'order' link to request the asset be activated
    • in response, the browser renders the StacItem again, but with some updated properties on the status of the order and a link that the user can select to check the progress
    • the user-browser interaction loops until the browser display's the StacItem's 'download' link, which the user selects to access the image.

    There is more detail about our scheme below for the interested reader. If you read it, then I'd be very grateful:

    • for people's thoughts on this scheme; example: is there a simpler approach? are we using the notion of an asset correctly?
    • to know if we need to propose a STAC extension (like an 'order' extension) to implement this scheme? Or does the existing STAC 1.0.0-beta support it

    Now to the detail. In our scheme, a StacItem has an 'order' asset, the URL to which is a request to our server to 'activate' the asset. The server responds with a 202 (processing) and a StacItem in the body. The StacItem's properties has an order:status field with the status of the order ('offline', 'processing', or 'online'). The returned StacItem's assets has a URL so the status can be queried again.

    The following example workflow may make the scheme clearer. Below is a snippet of a StacItem returned from a search result. Note:

    • The 'order' asset; it contains a link to our order endpoint so the file can be ordered
    • The 'order:status' attribute in the properties; it is currently offline
    {
      "id": "cemsre_t54jvn_20161118_abam4.img",
      "assets": {
        "metadata": {
          "href": "https://data-access.jrsrp.org.au/productmetadata"
        }
        "order": {
          "href": "https://stac.jrsrp.org.au/order?filenames=cemsre_t54jvn_20161108_abam4.img&state=NSW"
        }
      },
      ...
    
      "properties": {
        "collection": "nsw-Sentinel2-reflectance_10m",
         ...
        "order:status": "offline"
      },
    }

    A request is placed to order the file using the order-asset link. The response is a 202 (processing) and the StacItem is returned. Note:

    • the order:status attribute in the returned StacItem has changed to 'processing'
    "id": "cemsre_t54jvn_20161118_abam4.img",
    "assets": {
        "metadata": {
          "href": "https://data-access.jrsrp.org.au/productmetadata"
        }
        "order": {
          "href": "https://stac.jrsrp.org.au/order?filenames=cemsre_t54jvn_20161108_abam4.img&state=NSW"
        }
    
      ...
    
      "properties": {
        "collection": "nsw-Sentinel2-reflectance_10m",
        ...
        "order:status": "processing",
      }

    The client may use the order-asset link again to query the order's progress. When the file becomes available for download, the response will be code 200 with the body containing the StacItem as follows. Note:

    • there is now a 'download' asset
    • and the order:status is 'online'
    ...
    ...
    "id": "cemsre_t54jvn_20161118_abam4.img",
    "assets": {    
        "metadata": {
          "href": "https://data-access.jrsrp.org.au/productmetadata"
        }
        "download": {
          "href": "http://qld.auscover.org.au/public/data/spatial_other/jrsrp/jrsrp-data-access/order_506/cemsre_t54jup_20180224_abam4.img"
        }
    
      ...
      ...
    
      "properties": {
        "collection": "nsw-Sentinel2-reflectance_10m",
        ...
        "order:status": "online",
      }
    Matthias Mohr
    @m-mohr
    @gillt_gitlab I'd suggest copying this into a GitHub issue so that all your good thoughts don't get lost and ate visible more permanently.
    ycespb
    @ycespb
    @drwelby @cholmes , Hi, about "A schema as a collection asset would work too" (sorry for my late reply). What about using the same approach as used in a OGC API - Features Core Collection, i.e. adding a link with rel="describedby", type="application/schema+json", and href referring to the actual JSON schema for the items in the collection ?
    Matthias Mohr
    @m-mohr
    @ycespb Good call. That is also a possibility, although STAC Items usually have multiple Schemas (1 Item + 1 for each extension). But I guess nothing prevents us to use multiple links? If that's standardized in OAFeat Collection, we should go that route, too and could likely also add it to the spec. Do you have a pointer to where this is specified in OAFeat? I found some references of describedBy, bot nothing in particular that mentions it can be used for item schemas. From the docs, I'd expect that describedBy + type in a Collection refers to a Collection Schema, not an Item schema.
    Marc Pfister
    @drwelby
    Any ideas on a good way to describe the asset naming convention in item-assets? While there's no href, it would be helpful to know that the a file is named '{{properties.id}}-pan.tif'
    Chris Holmes
    @cholmes
    @gillt_gitlab - lots of really great stuff here. I have thoughts on this for sure. And I know Planet and Maxar share a need for a similar 'order' workflow. But I'm slammed with organizing the sprint. If you could put it in a github issue at https://github.com/radiantearth/stac-spec/issues like @m-mohr suggests that'll ensure I come back to it and comment on it, and it'll save the conversation for other interested parties. But looks like a great start.
    @ycespb @m-mohr - Agreed, reusing the Features API relations would be good, if we can sort out the best way to do it.
    Matthias Mohr
    @m-mohr
    @drwelby There's nothing like that yet. The closest you can get is likely the tiled asset extension, which has URI templates for their stuff.
    Tony Gill
    @gillt_gitlab
    Thank you @m-mohr and @cholmes. I have created issue 891. https://github.com/radiantearth/stac-spec/issues/891#issue-690674989.
    ycespb
    @ycespb
    @m-mohr : the pointer should be Recommendation 9 Point D - /rec/core/fc-md-descriptions in the spec at http://docs.opengeospatial.org/is/17-069r3/17-069r3.html. Point D mentions JSON schemas as example structural definitions to be included in this way.
    Bernard Wright
    @space_bernard_twitter
    Hi folks, Bernard from Geo Gecko here. Can someone help me figure out how does STAC apply to ground collected data which will later be used for imagery machine learning? Is there standards that we should be adhering to? Appreciate it
    ycespb
    @ycespb
    @m-mohr Example 4 in the same document 17-069r3 uses this approach (describedby) with a GML application schema which applies to all items in the collection.
    Chris Holmes
    @cholmes
    Welcome @space_bernard_twitter! The most closely related STAC stuff is the label extension: https://github.com/radiantearth/stac-spec/tree/master/extensions/label
    But I'm not sure it exactly fits, as it sounds like you're not yet associating it with an image? The label extension I believe needs to have an image and the label that applies to it.
    Actually it looks like only the geojson is required, see https://github.com/radiantearth/stac-spec/tree/master/extensions/label#assets @jisantuc @notthatbreezy probably have more insight than I do if ground collected data without an image is in scope.
    James Santucci
    @jisantuc
    "links to any source imagery" sounds like not having a source is Fine :tm:, though it sounds a little odd -- what do you mean "ground collected"? do you mean there is no corresponding imagery for some reason? @space_bernard_twitter
    Bernard Wright
    @space_bernard_twitter
    We're not associating it with an image yet. Its geojson'd field outlines, crop type and yield. Its a training dataset for others to link to relevant imagery.
    @cholmes @jisantuc
    Matthias Mohr
    @m-mohr
    @jbants @jonhealy1 I saw that you released STAC Validator 1.0.0 - Could you update https://github.com/m-mohr/stac-node-validator/blob/master/COMPARISON.md for the sprint, please? :-)
    Matthias Mohr
    @m-mohr
    Anyone has any insight on how to work with STAC and CARD4L? Their website says they are working on something, but no specifics.
    pieschker
    @pieschker
    @m-mohr I'm assuming CARD4L is supportive but not implementing anything yet.
    Matthew Hanson
    @matthewhanson
    We recently have done a CARD4L self assessment for the Sentinel-2 COG work, which is going into review now.
    Now that we’ve done that, I’d like to do an extension for it, if anyone else is interested, shooting to have something I can present on at ARD 20 for it.
    ycespb
    @ycespb
    Hi, I have a question related to staclint.com. When we validate our landing/root page (a STAC Catalog 1.0.0-beta2), it complains that "'extent' is a required property of []". Is my understanding not correct that the Landing Page (i.e. a STAC catalog) is not necessarily a STAC Collection and therefore may not have extent and providers which are properties of a STAC Collection, but not properties of a STAC Catalog ? I also checked the UML representation (draw.io / PDF) which seems to confirm that "extent" and "providers" are only properties of STAC Collections and not of STAC Catalogs. All Collections are Catalogs, but not all Catalogs are Collections in my understanding... Maybe staclint.com is only meant to validate STAC Collections and not STAC Catalogs ? Does somebody has experience with this tool ?
    Chris Holmes
    @cholmes
    @jbants can probably help. I agree that starting with a catalog should not require an extent.
    Tim Martin
    @TimJMartin
    Hi - new member here. I work at Ordnance Survey (GB's National Mapping Agency) and have started investigating us potentially using COGs and STACs for our raster and imagery data. We have about a dozen different products that are currently in different formats (TIF+World Files, GeoTiffs), they all have different refresh cycles, we have archives going back to 2001, and some are Open Data and some are Premium paid datasets. Having read some of the STAC spec and looked at a few examples I have a few questions on how best to store the data in S3 and a few other questions. Where would be best to ask those questions, here or over on the Github issue page? Many thanks Tim
    Matthias Mohr
    @m-mohr
    Welcome @TimJMartin! If it's going to be very long, it might be better to use GH issues, but otherwise I think you can just ask here.
    Tim Martin
    @TimJMartin
    @m-mohr ok cool. I will post a question on GH as it might help other beginners
    Alex Leith
    @alexgleith
    Hey @m-mohr please let me know if I can put you in touch with CARD4L people :-)
    Matthias Mohr
    @m-mohr
    @alexgleith Thanks, I'll get back to you soon :-)
    ycespb
    @ycespb
    Hey @lossyrob , does the STAC-BROWSER use/display the assets with role "overview" and "thumbnail" if both are present in an item or is only the "thumbnail" displayed ? In many cases, our thumbnails are 100px only, and we have in many cases quicklooks a bit larger than 600px (e;g. 700x400px)... Better to use the quicklooks and include them as "thumbnail" assets ? Any recommendation ?
    Rob Emanuele
    @lossyrob
    Only the “thumbnail” asset is displayed (on the thumbnail tab for the item). Is the overview also a png/jpeg that can be displayed in that tab? The browser could be tweaked to look for an “overview” role first and fall back to the “thumbnail” asset, if this is a common use case
    Rob Emanuele
    @lossyrob
    I’m not sure ‘overview’ is a common role (folks can correct me if I’m wrong). So maybe it’d be better to allow overridding of the thumbail asset key (defaulting to thumbnail) via an environment variable so that you can modify it on startup
    ycespb
    @ycespb
    @rob, the terminology used since OGC 10-157r4 for these two is THUMBNAIL and
    QUICKLOOK. (see https://docs.opengeospatial.org/is/10-157r4/10-157r4.html#28). Maybe these terms could be adopted as common "roles" in the spec ? Being able to overwrite the "thumbnail" asset key on startup would be OK as well... Would be nice if they would appear nicely in stacindex.org as well and not only in our instance of STAC-BROWSER...