Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    MacPingu
    @cehbrecht
    @tomkralidis I have updated the owslib slide.
    Tom Kralidis
    @tomkralidis
    Thanks @cehbrecht
    Tom Kralidis
    @tomkralidis
    @cehbrecht are you attending the AGM today? Would you be able to talk to the OWSLib slide (slide 84)?
    MacPingu
    @cehbrecht
    @tomkralidis sorry … I’m not at AGM ...
    Tom Kralidis
    @tomkralidis
    ok np, just presented, thanks again for the input
    MacPingu
    @cehbrecht
    @tomkralidis thanks for taking care of this :)
    MacPingu
    @cehbrecht
    Has someone more experience with WFS to check this pull request?
    geopython/OWSLib#706
    @huard Can we add someone from your side to the maintainers of owslib? (ping @tomkralidis).
    David Huard
    @huard
    We're a bit stretched thin at the moment. We're currently hiring a new programmer. Give me a few weeks and I'll come back to you about this.
    paul van genuchten
    @pvgenuchten
    hi, dummy question here, i try to follow the instructions at https://github.com/geopython/OWSLib/blob/master/docs/en/index.rst#csw, for this line: csw.transaction(ttype='insert', typename='gmd:MD_Metadata', record=open("file.xml").read()), it seems to always throw "xml.etree.ElementTree.ParseError: mismatched tag: line 22, column 5", no matter what xml i try to feed it, any idea how to debug this?
    Tom Kralidis
    @tomkralidis
    Hi @pvgenuchten do you have a sample metadata to test?
    Tom Kralidis
    @tomkralidis
    @pvgenuchten fwiw, I tested with some metadata from WOUDC. So, using a local pycsw setup as well as a test metadata record from https://geo.woudc.org/csw?service=CSW&version=2.0.2&request=GetRepositoryItem&elementsetname=full&typenames=csw:Record&resulttype=results&id=urn:x-wmo:md:int.wmo.wis::https://geo.woudc.org/def/data/ozone/total-column-ozone/totalozone saved to woudc-total-ozone.xml , the following works for me :
    from owslib.csw import CatalogueServiceWeb
    
    f = 'woudc-total-ozone.xml'
    
    csw = CatalogueServiceWeb('http://localhost:8000')
    
    csw.transaction(ttype='insert', typename='gmd:MD_Metadata', record=open(f).read())
    epifanio
    @epifanio
    HI, trying to query a CSW (pycsw) endpoint using owslib - following the docs I tried the following query (which works):
    from owslib.csw import CatalogueServiceWeb
    from owslib.fes import PropertyIsEqualTo, PropertyIsLike, BBox
    
    endpoint = 'https://csw.epinux.com/'
    csw = CatalogueServiceWeb(endpoint,timeout=30)
    csw.version
    # prints out:
    #  '2.0.2'
    
    [op.name for op in csw.operations]
    # prints out:
    """
    ['GetCapabilities',
     'DescribeRecord',
     'GetDomain',
     'GetRecords',
     'GetRecordById',
     'GetRepositoryItem',
     'Transaction',
     'Harvest']
    """
    csw.getdomain('GetRecords.resultType')
    csw.results
    
    # prints out: 
    """
    {'type': 'csw:Record',
     'parameter': 'GetRecords.resultType',
     'values': ['hits', 'results', 'validate']}
    """
    finally a query:
    sentinel_query = PropertyIsEqualTo('csw:AnyText', 'sentinel')
    csw.getrecords2(constraints=[sentinel_query], maxrecords=20)
    csw.results
    # prints out:
    """
    {'matches': 442291, 'returned': 10, 'nextrecord': 11}
    """

    and:

    for rec in csw.records:
        print(csw.records[rec].title)

    returns the expected values.

    however, when I run the following:

    bbox_query = BBox([-141,42,-52,84])
    csw.getrecords2(constraints=[sentinel_query, bbox_query])

    it returns the following exception:

    ---------------------------------------------------------------------------
    ExceptionReport                           Traceback (most recent call last)
    <ipython-input-138-0a44e99fbd69> in <module>
    ----> 1 csw.getrecords2(constraints=[sentinel_query, bbox_query])
    
    /usr/lib/python3/dist-packages/owslib/csw.py in getrecords2(self, constraints, sortby, typenames, esn, outputschema, format, startposition, maxrecords, cql, xml, resulttype)
        377             self.request = node0
        378 
    --> 379         self._invoke()
        380 
        381         if self.exceptionreport is None:
    
    /usr/lib/python3/dist-packages/owslib/csw.py in _invoke(self)
        709         val = self._exml.find(util.nspath_eval('ows:Exception', namespaces))
        710         if val is not None:
    --> 711             raise ows.ExceptionReport(self._exml, self.owscommon.namespace)
        712         else:
        713             self.exceptionreport = None
    
    ExceptionReport: 'Invalid query syntax'
    epifanio
    @epifanio
    seems the bbox query has an invalid syntax, any clue?
    Tom Kralidis
    @tomkralidis
    @pvgenuchten not sure what the issue issues, testing worked with the nationaalgeoregister metadata as well :|
    @epifanio works for me without issues, what version of OWSLib are you using?
    tomkralidis @tomkralidis one of those days
    epifanio
    @epifanio
    @tomkralidis '0.20.0'
    root@874f513999fd:~# pip3.8 install -U owslib
    Requirement already up-to-date: owslib in /usr/lib/python3/dist-packages (0.20.0)
    WARNING: You are using pip version 20.2.2; however, version 20.2.3 is available.
    You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.
    root@874f513999fd:~#
    Tom Kralidis
    @tomkralidis
    @epifanio can you turn on logging for the pycsw endpoint and traceback
    epifanio
    @epifanio
    how to do thatg?
    :) i should know
    Tom Kralidis
    @tomkralidis
    lol
    epifanio
    @epifanio
    it was already enabled, this is a snippet of it: https://gist.github.com/0cee2e5190944188bc97dfa7d108197b
    seems a postgis issue: psycopg2.errors.UndefinedFunction: function get_spatial_overlay_rank(text, unknown) does not exist
    can it be?
    Tom Kralidis
    @tomkralidis
    looks like a pycsw config issue. Let’s discuss elsewhere (given this is not an OWSLib issue)
    epifanio
    @epifanio
    ok :) thanks!!!
    epifanio
    @epifanio
    that worked!
    Tom Kralidis
    @tomkralidis
    :+1:
    epifanio
    @epifanio
    demo in the readme
    TimeSeriesProfile is the less boring
    epifanio
    @epifanio
    quick question about filtering .. with owslib .. if I use this syntax in the constraints : csw.getrecords2(constraints=[sentinel_query, bbox_query]) i will get the union as a results
    how to get an intersection :) ?
    between the 2 query: [sentinel_query, bbox_query]
    Tom Kralidis
    @tomkralidis
    epifanio
    @epifanio
    +100
    epifanio
    @epifanio
    testing query .. I can't get my head around the box query

    why thius query:

    from owslib.csw import CatalogueServiceWeb
    from owslib.fes import PropertyIsEqualTo, PropertyIsLike, BBox
    
    endpoint='https://csw.epinux.com'
    csw = CatalogueServiceWeb(endpoint,timeout=30)
    
    bbox=[2,50,15,70]
    bbox_query = BBox(bbox)
    
    equalTo_query = [PropertyIsEqualTo('csw:AnyText', 'sentinel')]
    
    csw.getrecords2(constraints=[equalTo_query, bbox_query], maxrecords=20)

    returns 10 records
    while this one:

    csw.getrecords2(constraints=[bbox_query], maxrecords=20)

    returns 0

    epifanio
    @epifanio
    for the first query: csw.results gives me: {'matches': 442291, 'returned': 10, 'nextrecord': 11}
    for the second one 0
    and also: csw.getrecords2(constraints=equalTo_query, maxrecords=20) compared with csw.getrecords2(constraints=[equalTo_query, bbox_query], maxrecords=20)
    returns the same result .. so bbox is not taking place
    Tom Kralidis
    @tomkralidis
    @epifanio this works (note the double list notation is now in the getrecords2 function:
    from owslib.csw import CatalogueServiceWeb
    from owslib.fes import PropertyIsEqualTo, PropertyIsLike, BBox
    
    endpoint='https://csw.epinux.com'
    csw = CatalogueServiceWeb(endpoint,timeout=30)
    
    bbox=[2,50,15,70]
    bbox_query = BBox(bbox)
    
    equalTo_query = PropertyIsEqualTo('csw:AnyText', 'sentinel')
    
    csw.getrecords2(constraints=[[equalTo_query, bbox_query]], maxrecords=20)
    print(csw.request)
    print(csw.results)
    
    csw.getrecords2(constraints=[bbox_query], maxrecords=20)
    print(csw.request)
    print(csw.results)
    epifanio
    @epifanio

    thanks! I think I am getting there, thanks for the working example :) - now dealing with the extraction of the extent for each dataset - I got it with something like:

    for rec in csw.records:
        bbox = vars(csw.records[rec].bbox)

    which I'm gonna use to create a geojson feature.
    I am not quite sure about how to work with the generator returned by the query.
    I can get the first 10 records .. but how to access the next in the results? Do I need to repeat the query by changing the startposition parameter .. until I get all of them?

    say, csw.results gives me:

    {'matches': 274536, 'returned': 10, 'nextrecord': 11}

    I can get the values for the first "returned" 10 elements, by looping over it, like:

    for rec in csw.records:
        appendJsonFeature(name = csw.records[rec].identifiers, 
                                                   geom = vars(csw.records[rec].bbox))

    How to get the next 10 ,, and so on .. ?
    do I need to re-issue the query, like:

    startposition = csw.results['nextrecord']
    csw.getrecords2(constraints=[[equalTo_query, bbox_query]], maxrecords=20, esn='full', startposition=startposition)

    until startposition reaches the 'matches' records?

    epifanio
    @epifanio
    Got it :)