Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Tom Kralidis
    @tomkralidis
    @cehbrecht around for a 0.20.0 release?
    MacPingu
    @cehbrecht
    @tomkralidis today? I have a telco in the next minutes … after that I should be available for half an hour ...
    Tom Kralidis
    @tomkralidis
    Excellent. I’ve staged it at geopython/OWSLib#694 for your review.
    MacPingu
    @cehbrecht
    Ok … I check it.
    @tomkralidis I’m ok with it :) Should I add the PR links to the changes?
    Tom Kralidis
    @tomkralidis
    sure, you can do that or make your own PR on CHANGES.rst and I’ll remove mine. Whatever is easier for you
    MacPingu
    @cehbrecht
    I can make the change on the existing PR branch.
    I will let you know when done.
    Tom Kralidis
    @tomkralidis
    kk
    MacPingu
    @cehbrecht
    @tomkralidis changes done.
    Tom Kralidis
    @tomkralidis
    thanks. Moving ahead.
    MacPingu
    @cehbrecht

    @tomkralidis I have a question about the catalog service csw. Is it possible to search for updated/new records? We harvest records from a CSW service and after the initial full request we would like to search only for updated and new records. This is a behaviour supported by OAI services … using the from parameter:
    https://sickle.readthedocs.io/en/latest/tutorial.html?highlight=from#using-the-from-parameter

    The current dev code for harvesting with CSW looks like this:
    https://github.com/cehbrecht/md-ingestion/blob/pingudev/mdingestion/ng/harvester/csw.py

    Tom Kralidis
    @tomkralidis
    @cehbrecht you’ll need to issue CSW requests with a filter (XML) like modified > YYYY-MM-DD type thing
    Tom Kralidis
    @tomkralidis
    so like (untested):
    from owslib import fes
    from owslib.csw import CatalogueServiceWeb
    
    filter_ = fes.PropertyIsGreaterThan(‘modified', ‘2020-05-10’)
    
    c = CatalogueServiceWeb(‘http://example.org/csw')
    
    c.getrecords2(constraints=[filter_])
    MacPingu
    @cehbrecht
    @tomkralidismodified is a field which is always available on a CSW? … or does this depend on the records schema?
    Tom Kralidis
    @tomkralidis
    @cehbrecht sorry, should be dct:modified, which is CSW core (see Subclause 10.2.5.3.2 and Table 53 of http://portal.opengeospatial.org/files/?artifact_id=20555)
    MacPingu
    @cehbrecht
    @tomkralidis ok. Thanks :)
    Luís de Sousa
    @ldesousa

    Hi folks. Some of my WCS notebooks stopped working with time outs. The services reply correctly to direct requests, but GetCoverage requests through OWSLib fail. Here is an example:

    from owslib.wcs import WebCoverageService

    wcs = WebCoverageService('http://maps.isric.org/mapserv?map=/map/phh2o.map',
    version='2.0.1')
    cov_id = 'phh2o_0-5cm_mean'
    ph_0_5 = wcs.contents[cov_id]
    subsets = [('X', -1784000, -1140000), ('Y', 1356000, 1863000)]
    crs = "http://www.opengis.net/def/crs/EPSG/0/152160"

    response = wcs.getCoverage(
    identifier=[cov_id],
    crs=crs,
    subsets=subsets,
    resx=250, resy=250,
    format=ph_0_5.supportedFormats[0])

    But the equivalent request directly sent through HTTP is successful:

    https://maps.isric.org/mapserv?map=/map/phh2o.map&
    SERVICE=WCS&
    VERSION=2.0.1&
    REQUEST=GetCoverage&
    COVERAGEID=phh2o_0-5cm_mean&
    FORMAT=GEOTIFF_INT16&
    SUBSET=X(-1784000,-1140000)&
    SUBSET=Y(1356000,1863000)&
    SUBSETTINGCRS=http://www.opengis.net/def/crs/EPSG/0/152160&
    OUTPUTCRS=http://www.opengis.net/def/crs/EPSG/0/152160

    Any clues?

    Tom Kralidis
    @tomkralidis

    hey @cehbrecht FYI for the OSGeo AGM, given OWSLib is a community project, any chance you/others can update the AGM deck at https://docs.google.com/presentation/d/1C6llSnWZ28c2aWQgttPiOnoo6dttdqKg07ugU_yr6Uc/edit#slide=id.g9401e4b2da_178_10 (currently slide 75).

    FYI AGM is this Thursday.

    Thanks

    MacPingu
    @cehbrecht
    @tomkralidis I have updated the owslib slide.
    Tom Kralidis
    @tomkralidis
    Thanks @cehbrecht
    Tom Kralidis
    @tomkralidis
    @cehbrecht are you attending the AGM today? Would you be able to talk to the OWSLib slide (slide 84)?
    MacPingu
    @cehbrecht
    @tomkralidis sorry … I’m not at AGM ...
    Tom Kralidis
    @tomkralidis
    ok np, just presented, thanks again for the input
    MacPingu
    @cehbrecht
    @tomkralidis thanks for taking care of this :)
    MacPingu
    @cehbrecht
    Has someone more experience with WFS to check this pull request?
    geopython/OWSLib#706
    @huard Can we add someone from your side to the maintainers of owslib? (ping @tomkralidis).
    David Huard
    @huard
    We're a bit stretched thin at the moment. We're currently hiring a new programmer. Give me a few weeks and I'll come back to you about this.
    Paul van Genuchten
    @pvgenuchten
    hi, dummy question here, i try to follow the instructions at https://github.com/geopython/OWSLib/blob/master/docs/en/index.rst#csw, for this line: csw.transaction(ttype='insert', typename='gmd:MD_Metadata', record=open("file.xml").read()), it seems to always throw "xml.etree.ElementTree.ParseError: mismatched tag: line 22, column 5", no matter what xml i try to feed it, any idea how to debug this?
    Tom Kralidis
    @tomkralidis
    Hi @pvgenuchten do you have a sample metadata to test?
    Tom Kralidis
    @tomkralidis
    @pvgenuchten fwiw, I tested with some metadata from WOUDC. So, using a local pycsw setup as well as a test metadata record from https://geo.woudc.org/csw?service=CSW&version=2.0.2&request=GetRepositoryItem&elementsetname=full&typenames=csw:Record&resulttype=results&id=urn:x-wmo:md:int.wmo.wis::https://geo.woudc.org/def/data/ozone/total-column-ozone/totalozone saved to woudc-total-ozone.xml , the following works for me :
    from owslib.csw import CatalogueServiceWeb
    
    f = 'woudc-total-ozone.xml'
    
    csw = CatalogueServiceWeb('http://localhost:8000')
    
    csw.transaction(ttype='insert', typename='gmd:MD_Metadata', record=open(f).read())
    Massimo Di Stefano
    @epifanio
    HI, trying to query a CSW (pycsw) endpoint using owslib - following the docs I tried the following query (which works):
    from owslib.csw import CatalogueServiceWeb
    from owslib.fes import PropertyIsEqualTo, PropertyIsLike, BBox
    
    endpoint = 'https://csw.epinux.com/'
    csw = CatalogueServiceWeb(endpoint,timeout=30)
    csw.version
    # prints out:
    #  '2.0.2'
    
    [op.name for op in csw.operations]
    # prints out:
    """
    ['GetCapabilities',
     'DescribeRecord',
     'GetDomain',
     'GetRecords',
     'GetRecordById',
     'GetRepositoryItem',
     'Transaction',
     'Harvest']
    """
    csw.getdomain('GetRecords.resultType')
    csw.results
    
    # prints out: 
    """
    {'type': 'csw:Record',
     'parameter': 'GetRecords.resultType',
     'values': ['hits', 'results', 'validate']}
    """
    finally a query:
    sentinel_query = PropertyIsEqualTo('csw:AnyText', 'sentinel')
    csw.getrecords2(constraints=[sentinel_query], maxrecords=20)
    csw.results
    # prints out:
    """
    {'matches': 442291, 'returned': 10, 'nextrecord': 11}
    """

    and:

    for rec in csw.records:
        print(csw.records[rec].title)

    returns the expected values.

    however, when I run the following:

    bbox_query = BBox([-141,42,-52,84])
    csw.getrecords2(constraints=[sentinel_query, bbox_query])

    it returns the following exception:

    ---------------------------------------------------------------------------
    ExceptionReport                           Traceback (most recent call last)
    <ipython-input-138-0a44e99fbd69> in <module>
    ----> 1 csw.getrecords2(constraints=[sentinel_query, bbox_query])
    
    /usr/lib/python3/dist-packages/owslib/csw.py in getrecords2(self, constraints, sortby, typenames, esn, outputschema, format, startposition, maxrecords, cql, xml, resulttype)
        377             self.request = node0
        378 
    --> 379         self._invoke()
        380 
        381         if self.exceptionreport is None:
    
    /usr/lib/python3/dist-packages/owslib/csw.py in _invoke(self)
        709         val = self._exml.find(util.nspath_eval('ows:Exception', namespaces))
        710         if val is not None:
    --> 711             raise ows.ExceptionReport(self._exml, self.owscommon.namespace)
        712         else:
        713             self.exceptionreport = None
    
    ExceptionReport: 'Invalid query syntax'
    Massimo Di Stefano
    @epifanio
    seems the bbox query has an invalid syntax, any clue?
    Tom Kralidis
    @tomkralidis
    @pvgenuchten not sure what the issue issues, testing worked with the nationaalgeoregister metadata as well :|
    @epifanio works for me without issues, what version of OWSLib are you using?
    tomkralidis @tomkralidis one of those days
    Massimo Di Stefano
    @epifanio
    @tomkralidis '0.20.0'
    root@874f513999fd:~# pip3.8 install -U owslib
    Requirement already up-to-date: owslib in /usr/lib/python3/dist-packages (0.20.0)
    WARNING: You are using pip version 20.2.2; however, version 20.2.3 is available.
    You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.
    root@874f513999fd:~#
    Tom Kralidis
    @tomkralidis
    @epifanio can you turn on logging for the pycsw endpoint and traceback
    Massimo Di Stefano
    @epifanio
    how to do thatg?
    :) i should know
    Tom Kralidis
    @tomkralidis
    lol
    Massimo Di Stefano
    @epifanio
    it was already enabled, this is a snippet of it: https://gist.github.com/0cee2e5190944188bc97dfa7d108197b
    seems a postgis issue: psycopg2.errors.UndefinedFunction: function get_spatial_overlay_rank(text, unknown) does not exist
    can it be?
    Tom Kralidis
    @tomkralidis
    looks like a pycsw config issue. Let’s discuss elsewhere (given this is not an OWSLib issue)
    Massimo Di Stefano
    @epifanio
    ok :) thanks!!!