Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 17 23:41

    tomkralidis on eo_model

    add eo:processingLevel, update … (compare)

  • Jan 17 17:21

    tomkralidis on eo_model

    complete attribute group (compare)

  • Jan 17 16:38

    tomkralidis on eo_model

    add singleChar to attribute gro… (compare)

  • Jan 17 16:36

    tomkralidis on eo_model

    apply EO queryables to OGC Filt… (compare)

  • Jan 16 03:11

    tomkralidis on eo_model

    first pass OpenSearch queryable… (compare)

  • Jan 15 20:06

    kalxas on eo_model

    Using --build-arg for developme… (compare)

  • Jan 15 16:01

    kalxas on eo_model

    Switching to OWSLib master unti… (compare)

  • Jan 15 02:19

    tomkralidis on eo_model

    parse incoming EO fields (compare)

  • Jan 13 13:01

    kalxas on fix-python3-interp

    (compare)

  • Jan 13 13:01

    kalxas on master

    update Python / pip interpreter… Merge pull request #641 from ge… (compare)

  • Jan 13 13:01
    kalxas closed #641
  • Jan 13 12:13

    tomkralidis on gh-pages

    update copyright year (compare)

  • Jan 12 14:33

    kalxas on eo_model

    Adding EO related fields in the… (compare)

  • Jan 11 17:44

    kalxas on master

    This group of columns is here f… (compare)

  • Dec 13 2020 14:32

    tomkralidis on gh-pages

    add FAQ on server.url setup (compare)

  • Dec 12 2020 15:45
    tomkralidis commented #641
  • Dec 12 2020 14:17
    tomkralidis review_requested #641
  • Dec 12 2020 14:17
    tomkralidis opened #641
  • Dec 12 2020 14:16

    tomkralidis on fix-python3-interp

    update Python / pip interpreter… (compare)

  • Dec 07 2020 10:38

    tomkralidis on gh-pages

    10 years -- Happy birthday pycs… update copyright, links (compare)

Ricardo Garcia Silva
@ricardogsilva
keeping it in config files (or as environment variables) makes it easier to automate installs
lets think about it some more and try to come up with a spec for this
Tom Kralidis
@tomkralidis
k
Ricardo Garcia Silva
@ricardogsilva
in the wiki perhaps?
Tom Kralidis
@tomkralidis
+1.
ok, I'll take that action
Ricardo Garcia Silva
@ricardogsilva
are mostly done then?
Tom Kralidis
@tomkralidis
anything else to add? Next meeting in 4 weeks? Say 06 March?
Ricardo Garcia Silva
@ricardogsilva
I can write up the minutes from the meeting, like last time
yes, 06 march seems OK
Tom Kralidis
@tomkralidis
thanks @ricardogsilva!
thanks and have a good one ttyl
Ricardo Garcia Silva
@ricardogsilva
bye bye
happy sprinting!
Tom Kralidis
@tomkralidis
thanks! let's try to move pyfes forward this week!
Ricardo Garcia Silva
@ricardogsilva
fyi, notes from the second gitter meeting are available at https://github.com/geopython/pycsw/wiki/Meetings-2017-02-06 please review/edit them at your discretion
Tom Kralidis
@tomkralidis
thanks @ricardogsilva can you send to pycsw-devel mailing list if you have time?
Tom Kralidis
@tomkralidis
@ricardogsilva for pyfes I added a task to review existing implementations https://github.com/geopython/pyfes/projects/1#card-1627023
there may be others
Tom Kralidis
@tomkralidis
@ricardogsilva can we remove the requirements-pg.txt include in https://github.com/geopython/pycsw/blob/master/requirements-dev.txt#L3 ? It's tripping up non PG workflows for me. Do we need this in the dev requirements?
Tom Kralidis
@tomkralidis
or is there some flag to 'turn off' PG based testing ?
Angelos Tzotsos
@kalxas
hi all
sorry for missing the meeting, crazy day, forgot about it
will catch up from logs
Tom Kralidis
@tomkralidis
hi all: great first day at code sprint; I had numerous pycsw discussions which I would summarize as requests for supporting multiple backends (ES/SOLR, other HTTP APIs, etc.). I would say pyfes is job 1 at this point.
Ricardo Garcia Silva
@ricardogsilva

@tomkralidis not requiring requirements-pg.txt in requirements-dev.txt is not something I thought of while implementing it. My rationale was that usually tests would be run for all backends so I was expecting all dependencies to be installed.

In order to be able to skip on requirements-pg.txt we'd need to have some sort of conditional import of psycopg2 in tests/functionaltests/conftest.py. It is doable, but requires a bit of additional work.

If you just want to disable testing pg stuff when running tests, with tox you can run tox --env py27-sqlite --env py34-sqlite in a similar way as is done on pycsw's .travis.yml file. You'd still need to have requirements-pg.txt installed though...

If this is a showstopper for you, please open a ticket and assign me to it

Ricardo Garcia Silva
@ricardogsilva
@tomkralidis thanks for the research on other FES implementations, I'll look into them too, when I have the time. Did you read the architecture sketches page that I've posted on the wiki?
Tom Kralidis
@tomkralidis
@ricardogsilva thanks for the info. The issue I ran into is that my dev env cannot install psycopg2 probably because I don't have underlying PG libs. So as a poor dev I need to be able to test without PG on my system :)
Tom Kralidis
@tomkralidis
ok, filed: geopython/pycsw#497
@ricardogsilva for pyges I had a look at https://github.com/geopython/pyfes/wiki/Architecture-sketches and have a bunch of q's
Tom Kralidis
@tomkralidis
like what's the difference between a filter parser and a query parser?
Ricardo Garcia Silva
@ricardogsilva
  • A filter parser is a parser for the FILTER_LANGUAGE (using the KVP parameter name) specified in the query. In CSW we typically use two different filter languages: FES and CQL. So I guess we'd need a filter parser for FES stuff and another for CQL stuff in pyfes.

  • A query parser is a parser for the whole query. In the FES spec, a query is composed of (again using the KVP parameter names) TYPENAMES, PROPERTYNAMES, FILTER and SORTBY. So a query parser needs to know how to extract this stuff from the request. Also, a query parser may use more than one filter parser. This may happen for example in a hypothetical parser trying to parse a CSW GetRecords request, where either one of FES or CQL might be used as the filter language

I guess this is rather dense... Am I making any sense to you?

Tom Kralidis
@tomkralidis
it's dense territory :) we're getting there :)
I spent a big part of yesterday thinking about the backends. Question is how much real filter processing does a backend need to do?
Ricardo Garcia Silva
@ricardogsilva
can you elaborate on that?
Tom Kralidis
@tomkralidis
e.g. currently pycsw.ogc.fes.fes1 parses the query into an SQL where clause. pyfes would abstract that away instead into some object/thing. So then would a backend need to crawl through the pyfes object with similar work to be able to construct the relevant query in the backend's space?
Ricardo Garcia Silva
@ricardogsilva
yeah, I guess so. The rationale being that pyfes could abstract the actual query language in use (which could be FES, CQL or other) into a common query format
this means that the backend still has to do the work of translating the query into its own domain
Tom Kralidis
@tomkralidis
yeah. There is a slippery slope between that and a backend just processing the FES XML straight. But then again expecting a backend to process FES XML is heavy/won't scale.
Ricardo Garcia Silva
@ricardogsilva
ideally we'd be able to make pyfes easier to use than directly parsing the XML (or other language) and thus more attractive
Tom Kralidis
@tomkralidis
agree.
so a backend would process a pyfes type?
Ricardo Garcia Silva
@ricardogsilva
that is why I'd like to have a simple API for pyfes clients, with just using some parse(some_data) function that would take care of instatiating these more esoteric query and filter parser objects in the backseat and it would return the already digested fes type for the backend to process afterwards
so something like:

fes_stuff = pyfes.parser.parse(my_xml_data, **additional_arguments)
records = repository.perform_pyfes_query(fes_stuff)
or close to that
Tom Kralidis
@tomkralidis
exactly
Ricardo Garcia Silva
@ricardogsilva
those additional_aguments could be stuff to customize, e.g. some validation on ValueReference and Literal expression types, maybe a custom etree parser (hardened against XML attacks), etc
Tom Kralidis
@tomkralidis
and the serializers to convert back?
Ricardo Garcia Silva
@ricardogsilva
I think those would be more useful for clients of an OGC service (like owslib), where the user could specify a query by using CQL (as is the case in geotools) or by creating the fes types directly, and then serializing to XML before sending a request to the server