Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jun 18 18:18
    epifanio edited #695
  • Jun 18 18:17
    epifanio opened #695
  • Jun 15 10:21
    tomkralidis closed #692
  • Jun 15 10:21
    tomkralidis commented #692
  • Jun 15 03:59
    scku208 commented #692
  • Jun 15 01:45
    scku208 edited #692
  • Jun 14 17:25
    tomkralidis commented #692
  • Jun 14 17:24
    tomkralidis milestoned #694
  • Jun 14 17:24
    tomkralidis labeled #694
  • Jun 14 17:24
    tomkralidis edited #694
  • Jun 14 17:24
    tomkralidis edited #694
  • Jun 14 17:23
    tomkralidis assigned #692
  • Jun 14 17:23
    tomkralidis milestoned #692
  • Jun 14 17:23
    tomkralidis labeled #692
  • Jun 14 17:21

    kalxas on fix-url

    (compare)

  • Jun 14 17:21

    kalxas on master

    fix URL settings Merge pull request #694 from ge… (compare)

  • Jun 14 17:21
    kalxas closed #694
  • Jun 14 17:18
    tomkralidis opened #694
  • Jun 14 17:18

    tomkralidis on fix-url

    fix URL settings (compare)

  • Jun 14 07:09
    CL-SHLOMIKONCHA closed #693
epifanio
@epifanio
i need some of my collegues to run some test on my testing endpoint .. but seems I no longer have the error in initializing the db
but I also added a postgresql.conf exposed as volume - with the instruction suggested by pgtune -- @tomkralidis thank you so much for the tips!
jmckenna
@freenode_jmckenna:matrix.org
[m]
@tomkralidis: will fix backport....
my fault
thanks man!
epifanio
@epifanio
@tomkralidis testing the new addition for reindexing
thanks!
epifanio
@epifanio
works great! just saw there is also the optimization -- would it make sense to have this exposed in the transaction API ?
Tom Kralidis
@tomkralidis
what do you mean?
epifanio
@epifanio
use the pcsw server transaction to execute the same commands available in pycsw_admin .. to avoid log-in in the container to perform such action
Tom Kralidis
@tomkralidis
Certainly doable as extended operations. How would envision security?
epifanio
@epifanio
transaction enabled already allows to delete a record, right? so I don't see any extra risk in enablig reindexing/cacooum/analyze - as long as it is restricted to certain IP. in my use case thereserved IP will be frm inside a vpn, so I consider it enogh safe
Tom Kralidis
@tomkralidis

transaction enabled already allows to delete a record, right? so I don't see any extra risk in enablig reindexing/cacooum/analyze - as long as it is restricted to certain IP. in my use case thereserved IP will be frm inside a vpn, so I consider it enogh safe

Good points, then the question becomes do CSW-T operations have the same security level/concept?

In a pure DB security sense, i.e. if one allows transactions (insert/update/delete) to a database to a given user, should it also allow that same user role the ability to vacuum/reindex?

epifanio
@epifanio
@tomkralidis working oin this now- quick question about transaction

I am using (pseudo code) to index, the following:

    headers = CaseInsensitiveDict()
    headers["Content-Type"] = "application/xml"
    headers["Accept"] = "application/xml"
    header = """<?xml version="1.0" encoding="UTF-8"?><csw:Transaction xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:ows="http://www.opengis.net/ows" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-publication.xsd" service="CSW" version="2.0.2"><csw:Insert>"""
    footer = """</csw:Insert></csw:Transaction>"""
    xml_as_string = header + xml_as_string + footer
    requests.post(url, headers=headers, data=xml_as_string.encode("utf-8"))

How can I pass the equivalent of -y flag from pycsw_admin.py ? so to overwrite a record with same ID - at the moment I get ERROR: duplicate key value violates unique constraint "records_pkey"

epifanio
@epifanio
i guess i need to use "Update" instead of "Insert"
epifanio
@epifanio
but I am still getting the same error log
epifanio
@epifanio
I am probably improperly using the Update .. as i need to specify the field to Update using ogc filters - I may try a delete and insert
epifanio
@epifanio
delete and then update works fine
Tom Kralidis
@tomkralidis
Tom Kralidis
@tomkralidis
Hi everyone: happy to report that pycsw will be presenting at this year’s FOSS4G conference : https://twitter.com/tomkralidis/status/1403712666032553985
Angelos Tzotsos
@kalxas
:+1:
epifanio
@epifanio

HI, I recently built QGIS frm src and noticed I can't load my CSW endpoints (it works with the default endpoints from the QGIS src)

thew error I got is:

Unknown Error: 'Authentication' object has no attribute '_auth_delegate'

thew endpoint https://testing.csw.met.no/

Tom Kralidis
@tomkralidis
That looks like an OWSLib issue. Try installing OWSLib master as part of your QGIS setup?
epifanio
@epifanio
I will check once at home, this was with QGIS master on my own workstation, using owsib from pip
  • from the office, same csw endpoint, QGIS from repo- it loads correctly
I now tried to build a new CSW image which should be in synk with master - I see it has a new interface as landing page

https://testing.csw.met.no/

attemting to access the swagger API:

https://testing.csw.met.no/openapi?f=html

I got:

Errors
Hide

Fetch errorFailed to fetch http://testing.csw.met.no/openapi?f=json
Fetch errorPossible mixed-content issue? The page was loaded over https:// but a http:// URL was specified. Check that you are not attempting to load mixed content.
Anything I need to fix in the csw.cfg maybe?
Tom Kralidis
@tomkralidis
Try changing the URL to HTTPS?
epifanio
@epifanio
I ss in the config the main url to server was with http, I'll fix that and try it out - @tomkralidis I confirm metasearch in QGIS master works fine installing owslib master 0.25-dev
paul van genuchten
@pvgenuchten
hi gitter, what is the convention for logging to stdout (as common in docker setups), when i set loglevel to debug, but no logfile, pycsw complains:
  File "/home/pycsw/pycsw/pycsw/core/log.py", line 71, in setup_logger
    raise RuntimeError(
RuntimeError: Invalid server configuration (server.loglevel set,                but server.logfile missing).
paul van genuchten
@pvgenuchten
image.png
ah, i found answer in docker folder
paul van genuchten
@pvgenuchten
bit stuck here, i deployed the latest pycsw docker image, but any url i try /pycsw, /pycsw/csw.py, /pycsw/pycsw/csw.py throws 404, in logs i see requests arriving, any idea what may be happening
image.png
image.png
sorry the problem is on my side, we routed pycsw requests to be in a subfolder, but it expects to run in root
paul van genuchten
@pvgenuchten
is there an option to indicate pycsw to run in a subpath?
i think in pygeoapi we use: environment-var SCRIPT_NAME=/pygeoapi
paul van genuchten
@pvgenuchten
mmm, i actually got it to work using
environment:
      - SCRIPT_NAME=/pycsw/csw.py
paul van genuchten
@pvgenuchten
but https://apitestbed.geonovum.nl/pycsw still throws a 500 for me, ideas?
paul van genuchten
@pvgenuchten
So https://apitestbed.geonovum.nl/pycsw/csw.py/collections/metadata:main/items/urn:uuid:94bc9c83-97f6-4b40-9eb8-a8e8787a5c63?f=xml returns a csw:record, is that intentional or is it stored like that in the database, what happens if that record where a iso19115 record, would it return the iso document?
image.png
indeed
Tom Kralidis
@tomkralidis
cool! yes, it will return the native representation (as ingested in the catalogue)
paul van genuchten
@pvgenuchten
Does/will pycsw have schema negotiation as discussed in geopython/pygeoapi#625, it does already have the mechanism in place, no? So it should be relatively trivial to add?
Tom Kralidis
@tomkralidis
not yet, I think we still need to sort out how this happens in OARec proper?
paul van genuchten
@pvgenuchten
correct, in conneg-by-profile they suggest to use a request header to indicate the schema, and we probably need an explicit parameter, such as ?schema=iso19139 to override the header (like encoding and language)