Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    collinjc
    @collinjc:matrix.org
    [m]
    Possibly related, it seems that the out-of-the-box configuration still points to the now outdated results.openaddresses.io
    Juras Norkus
    @jnorkus

    Hi guys,
    I'm solving an issue where WOF data is very limited for an area. Cities, neighbourhoods missing or having weird names that nobody knows about. The interesting thing is that OSM data has all of this information but is not used at all in Pelias importer.

    This config just skips the lookups and leaves the data without admin properties.

    "adminLookup": {
            "enabled": false
    },

    I also found this code that I assume was once used to extract admin data. Uncommenting it does not work, as I understand because model changed since then.

    Any directions on solving this?

    Keshav Nandan
    @keshavnandan

    Hi Team, I am trying to run the openstreetmap importer for full planet data but the importer gets stuck at pbf2json converter step for a really long time and ultimately the importer node gets OOMKilled terminated. The importer works fine for smaller datasets like portland. I kept increasing the compute and mem resources but looks like it needs much more than 8 GB RAM as mentioned in the documentation.

    2022-04-04T21:48:28.232Z - info: [openstreetmap] Creating read stream for: /dl/openstreetmap/planet-220307.osm.pbf
    
    Process finished with exit code 0

    Container status

    Containers:
      openstreetmap-import-container:
        Image:         pelias/openstreetmap:latest
        Port:          <none>
        Host Port:     <none>
        Command:
          ./bin/start
        State:          Terminated
          Reason:       OOMKilled
          Exit Code:    0
          Started:      Mon, 04 Apr 2022 17:48:27 -0400
          Finished:     Mon, 04 Apr 2022 19:48:10 -0400
        Ready:          False
        Restart Count:  0
        Limits:
          cpu:     3
          memory:  16Gi
        Requests:
          cpu:     1500m
          memory:  8Gi
        Environment:
          PELIAS_CONFIG:  /conf/pelias.json
        Mounts:
          /conf from pelias-config (rw)
          /dl from planet-volume (rw)

    How much cpu and mem is needed to run the openstreetmap importer for full planet build?

    Keshav Nandan
    @keshavnandan
    The importer is working after I bumped up the mem req to 16 Gi
    Adrien Lagamelle
    @adrienlagamelle
    Anyone here knows where is elasticsearch installed using the docker installation steps. Im getting this:/usr/bin/which: no elasticsearch in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin)
    gelsas
    @gelsas
    I am getting this error shown: Error: EACCES: permission denied, mkdir '/data/geonames'
    Brad Hards
    @bradh
    Probably missing a prefix somewhere. Hard to say without more context.
    gelsas
    @gelsas
    Started over completly new now getting this:

    pelias compose pull
    Pulling libpostal ... done
    Pulling schema ... done
    Pulling api ... done
    Pulling placeholder ... done
    Pulling whosonfirst ... done
    Pulling openstreetmap ... done
    Pulling openaddresses ... done
    Pulling geonames ... done
    Pulling csv-importer ... done
    Pulling transit ... done
    Pulling polylines ... done
    Pulling interpolation ... done
    Pulling pip ... done
    Pulling elasticsearch ... done
    Pulling fuzzy-tester ... done
    [ttap.io@app planet]$ pelias elastic start
    Removing pelias_elasticsearch
    Recreating abf1253d5782_pelias_elasticsearch ... error

    ERROR: for abf1253d5782_pelias_elasticsearch Cannot start service elasticsearch: driver failed programming external connectivity on endpoint pelias_elasticsearch (6b4217b570655a656a53251fb77701a3b00d6e9714f8aad29de47fdd6d56daea): (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 127.0.0.1 --dport 9300 -j DNAT --to-destination 172.18.0.7:9300 ! -i br-92febc9bc408: iptables: No chain/target/match by that name.
    (exit status 1))

    ERROR: for elasticsearch Cannot start service elasticsearch: driver failed programming external connectivity on endpoint pelias_elasticsearch (6b4217b570655a656a53251fb77701a3b00d6e9714f8aad29de47fdd6d56daea): (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 127.0.0.1 --dport 9300 -j DNAT --to-destination 172.18.0.7:9300 ! -i br-92febc9bc408: iptables: No chain/target/match by that name.
    (exit status 1))
    ERROR: Encountered errors while bringing up the project.

    and when running pelias download all this:

    pelias download all
    info: [openaddresses-download] Attempting to download all data
    error: [openaddresses-download] error making directory /data/openaddresses message=EACCES: permission denied, mkdir '/data/openaddresses', stack=Error: EACCES: permission denied, mkdir '/data/openaddresses', errno=-13, code=EACCES, syscall=mkdir, path=/data/openaddresses
    error: [openaddresses-download] Failed to download data message=EACCES: permission denied, mkdir '/data/openaddresses', stack=Error: EACCES: permission denied, mkdir '/data/openaddresses', errno=-13, code=EACCES, syscall=mkdir, path=/data/openaddresses
    internal/fs/utils.js:269
    throw err;
    ^

    Error: EACCES: permission denied, mkdir '/data/geonames'
    at Object.mkdirSync (fs.js:921:3)
    at module.exports (/code/pelias/geonames/lib/tasks/download.js:13:6)
    at Object.<anonymous> (/code/pelias/geonames/bin/downloadData.js:11:1)
    at Module._compile (internal/modules/cjs/loader.js:999:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
    at Module.load (internal/modules/cjs/loader.js:863:32)
    at Function.Module._load (internal/modules/cjs/loader.js:708:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12)
    at internal/main/run_main_module.js:17:47 {
    errno: -13,
    syscall: 'mkdir',
    code: 'EACCES',
    path: '/data/geonames'
    }
    internal/fs/utils.js:269
    throw err;
    ^

    Error: EACCES: permission denied, mkdir '/data/whosonfirst/sqlite'
    at Object.mkdirSync (fs.js:921:3)
    at download (/code/pelias/whosonfirst/utils/download_sqlite_all.js:34:6)
    at Object.<anonymous> (/code/pelias/whosonfirst/utils/download_sqlite_all.js:126:1)
    at Module._compile (internal/modules/cjs/loader.js:999:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
    at Module.load (internal/modules/cjs/loader.js:863:32)
    at Function.Module._load (internal/modules/cjs/loader.js:708:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12)
    at internal/main/run_main_module.js:17:47 {
    errno: -13,
    syscall: 'mkdir',
    code: 'EACCES',
    path: '/data/whosonfirst/sqlite'
    }
    warn: [csv-download] No files to download, quitting
    info: [openstreetmap-download] Downloading sources: https://planet.openstreetmap.org/pbf/planet-latest.osm.pbf
    error: [openstreetmap-download] error making directory /data/openstreetmap message=EACCES: permission denied, mkdir '/data/openstreetmap', stack=Error: EACCES: permission denied, mkdir '/data/openstreetmap', errno=-13, code=EACCES, syscall=mkdir, path=/data/openstreetmap
    error: [openstreetmap-download] Failed to download data message=EACCES: permission denied, mkdir '/data/openstreetmap', stack=Error: EACCES: permission denied, mkdir '/data/openstreetmap', errno=-13, code=EACCES, syscall=mkdir, path=/data/openstreetmap
    info: [interpolation(TIGER)] downloading all TIGER data
    warn: [transit] 'pelias.json' config lacks a transit object entry. Transit importer quitting after taking no action
    error: [interpolation(TIGER)] message=getaddrinfo EAI_AGAIN data.geocode.earth, stack=Error: getaddrinfo EAI_AGAIN data.geocode.earth
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:66:26), errno=EAI_AGAIN, code=EAI_AGAIN, syscall=getaddrinfo, hostname=data.geocode.earth, response=undefined
    error: [interpolation(TIGER)] message=getaddrinfo EAI_AGAIN data.geocode.earth, stack=Error: getaddrinfo EAI_AGAIN data.geocode.earth
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:66:26), errno=EAI_AGAIN, code=EAI_AGAIN, syscall=getaddrinfo, hostname=data.geocode.earth, response=undefined

    gelsas
    @gelsas
    I do not really understand why I am seeing this error
    getting the same errors when using: sudo pelias download all
    I would be willing to pay for help to set it up correctly
    gelsas
    @gelsas
    I am trying to get stuff working for hours now
    gelsas
    @gelsas
    Got it
    Lucas Hänke de Cansino
    @l4b4r4b4b4
    Hey everyone,
    For some reason I can't seem to make reverse geoconding work. I setup the docker-compose stack behind traefik 2, moved the Germany Dataset germany-valhalla.polylines.0sv to ./data/polylines, edited the files .env and pelias.json accordingly and ran the installation commands, excluding pelias prepare polylines.Everything went through and I can send queries to search and autocomplete endpoints succesfully getting a proper answer with GeoFeatures in the FeatureCollection Array. When querrying the reverse endpoint I still get an answer but the FeatureCollection Array is empty.
    My research brought me to the point, that I might have to adjust ./stream/pipeline.js but honstly I couldn't find any further information, where to find it and how to edit it, to process the provided Dataset.
    Suggestions, hints and help on how to get Reverse API working would be highly appreciated :)
    Cheers Luke
    With pelias.json as follows, when importing it still takes the generated extract.0sv instead of the provided germany-valhalla.polylines.0sv:
    "imports": {
        "polyline": {
          "datapath": "/data/",
          "files": [ "germany-valhalla.polylines.0sv" ]
        },
    The following on the other hand does seem to take the provided dataset, but reverse endpoint still doesnt give back contentfull answers:
    "imports": {
        "polyline": {
          "datapath": "/data/polylines",
          "files": [ "germany-valhalla.polylines.0sv" ]
        },
    Sorry other way around.
    Lucas Hänke de Cansino
    @l4b4r4b4b4
    hmmm taking everything back, i guess. It seems to work now xD
    Keshav Nandan
    @keshavnandan
    Whats the download path for getting the interpolation data for full planet? I am able to download the data for portland using path "https://s3.amazonaws.com/pelias-data.nextzen.org/portland-metro/interpolation" but am not sure whats the path for the planet, I tried planet and planet-latest but its not working
    MALKARAJ
    @MALKARAJ
    how to configure custom elasticsearch cluster instead of docker image ??
    2 replies
    gelsas
    @gelsas
    hey guys, does anyone know how do we enable fuzzy search for autocomplete ?
    winston01
    @winston01
    Hi guys
    I noticed a strange behaviour while testing autocomplete in various languages:
    I was running this query:
    which works fine
    but then I tried with Kanada (Hungarian spelling):
    and no results :(
    on the other hand, it does work fine in search:
    and also in autocomplete if looking for Kanada directly:
    winston01
    @winston01
    can someone explain this? perhaps it's a known defect (although I didn't find an issue on github that would clearly fit this use case)
    Andre-Pars
    @Andre-Pars
    Does anyone know if there's a setting where we can return neighbourhood instead of other admin fields for autocomplete
    Higor Duarte
    @higorduarte98
    Hello, I need pelias logs used in production to perform experiments to analyze performance in a college project.
    Julian Simioni
    @orangejulius
    @higorduarte98 that would be a tough ask. There are serious potential privacy implications for sharing logs from most production systems. I'd push back on your professor a bit if they are requiring this of you
    Keshav Nandan
    @keshavnandan
    Hi team,
    I am planning to add a new data source to Pelias and am wiring an importer for that. Do I need to make some config changes or code changes to Pelias so that it recognises the new data source in the sources param and queries accordingly? I am able to ingest the data from this new data source with a specific source and sourceId field.
    Arne Setzer
    @arnesetzer
    What is the expected outcome, if the database only contains venues, and addresses and a street name is given? (CSV Import only, no interpolation)?
    I expect Pelias to list all addresses which contain the street name, but this isn't, at least in my setup, not the case.
    1 reply
    Quốc Nhật
    @tranquocnhat
    Hi team, whosonfirst contain wrong admin hierarchy(in my country), i has quesion why we don't using openstreetmap admin boundary
    instead of whosonfirst
    1 reply
    i meant openstreetmap become data for adminLookup
    Ben Moreau
    @benlexer:matrix.org
    [m]
    Hi all - am hoping you can help me. Do you know if there are ID <> address databases for Singapore, Thailand, Indonesia, Philippines, Vietnam, Malaysia...? Anything like the GNAF database in Aus or the National Change of Address database in US?
    István
    @istvanszoboszlai
    Hi Guys! Is pelias being maintained / developed currently? If I get it right, last commit on github is 8 months old.
    1 reply
    Brad Hards
    @bradh
    Looks like https://github.com/pelias/spatial has changes this week...
    @istvanszoboszlai so I think there is active development.
    1 reply
    3koozy
    @3koozy
    Hello there gents and gals , I wish you a good morning :),
    I have a small question and would love it if someone helps me out, can pelias support the Arabic language and middle east countries' addresses?
    because I have tested the commercial distribution (geocode Earth) and it was not that great, it only supports city-level address not street or neighbourhood