Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Florian Hoedt
    @gannebamm
    @kikislater
    administrator@gdit-ins-slave1:~/tisdar/scripts/spcgeonode$ docker-compose exec django /spcgeonode/manage.py collect_metrics -n -t xml -f
    WARNING: The LETSENCRYPT_MODE variable is not set. Defaulting to a blank string.
    Cannot collect from Service: default-geoserver@nginx: HTTPConnectionPool(host='172.18.0.3', port=80): Max retries exceeded with url: /geoserver/rest/monitor/requests.html?to=2019-12-03T12%3A47%3A29&from=2019-12-03T09%3A21%3A48 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6f9e3ffb90>: Failed to establish a new connection: [Errno 111] Connection refused',))
    Traceback (most recent call last):
      File "/spcgeonode/geonode/monitoring/management/commands/collect_metrics.py", line 99, in handle
        format=options['format'])
      File "/usr/local/lib/python2.7/site-packages/timeout_decorator/timeout_decorator.py", line 81, in new_function
        return function(*args, **kwargs)
      File "/spcgeonode/geonode/monitoring/management/commands/collect_metrics.py", line 143, in run_check
        data_in = h.collect(since=last_check, until=until, format=format)
      File "/spcgeonode/geonode/monitoring/service_handlers.py", line 129, in collect
        _collected = self._collect(since.astimezone(utc), until.astimezone(utc), **kwargs)
      File "/spcgeonode/geonode/monitoring/service_handlers.py", line 172, in _collect
        requests = list(self.gs_monitor.get_requests(format=format, since=since, until=until))
      File "/spcgeonode/geonode/monitoring/utils.py", line 139, in get_requests
        verify=False)
      File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 75, in get
        return request('get', url, params=params, **kwargs)
      File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 60, in request
        return session.request(method=method, url=url, **kwargs)
      File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 533, in request
        resp = self.send(prep, **send_kwargs)
      File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 646, in send
        r = adapter.send(request, **kwargs)
      File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 516, in send
        raise ConnectionError(e, request=request)
    ConnectionError: HTTPConnectionPool(host='172.18.0.3', port=80): Max retries exceeded with url: /geoserver/rest/monitor/requests.html?to=2019-12-03T12%3A47%3A29&from=2019-12-03T09%3A21%3A48 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6f9e3ffb90>: Failed to establish a new connection: [Errno 111] Connection refused',))
    Cannot collect from Service: nginx-hostgeoserver@nginx: HTTPConnectionPool(host='172.18.0.3', port=80): Max retries exceeded with url: /geoserver/rest/about/monitoring.json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6f9e14c350>: Failed to establish a new connection: [Errno 111] Connection refused',))
    Traceback (most recent call last):
      File "/spcgeonode/geonode/monitoring/management/commands/collect_metrics.py", line 99, in handle
        format=options['format'])
      File "/usr/local/lib/python2.7/site-packages/timeout_decorator/timeout_decorator.py", line 81, in new_function
        return function(*args, **kwargs)
      File "/spcgeonode/geonode/monitoring/management/commands/collect_metrics.py", line 143, in run_check
        data_in = h.collect(since=last_check, until=until, format=format)
      File "/spcgeonode/geonode/monitoring/service_handlers.py", line 129, in collect
        _collected = self._collect(since.astimezone(utc), until.astimezone(utc), **kwargs)
      File "/spcgeonode/geonode/monitoring/service_handlers.py", line 192, in _collect
        rdata = requests.get(url, timeout=10, verify=False)
      File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 75, in get
        return request('get', url, params=params, **kwargs)
      File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 60, in request
        return session.request(method=method, url=url, **kwargs)
      File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 533, in reque
    after changing the geoserver location variable in the composition AND changing nginx to the nginx containers (!) IP inside the composition in the admin interface for monitoring I get the error log from above. Something is not working properly for me...
    Alessio Fabiani
    @afabiani
    @/all master branch is moving towards Python3 so it might become unstable. I kindly suggest to use the branch 2.10.x (aka 2.10.2)
    Florian Hoedt
    @gannebamm
    thanks @afabiani
    Francesco Bartoli
    @francbartoli
    👍
    Florian Hoedt
    @gannebamm
    I see there is a lot of progress going on.
    Sylvain POULAIN
    @kikislater
    what do you have in settings ?
    I have this
    # Settings for MONITORING plugin
    CORS_ORIGIN_ALLOW_ALL = ast.literal_eval(os.environ.get('CORS_ORIGIN_ALLOW_ALL', 'False'))
    #GEOIP_PATH = os.getenv('GEOIP_PATH', os.path.join(PROJECT_ROOT, 'GeoIPCities.dat'))
    
    MONITORING_CONFIG = os.getenv("MONITORING_CONFIG", None)
    MONITORING_HOST_NAME = os.getenv("MONITORING_HOST_NAME", HOSTNAME)
    MONITORING_SERVICE_NAME = os.getenv("MONITORING_SERVICE_NAME", 'local-geonode')
    
    # this will disable csrf check for notification config views,
    # use with caution - for dev purpose only
    MONITORING_DISABLE_CSRF = ast.literal_eval(os.environ.get('MONITORING_DISABLE_CSRF', 'False'))
    
    if MONITORING_ENABLED:
        if 'geonode.monitoring' not in INSTALLED_APPS:
            INSTALLED_APPS += ('geonode.monitoring',)
        if 'geonode.monitoring.middleware.MonitoringMiddleware' not in MIDDLEWARE_CLASSES:
            MIDDLEWARE_CLASSES += \
                ('geonode.monitoring.middleware.MonitoringMiddleware',)
    
    # skip certain paths to not to mud stats too much
    MONITORING_SKIP_PATHS = ('/api/o/',
                             '/monitoring/',
                             '/admin',
                             '/lang.js',
                             '/jsi18n',
                             STATIC_URL,
                             MEDIA_URL,
                             re.compile('^/[a-z]{2}/admin/'),
                             )
    
    # configure aggregation of past data to control data resolution
    # list of data age, aggregation, in reverse order
    # for current data, 1 minute resolution
    # for data older than 1 day, 1-hour resolution
    # for data older than 2 weeks, 1 day resolution
    MONITORING_DATA_AGGREGATION = (
                                   (timedelta(seconds=0), timedelta(minutes=1),),
                                   (timedelta(days=1), timedelta(minutes=60),),
                                   (timedelta(days=14), timedelta(days=1),),
                                   )
    Florian Hoedt
    @gannebamm
    # Settings for MONITORING plugin
    MONITORING_ENABLED = ast.literal_eval(os.environ.get('MONITORING_ENABLED', 'False'))
    
    MONITORING_CONFIG = os.getenv("MONITORING_CONFIG", None)
    MONITORING_HOST_NAME = os.getenv("MONITORING_HOST_NAME", HOSTNAME)
    MONITORING_SERVICE_NAME = os.getenv("MONITORING_SERVICE_NAME", 'geonode')
    
    # how long monitoring data should be stored
    MONITORING_DATA_TTL = timedelta(days=int(os.getenv("MONITORING_DATA_TTL", 7)))
    
    # this will disable csrf check for notification config views,
    # use with caution - for dev purpose only
    MONITORING_DISABLE_CSRF = ast.literal_eval(os.environ.get('MONITORING_DISABLE_CSRF', 'False'))
    
    if MONITORING_ENABLED:
        if 'geonode.monitoring' not in INSTALLED_APPS:
            INSTALLED_APPS += ('geonode.monitoring',)
        if 'geonode.monitoring.middleware.MonitoringMiddleware' not in MIDDLEWARE_CLASSES:
            MIDDLEWARE_CLASSES += \
                ('geonode.monitoring.middleware.MonitoringMiddleware',)
    
        # skip certain paths to not to mud stats too much
        MONITORING_SKIP_PATHS = ('/api/o/',
                                '/monitoring/',
                                '/admin',
                                '/lang.js',
                                '/jsi18n',
                                STATIC_URL,
                                MEDIA_URL,
                                re.compile('^/[a-z]{2}/admin/'),
                                )
    
        # configure aggregation of past data to control data resolution
        # list of data age, aggregation, in reverse order
        # for current data, 1 minute resolution
        # for data older than 1 day, 1-hour resolution
        # for data older than 2 weeks, 1 day resolution
        MONITORING_DATA_AGGREGATION = (
            (timedelta(seconds=0), timedelta(minutes=1),),
            (timedelta(days=1), timedelta(minutes=60),),
            (timedelta(days=14), timedelta(days=1),),
        )
    
    USER_ANALYTICS_ENABLED = ast.literal_eval(os.getenv('USER_ANALYTICS_ENABLED', 'False'))
    GEOIP_PATH = os.getenv('GEOIP_PATH', os.path.join(PROJECT_ROOT, 'GeoIPCities.dat'))
    # -- END Settings for MONITORING plugin
    Sylvain POULAIN
    @kikislater
    I have Geoserver 2.15.2, postgres 10 and geonode 2.10.x. Same for you ?
    Florian Hoedt
    @gannebamm
    2.15.2, postgres 10 and geonode 2.10.x (= 2.10.x)
    I have added monitoring as plugin to geoserver in spc
    Will test again
    Sylvain POULAIN
    @kikislater
    What I will do is enter in django container and ping geoserver. Same with nginx container
    Florian Hoedt
    @gannebamm
    Ha!
    does work now
    image.png
    Sylvain POULAIN
    @kikislater
    What did you do ?
    Florian Hoedt
    @gannebamm

    Dockerfile for Geoserver:

    FROM openjdk:8-jre-alpine
    
    ARG version=2.15.2
    ARG branch=2.15.2
    ARG plugins=monitor
    [...]

    .env file:

    [...]
    HOST_IP=134.110.18.170
    [...]

    docker-compose.yml:

        - GEOSERVER_LOCATION=https://tisdar.dev.thuenen.de/geoserver/
        # MONITORING
        - MONITORING_ENABLED=True
        - USER_ANALYTICS_ENABLED=True
        - MONITORING_DATA_TTL=365
      extra_hosts:
        - "${HTTPS_HOST}:${HOST_IP}"

    after successfull build on host:
    docker exec -it django_tisdar_dev /bin/bash
    and inside the django container:
    DJANGO_SETTINGS_MODULE=geonode.settings python manage.py collect_metrics

    Florian Hoedt
    @gannebamm
    @francbartoli @afabiani about the above. Do you think changing the nginx address to the hosts address for GEOSERVER_LOCATION will cause issues?
    Alessio Fabiani
    @afabiani
    nginx address ??
    changed it to:
    - GEOSERVER_LOCATION=https://tisdar.dev.thuenen.de/geoserver/
    to get collect_metrics working
    Francesco Bartoli
    @francbartoli
    @gannebamm uhm…that usually is going to be the internal address AFAIK
    Alessio Fabiani
    @afabiani
    @gannebamm I see, yes, not a big issue if you are on the same network. It is basically the same.
    it would be good, however, to understand why it doesn't work using an internal location
    probably a monitoring config issue? Wrong host on the DB? Or wrong proxy_base_url on GeoServer?
    Sylvain POULAIN
    @kikislater
    That's why I add extra_hosts
    I didn't need to add monitor in geoserver plugins
    Francesco Bartoli
    @francbartoli
    @afabiani in geonode-project https://github.com/GeoNode/geonode-project/blob/master/tasks.py#L415-L493 this apparently would work
    @gannebamm can you look at and see if it solves the issue?
    Alessio Fabiani
    @afabiani
    yeps (I believe). There's a tentative work to auto-update the fixture
    Florian Hoedt
    @gannebamm
    I have to get the prototype running till friday, therefore I wont test anything this week. But I will take a look at it next week and write some documentation about SPC + monitoring
    Toni
    @t-book
    Team New Zealand rocks! :collision: :thumbsup:
    Bildschirmfoto 2019-12-04 um 08.09.14.png
    Tobi
    @GeoTob
    @t-book I'll let matt know tomorrow
    he's been running 2to3.py like crazy, while the rest is working on the hard problems :P
    Toni
    @t-book
    :)) awesome! nice to see this growing from day to day!
    Tobi
    @GeoTob
    we're held up by some of the django-autocomplete stuff and trying to get the django-osgeo-importer tests to beahave -- hopefully some time soon
    Toni
    @t-book

    we're held up by some of the django-autocomplete stuff

    Oh this happens to me all the time as well :/ ... The devil is in the details

    Tobi
    @GeoTob
    indeed :) also this wouldn't be fun without at least some hard problems..
    speaking of hard problems... @travislbrundage any news on the tests at all?
    Travis
    @travislbrundage
    I'd been out for a bit on some other projects and had been trying to get a big refactor finished, but I think submitting it in pieces might make more sense at this point, so I'll go ahead and submit a smaller PR with some changes this afternoon
    Tobi
    @GeoTob
    @travislbrundage yay!:
    )
    Diego Padron
    @dpadron
    Good day friends,
    I have a question: How to increase the limit of Results for FeatureType. Only list 10 results
    image.png
    Florian Hoedt
    @gannebamm

    @dpadron @afabiani
    Regarding mapstore2 modifications:

    I´ve found this old info on modifying MapStore2 for geonode. Is it still relevant? From the conversation:


    In order to add your own provider, you should roughly follow those steps:

    1) download the geonode-mapstore-client source code

    git clone https://github.com/geosolutions-it/geonode-mapstore-client.git

    2) go to folder geonode-mapstore-client/geonode_mapstore_client/client and run

    git submodule update --recursive

    3) edit the file ConfigProvider.js and add your new provider

    vim MapStore2/web/client/utils/ConfigProvider.js

    4) rebuild the library; from geonode-mapstore-client/geonode_mapstore_client/client

    npm run compile

    5) install your local version of the python library; from geonode-mapstore-client

    pip install -e .

    6) go back to geonode source folder and run

    python manage.py collectstatic

    7) restart the server after having updated the MS2_DEFAULT_BACKGROUNDs config accordingly

    Team New Zealand rocks! :collision: :thumbsup:

    !!!!