Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    doctorluz
    @doctorluz
    hello!
    Joan Masó
    @joanma747
    Hi
    Valantis
    @Tsiakos_gitlab
    Hi
    doctorluz
    @doctorluz
    Hi all... I have had to join by phone because my data keeps running out, so I am not seeing any screens. I am following along as far as I can but I can't offer any data service and my phone minutes are also limited :-(
    So I apologise for lack of contribution
    Valantis
    @Tsiakos_gitlab
    Hi all, the server is temporarily down https://mariaisawsome.iccs.gr:8443/
    will let you know in a while as soon as it is available
    Joan Masó
    @joanma747
    No worries Luzy and Valantis. Lets reconvine at 15:00 and lets have a more general conversation about our aims toward the CORIV19 and other staff.
    doctorluz
    @doctorluz
    image.png
    Hi all! There is some good news about the CAMS data in general: it is allegedly freely available but I have hardly any wifi here at home so I have not yet managed to discover whether we can get raw usable data and at what sort of temporal frequency. However, it looks just right for the case study, quite good but not TOO good (i.e. there are spatial gaps that could be usefully filled with Luftdaten data) . The link for their data store is in the below summary if anyone has more luck than me: at the moment this is what I see...

    Here's a brief summary with some starting questions/ ideas: The CAMS regional air quality analyses

    • come in the form of hourly gridded maps
    • are obtained by combining satellite observations, surface in situ observations and numerical modelling that mimic the laws of physics and chemistry in the atmosphere
    • are of very high quality and can be considered as “ground truth”.

    The main limitation of this dataset is that it cannot account for fine spatial variability and local effects that occur within few meters to hundred meters near a source, like a road segment with heavy traffic or a power plant stack. The data are representative of the average background values over a grid with a scale of 10 km by 10 km.
    The data can be obtained openly and freely from CAMS’ Atmosphere Data Store (ads.atmosphere.copernicus.eu)

    Interesting questions - are there areas of Italy where this modelled information can be further disaggregated for personal decision making with contributed observations?
    Are there observations which are strongly disparate from the model and is this because of weather, sensor error, or better representation of the ground situation at local level?

    Q. Are the contributed data definitely independent? in the case of weather you often find that citizen networks are also being federated into predictions. Action - ask CAMS
    Q. would other data like RIVM be useful as well / instead?

    doctorluz
    @doctorluz
    I also had a look at Google Earth Enigine to see what data is available there - various SENTINEL tropospheric stuff, whch is what's being used to generate many of the India/China visualisations - but not a lot of modelled / corrected datasets. https://developers.google.com/earth-engine/datasets/tags/air-quality. At the moment I am having to do most of my processing in GEE because we have no access to JRC servers and databases from outside... so if we can get datasets into that platform I am happy to help with any analysis that can be done there. What I can't do at the moment is download / upload large datasets...
    doctorluz
    @doctorluz

    OK, I have investigated the data a bit. Over Europe for PM2.5, there are 0.1 degree resolution versions of the following in NetCDF format for download:

    Seasonal (spring, summer, autumn, winter) and annual means from an ensemble of 6-8 models
    e.g. https://data.regional.atmosphere.copernicus.eu/openwis-user-portal/srv/en/iso19139.xml?id=3227

    Exceedence map of locations over the European recommended levels
    https://data.regional.atmosphere.copernicus.eu/openwis-user-portal/srv/en/iso19139.xml?id=3013

    Hourly maps of modelled levels at surface from 4 different models (Mocage, Match, Eurad, Lotos)
    e.g. https://data.regional.atmosphere.copernicus.eu/openwis-user-portal/srv/en/iso19139.xml?id=2627

    Much more useful (especially when we all have limited bandwidth working at home) would be WCS services that can be consumed directly into some online analysis platform. The services looks available and I have applied for a token to test, but the bandwith / processing / storage will be more of an issue. CEH in the UK have a great VRE for this kind of analysis but I think it only contains UK-specific data...https://eip.ceh.ac.uk/apps/atmospheric/

    image.png
    Have fired off various emails to find out what the options might be - and very happy to hear ideas from the rest of the team. Have a good evening!!
    Joan Masó
    @joanma747
    Nice findings made by @doctorluz on the CAMS and remote sensing area. Meanwhile, Ester has found this air quality global datat viewer https://waqi.info/. This has inspired me to look deaper for a possible the source of this data and I have found this project: https://openaq.org/. They have an open API with access to a rolling archive of 90 days that is more than enough for us to start setting up services and do some visualizations to study the effects of the COVID-19 in the air quality.
    Joan Masó
    @joanma747

    Anne has reported this morning:
    Hi Joan, all,

    Thanks for a very helpful meeting today....!

    I followed up on a few AQ data sets.

    From the citizen science side, we have:

    LuftDaten/SensorThings: http://maps.luftdaten.info/

    PurpleAir: https://www.mathworks.com/help/thingspeak/rest-api.html

    Turns out the State Department does not actually maintain monitors in Italy because the country has their own AQ network! Great news. Looks like some of the data are here:

    https://aqicn.org/map/italy/

    And probably elsewhere, too....

    Look forward to checking in tomorrow when I wake up. Have fun!

    Cheers,
    Anne

    doctorluz
    @doctorluz
    This is great! 207 stations across the whole of Italy, and they're denser in the North. One thing I'd like to check is whether this network is already feeding the CAMS model. I should be able to talk to a CAMS person today and will report back...
    Joan Masó
    @joanma747
    Impresive document on the efforts on the COVID-19 from the perspective of Citizen Science:
    https://docs.google.com/document/d/1qJhOG9Qw7kud-VKA1yucp-iGxO0X2mwELSaf8nL2Ulo/edit#heading=h.kildxnv8j04d
    (Sven reported that but it was empty the day before yesterday but itis progressing fast)
    Mementomoria
    @Mementomoria
    Hi!
    Anne here
    What if we can get data in ArcGIS- is that helpful @doctorluz ?
    Also, has anyone pinged Lukas? If we are going to work with Luftdaten data he would be really great to bring in.
    doctorluz
    @doctorluz
    I'm not sure, I have an account with ArcGIS online but never use it for analysis because of our commitment to open standards / tools and the fact that a lot of ESRI service standards are not 100% compatible with other implementations (viz. ESRIJSON and the ESRI flavour of REST for wfs and wms. We can try though... :-)
    Mementomoria
    @Mementomoria
    Here's what I heard:

    EPA AirNow data is in ArcGIS format straight from the source, EPA.

    The best location to search for data is the ArcGIS Living Atlas of the World: https://livingatlas.arcgis.com/en/
    I searched Air Now and 82 results for layers: https://livingatlas.arcgis.com/en/browse/#d=2&type=layers&q=Air%20Now

    The EPA AirNow layer description is here:
    https://www.arcgis.com/home/item.html?id=2d718d2733a74d1689d72b922c0ac4f4

    We can get you a license if you want one, though I totally understand the "only open source" commitment
    Actually AirNow does not seem to be very helpful for Italy
    Matth-March
    @Matth-March
    Hey guys I am working with Anne and am quite familiar with GIS, I have a few questions about the scope of this project. Do you want to do this analysis for the whole of Italy or just a part of Italy? How do you break down this part of Italy, by region, by province, by municipality? It would be best to define where you are looking and how you break that down before you move forward.
    Also how would you best normalize the data so that high values in urban areas do not overshadow values in more sparsely populated regions? I suggest by comparing percentages to some pre-pandemic average.
    asusd1
    @asusd1

    Additionally, you can directly access the US EPA AirNow Tech data at the following URLs:

    https://www.airnowtech.org/

    Register for an Account:
    https://www.airnowtech.org/requestAccnt.cfm

    API Signup:
    https://docs.airnowapi.org/account/request/?account%2Frequest=

    Data Download Site:

    https://www.epa.gov/outdoor-air-quality-data

    Andy Cobley
    @acobley
    Joan Masó
    @joanma747
    If you are curious on what is done in Catalonia, the Catalan government has released an app where members of the public can report symptoms of the disease without moving from home. About one week later they have received 400 000 reports. They have created a map with them: https://strategicsolutions.carto.com/u/strategicsolutions-admin/builder/fce617e1-58df-41c4-9c01-953d21e3b8ea/embed
    As you might suspect, the distribution resembles a human settlements map. Results agregated every 300m^2
    doctorluz
    @doctorluz

    Hi all - sorry for going a bit quiet but I've ben ill and still am... so I'm not sure how soon I will get back on track with looking at this hands on.
    I have had a really positive response from CAMS about what goes into their models (only data from the EEA, no CS observations, - they are extremely interested to know the results of any experimental aggregation of their data with other sources). They are especialy interested in the data quality aspects of any dta combination and can povide some quite detailed info on their own QA procedures. The CAMS modelled data for particulates and other pollutants is freely available through WCS and WMS with a token (https://www.regional.atmosphere.copernicus.eu/). Potentially it could be pulled directly into ArcGIs Online or other cloud analysis environments because of this compliance with standards.

    @Matth-March Re. spatial and temporal zones to consider:

    In Italy, the key area where effects should be seen from around 3 weeks ago is Lombardia, and especially some industrial zones within the region which can be seen from the NO2 drops here: https://lucabattistellageo.users.earthengine.app/view/no2itaviz (this is an app running direct on the TROPOMI sentinel data for NO2, see here: https://developers.google.com/earth-engine/datasets/tags/tropomi).

    @Matth-March again -yes, we should definitely be looking against a backdrop of past average to identify anomalies, as recommended here (https://atmosphere.copernicus.eu/amid-coronavirus-outbreak-copernicus-monitors-reduction-particulate-matter-pm25-over-china)
    to quote - "other variables could potentially play a role in the decrease ...China is actively trying to reduce emissions, and meteorological variability between different years must be considered as well. To subtract these variables, we set the duration of three years 2017-2019 to estimate the ‘business as usual’ conditions as a compromise to have a representative estimate of February’s monthly mean, while not considering too long a period over which emissions vary substantially because of long-term trends.”

    There could be seasonal differences in the peak pollen component of the small particulate measure - here in North Italy, the birch pollen levels exploded over the past 2-3 weeks (which would coincide with the big decrease in road traffic from the shutdown!) I suspect this may be earlier than in previous years - a chance to also bring in some phenological CS data?

    Joan Masó
    @joanma747
    I have found an effort to create an Air Quality Ontology: http://vocab.linkeddata.es/datosabiertos/def/medio-ambiente/calidad-aire/index-en.html. but it seems limited to chemicals that can be present in the air, such as http://vocab.linkeddata.es/datosabiertos/def/medio-ambiente/calidad-aire#oxidosDeNitrogeno
    Joan Masó
    @joanma747
    It seems that the OGC is is also collecting interoperable resources that serve COVID-19 data. Great initiative: https://www.ogc.org/resources-for-COVID-19-from-ogc
    doctorluz
    @doctorluz
    https://projects.iq.harvard.edu/covid-pm. Study on the relationship between historic pm2.5 levels and death rates across the us
    Joan Masó
    @joanma747
    Incredible. I was expecting a correlation between human activities and levels of pm2.5 but finding a direct relation between pm2.5 and immunity is incredible! Thanks for sending us this.
    Joan Masó
    @joanma747

    Two resources with official data in SensorThings API format:
    One STA about air quality. It might be interesting to compare it with the Anne developments on CitSci Air Quality services for the Earth Challange 2020

    https://airquality-frost.docker01.ilt-dmz.iosb.fraunhofer.de/v1.1/

    And one about COVID:

    http://193.196.138.56:8080/STACOVID
    This server contains global COVID-19 data from CSSE.

    Some Guide for who is new to STA

    REQUEST URL
    Request list of available Countries (Thing in STA)
    http://193.196.138.56:8080/STACOVID/v1.0/Things
    Request list of available Countries with location Lat,Lon
    http://193.196.138.56:8080/STACOVID/v1.0/Things?$expand=Locations
    Request Datastreams from Country's name = Germany
    http://193.196.138.56:8080/STACOVID/v1.0/Things?$filter=name%20eq%20%27Germany%27&$expand=Datastreams
    Request Datastreams and Observations from Germany
    (number of Total Death/ Total Recovery/ Total Confirmed cases)
    http://193.196.138.56:8080/STACOVID/v1.0/Things?$filter=name%20eq%20%27Germany%27&$expand=Datastreams/Observations

    doctorluz
    @doctorluz
    here are two more small studies (IK and Italy) on air quality and COVID lethality. I am not sure how well they have dealt with confounding factors but there does seem to be a salient general pattern.
    Joan Masó
    @joanma747
    People ingenuity has the capacity of surprising me. A MIT team came up with the idea of using AI to identify COVID-19 suspected cases only recording them coughing!!. They have released it as a mobile app. You can cough in front of you mobile phone and that is it!!.
    https://www.catalannews.com/tech-science/item/algorithm-aims-to-detect-covid-19-cases-with-app-that-records-coughing
    If you want to try it, please clean you phone afterwards!
    Andy Cobley
    @acobley

    https://github.com/yahoo/covid-19-data The Yahoo Knowledge Graph team at Verizon Media is responsible for providing critical COVID-19 data that feeds into Yahoo properties like Yahoo News, Yahoo Finance, and Yahoo Weather. The COVID-19 datasets include country, state, and county level information updated on a rolling basis, with updates occuring approximately hourly.

    The COVID-19 datasets are constructed entirely from primary (government and public agency) sources with a clear attribution of the primary sources used for each geographical region. While other aggregations of COVID-19 data are already available, we believe ours to be the only open source COVID-19 dataset that is constructed entirely from primary sources with clear attribution back to those sources. Our hope is that additional transparency will enable more accurate analysis, aiding researchers who seek to understand and prevent further spread of the disease.

    Joan Masó
    @joanma747

    Links in our #14 telecon:

    15:23

    Lukas@Sensor.Community

    where is it possible to find out what the earth challenge approach is doing ?

    15:25

    Lucy Bastin

    How Joan described story maps just then is exactly right

    15:26

    Lucy Bastin

    Not about hosting data but can lead you to data and help you explore it

    15:26

    Metis Meloche

    How StoryMaps are used by EC2020, is available on our webpage: https://earthchallenge2020.earthday.org/

    15:27

    Metis Meloche

    How we lead people to the data is we're trying a hubsite, called the citizen science cloud: https://cscloud-ec2020.opendata.arcgis.com/

    15:27

    Valantis Tsiakos

    https://www.tableau.com/

    15:30

    Clay Herbaugh

    https://storymaps.arcgis.com/stories/f99dd40c7e084ec19df7affc91efb95c

    15:41

    Lukas@Sensor.Community

    wanted to share with you: https://op.europa.eu/en/web/eudatathon

    15:41

    Steffen Fritz

    Dear all, sorry I have to leave thanks for the great input and lets be on touch soon!

    15:44

    Lucy Bastin

    Hi Lukas, is this the cams data?

    15:46

    Lukas@Sensor.Community

    Lukas@Sensor.Community

    Andy Cobley
    @acobley

    Here is a dataset from Google Community Mobility Reports. It's a massive worldwide dataset about the change in mobility during the covid 19 breakout.

    https://www.google.com/covid19/mobility/

    Joan Masó
    @joanma747
    I have found this resource that I have been told is related with the US department of state. The expose Citizen Science APIs about 4 topics. This is the one about mosquitoes: https://portal-data.cscloud.host/api-details#api=guest-mosquitoes&operation=get-daily
    Joan Masó
    @joanma747
    Hi, I just created the page for the next two days CoP plenary meeting. It is here: https://external.ogc.org/twiki_public/CitSciIE/VirtualMeeting01