Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Mike
@mikebelanger
I have a question that might be more of a NetCDF4 question, not sure
let's say I make a DataQuery object, call it query
and do a lonlat_point query
and my dataset is a grib2 file
does it look for values based on the nearest grid point?
like there's no interpolation values between points or anything?
Ryan May
@dopplershift
@mikebelanger Thanks for the kind words. Yes, the lonlat_point request tells the THREDDS server, using a service called the netCDF Subset Service (NCSS), to return the data for the grid point closest to the requested location.
Mike
@mikebelanger
ah ok cool
@dopplershift yeah I really like using this service - siphon in combination with this THREDDs-based technology makes things easier
@dopplershift I got a bunch of grib2s that are filed away actually. And I'm looking at a scalable way of processing them
@dopplershift Maybe this question is better asked to THREDDs at large, but I'm wondering: how much setup is involved with making our own THREDDs instance?
so something that serves grib2 data, has all that XML metadata, etc
Mike
@mikebelanger
like would this tutorial be up to date - fairly straightforward? https://www.unidata.ucar.edu/software/tds/current/tutorial/GettingStarted.html
Sean Arms
@lesserwhirls
Greetings @mikebelanger - that tutorial is a bit out of date with respect to the Java and Tomcat instructions. I updated the first part of that page to point to our 5.0 documentation, which includes updated instructions. Same thing with the Security page of the tutorial (https://www.unidata.ucar.edu/software/tds/current/tutorial/Security.html). The rest should be good to go for the 4.6 release, though. If you are most interested in serving GRIB output, then you really should run the 5.0 beta version, as many fixes have been added there that are not available in 4.6. We hope to get TDS 5.0 stable this fall.
Mike
@mikebelanger
@lesserwhirls Greetings! And thanks for the heads up. Yeah I'm definitely interested mainly in GRIB reading.
oh and thanks for revising that tutorial page too.
Mike
@mikebelanger
@lesserwhirls the 5.0 documentation looks pretty comprehensive! I guess once the grib-loading stuff for 5.0 is firmed up - sections like this will be filled in? https://docs.unidata.ucar.edu/tds/5.0/userguide/grib_files_cdm.html
hannah
@story645
Hi, is this one of your examples translated to Spanish? https://pybonacci.org/2018/11/27/creacion-de-videos-al-vuelo-en-el-notebook/
Ryan May
@dopplershift
Only if it's really old. I haven't dabbled with WMS.
Sean Arms
@lesserwhirls
@mikebelanger - sorry for the dead link! That's actually a casualty of splitting netcdf-java and the THREDDS Data Server into seperate code bases. That URL will work if you replace /tds/5.0/ with /netcdf-java/5.2/, so https://docs.unidata.ucar.edu/tds/5.0/userguide/grib_files_cdm.html As you run across issues like this with the docs, would you mind opening a github issue over at https://github.com/unidata/tds? Thanks!
Mike
@mikebelanger
@lesserwhirls hey no probs! and thanks for the newer links. And for sure, for now on I'll report as an issue. Also I'll stop posting general THREDDs stuff here, as its not quite on topic
Sean Arms
@lesserwhirls
No worries @mikebelanger - I just want to make sure things don't get lost!
Pete Pokrandt
@PTH1_twitter
Is it possible to use/construct an ncss query to pull a subset of multiple variables at multiple vertical levels? Let's say for example, I want 200 hPa U/V, 500 hPa and 1000 hPa height, and MSLP. Can I get all of that in a single ncss call, or do I have to do multiple ncss calls for each variable/level combo?
Ryan May
@dopplershift
Unfortunately, NCSS does not allow requesting multiple levels. (Correct me if I’m wrong @lesserwhirls )
Pete Pokrandt
@PTH1_twitter
Ok, I didn't think so, but wasn't sure if I was missing something
I was thinking I could do one ncss call and grab all of the data subset that I need for all of my maps in one call, but I guess not
Zach Bruick
@zbruick
I just tried playing around with it and it seems like if you try to add a vertical_level call to each variable you request, it just overrides it with the last level specified, or errors out if the variables don't share the same vertical coordinate name.
Mike
@mikebelanger
hey guys, I'm trying to perform a time query on UCAR's gfs forecast archives
ideally, I'd like to be able to specify any datetime once, and get the nearest grib2 file to that time.
but so far, I've had to pass in the year, month and day into a catalog URL, in order to get the grib2
for example: cat = TDSCatalog('https://rda.ucar.edu/thredds/catalog/files/g/ds084.1/2017/20171001/')
if I try something more general, like cat = TDSCatalog('https://rda.ucar.edu/thredds/catalog/files/g/ds084.1/') I don't get any datasets
Mike
@mikebelanger
if I try something like the time series tutorial, like cat = TDSCatalog('https://rda.ucar.edu/thredds/catalog/files/g/ds084.1/' '?dataset=files/g/ds084.1/2017')
like adding the dataset query parameter - I still get no datasets
Ryan May
@dopplershift
Try this:
cat = TDSCatalog('https://rda.ucar.edu/thredds/catalog/files/g/ds084.1/catalog.xml')
cat.catalog_refs
In that top-level catalog, there are not any datasets, but there are references to other catalogs, which are available from the .catalog_refs attribute.
Amelia
@apottr
hi all!! I was wondering if anyone knew if there was a reason for the UWYO data output of siphon to not include some of the columns (relative humidity, mixing ratio, etc)?
Ryan May
@dopplershift
@apottr I think because the other parameters are derived from the reported data, and for our original uses we would just use MetPy to calculate. Feel free to open an issue if having these values would help you...or even better I’d merge a PR doing it.
Mike
@mikebelanger
@dopplershift thanks - so to get a grib2 in this case, I'd have to first query the top level as you've demonstrated, get a particular catalog_ref, and then re-query with the catalog ref added on? Then repeat until I get all the way 'down the tree'?
Ryan May
@dopplershift
@mikebelanger Pretty much, unfortunately since the RDA does not have the data set up (as far as I can tell) using a TDS Grib Collection. Another option for you is to programmatically generate the url, like:
base_url = 'https://rda.ucar.edu/thredds/catalog/files/g/ds084.1/'
dt = datetime(2017, 10, 1)
url = base_url + f'{dt:%Y/%Y%m%d}'
Not completely sure I got the syntax completely correct, but hopefully the idea comes across
Mike
@mikebelanger
ah ok - yeah no probs. I've been doing something similar to the snippet you posted. Some time stuff and string interpolation is currently what I'm doing.
I just wanted to make sure I wasn't missing some other method, or some archive feature
Amelia
@apottr
@dopplershift I figured that was the case. I'll put something together and submit a PR.
Mike
@mikebelanger
hey guys I'm having an issue accessing RDA's archives. An issue similar to here: GIS4WRF/gis4wrf#25. Basically I'm getting a HTTPSConnectionPool(host='rda.ucar.edu', port=443): Max retries exceeded with url: /thredds/catalog/files/g/ds084.1/2019/20190224/catalog.xml (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fec30e04650>: Failed to establish a new connection: [Errno 110] Connection timed out',))
I'm getting that error from running a script on my remote machine.
I can still access that url from my local laptop
so its probably something to do with IP configuration?
Mike
@mikebelanger
hmm, looking more into this issue - I've found the following e-mail thread: https://www.unidata.ucar.edu/support/help/MailArchives/thredds/msg02351.html
The email from Unidata THREDDS Support, sent on Tuesday, April 18, 2017 11:36 AM, references the above error