Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Ryan May
@dopplershift
There’s not an easy way to do what you want. This is really where we need to go beyond static catalogs and directory-based metadata and into server-side collections and database-y things
Pete Pokrandt
@PTH1_twitter
@djhoese sweet!
David Hoese
@djhoese
@dopplershift what is your opinion of "Intake"?
Ryan May
@dopplershift
I don’t think it solves any new problems (at least in earth science) that we weren’t already solving with THREDDS.
James Simkins
@jsimkins2

Hello!

I've been unable to pull radar data via Siphon since 6PM last night. It seems there's an error with the AWS credentials.

line 161, in get_catalog
raise BadQueryError(self.get_catalog_raw(query))
siphon.http_util.BadQueryError: b'com.amazonaws.AmazonClientException: Unable to load AWS credentials from any provider in the chain

Can you send a bit of sample code that’s failing?
James Simkins
@jsimkins2
Here is an example snipet:
from siphon.radarserver import RadarServer, get_radarserver_datasets
from siphon.catalog import TDSCatalog, get_latest_access_url
from datetime import datetime, timedelta

site = 'KDOX'
cat = TDSCatalog('http://thredds-aws.unidata.ucar.edu/thredds/radarServer/catalog.xml')
rs = RadarServer(cat.catalog_refs['S3 NEXRAD Level II'].href)
rs.stations[site]
query = rs.query()
query.stations(site).time(datetime.utcnow())
rs.validate_query(query)
catalog = rs.get_catalog(query)
dataset = list(catalog.datasets.values())[0]
Ryan May
@dopplershift
I can reproduce with that. I think we switched the server over to a new instance, so I’m guessing it’s a config problem. Let me see what I can do here...
Ryan May
@dopplershift
@jsimkins2 It should be fixed now.
James Simkins
@jsimkins2
@dopplershift it works now, thanks! Were the files altered at all? Just curious if my new netcdf error is a result of a new file structure or a pyart problem
Ryan May
@dopplershift
Nothing changed about the underlying files. Could be pyart or some other underlying problem.
Warren Pettee
@WEP11
Would adding a serial interface for hardware like Arduino be in Siphon's realm? Assuming the user programmed the Arduino to return some kind of standardized output.
Ryan May
@dopplershift
I feel like that’s beyond Siphon’s scope, since it’s highly unlikely any random user of siphon would have access to the appropriate hardware. Would probably be best to put something like that in a separate package.
Warren Pettee
@WEP11
It was just a random Saturday morning idea. The more I think about it, the more difficult it would be to envision how the implementation would fit cleanly in Siphon.
Ryan May
@dopplershift
Nothing wrong with crazy ideas :wink:
Mike
@mikebelanger
Hi guys, I'm really digging siphon so far! simplifies a lot for me
Ryan May
@dopplershift
Glad to hear it! Let us know if you run into any problems...
Mike
@mikebelanger
thanks! actually I'm just wondering if I could access a url like this: https://rda.ucar.edu/thredds/catalog/files/g/ds084.1/2017/20170101/catalog.html
I tried initializing a TDSCatalog instance with that url, except I swapped .html for .xml
archives = TDSCatalog('https://rda.ucar.edu/thredds/catalog/files/g/ds084.1/2017/20170101/catalog.xml')
and while that url exists - TDSCatalog returns a No name found
I dunno maybe I'm doing it wrong - but it looks like the constructor just needs a valid url
and I see a 'name' variable in that xml
Mike
@mikebelanger
oh wait - hmm if I go archives.datasets, it does indeed return a list of years
Ryan May
@dopplershift
Yeah, only get that if I try print(archives)
Mike
@mikebelanger
ah yeah sorry I didn't mention that before
Ryan May
@dopplershift
Oh. I see that's been around before. See #260.
Mike
@mikebelanger
hmm so not a big deal - I guess printing accesses the 'meta' part of the object
ah :P sorry ok thanks. I'll search through the issues first next time
Ryan May
@dopplershift
Nah, no worries.
Mike
@mikebelanger
so I was just reading through the issue of HTTP authentication, located here: Unidata/siphon#117
so I've added the sessionmanager instance
but now I'm wondering where I'd get a username and password.
I see a "register" button on the unidata homepage.
Is that where I'd get one?
Ryan May
@dopplershift
No, since you're using the RDA, you'd need to get it from them. There's a register now link here: https://rda.ucar.edu
Mike
@mikebelanger
oh I see thanks
Mike
@mikebelanger
so really liking this library so far! lots of good helpers
I have a question that might be more of a NetCDF4 question, not sure
let's say I make a DataQuery object, call it query
and do a lonlat_point query
and my dataset is a grib2 file
does it look for values based on the nearest grid point?
Mike
@mikebelanger
like there's no interpolation values between points or anything?
Ryan May
@dopplershift
@mikebelanger Thanks for the kind words. Yes, the lonlat_point request tells the THREDDS server, using a service called the netCDF Subset Service (NCSS), to return the data for the grid point closest to the requested location.
Mike
@mikebelanger
ah ok cool
@dopplershift yeah I really like using this service - siphon in combination with this THREDDs-based technology makes things easier
@dopplershift I got a bunch of grib2s that are filed away actually. And I'm looking at a scalable way of processing them
@dopplershift Maybe this question is better asked to THREDDs at large, but I'm wondering: how much setup is involved with making our own THREDDs instance?