Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 06:49
    pep8speaks commented #5501
  • 06:49
    grouny opened #5501
  • 00:52
    github-actions[bot] commented #5500
  • 00:28

    max-sixty on autoupdate-0

    (compare)

  • 00:27

    max-sixty on master

    pre-commit: autoupdate hook ver… (compare)

  • 00:27
    max-sixty closed #5500
  • 00:08
    github-actions[bot] opened #5500
  • 00:08

    github-actions[bot] on autoupdate-0

    pre-commit: autoupdate hook ver… (compare)

  • Jun 19 23:59
    max-sixty commented #5495
  • Jun 19 23:51
    AyrtonB commented #1068
  • Jun 19 22:20
    malmans2 commented #5495
  • Jun 19 21:26
    max-sixty commented #5365
  • Jun 19 21:16
    max-sixty commented #5495
  • Jun 19 21:03
    Illviljan synchronize #5494
  • Jun 19 18:42
    Illviljan commented #5365
  • Jun 19 18:33
    pep8speaks commented #5365
  • Jun 19 18:33
    Illviljan synchronize #5365
  • Jun 19 18:27
    Illviljan commented #5365
  • Jun 19 18:26
    malmans2 opened #5495
  • Jun 19 17:02
    max-sixty commented #5365
Ray Bell
@raybellwaves
Screen Shot 2021-03-02 at 3.22.20 PM.png
hmm not sure what happened to my eastward_wind_at_10_metres
Ray Bell
@raybellwaves
Screen Shot 2021-03-02 at 3.31.31 PM.png
comes back if you expand the files
g2giovanni
@g2giovanni
Hello everyone, I'm trying to compute zonal statistics following this tutorial: https://corteva.github.io/geocube/stable/examples/zonal_statistics.html. I saw that when I put a data-array inside a dataset (out_grid["elevation"] = elevation) I see that the output has a lot of nodata. But "elevation" has not this amount of nodata. Can you help me to figure out what happens?
Following the output I get:
Schermata 2021-03-23 alle 12.51.40.png
white lines are nodata values
James A. Bednar
@jbednar
Looks to me like an alignment problem, with coordinates slightly shifted between similar-looking arrays and xarray dutifully creating output that reflects its input but isn't what you wanted. I'd check that the data you're using is all sampled on compatible coordinates, and resample/regrid if necessary. I haven't used geocube so I can't give any more details for that (I'd personally be using datashader for the rasterizing plus xarray-spatial for the zonal statistics).
g2giovanni
@g2giovanni
Ok thanks. I'm giving a try to xarray-spatial-
James A. Bednar
@jbednar
You can still have similar problems with xarray-spatial, but if you use datashader to rasterize everything onto the same grid before using xarray-spatial, you should avoid problems like that.
epifanio
@epifanio
Hi, do you know if xarray API for resampling are fully compatible with pandas API ?
I am trying to resample a xarray.core.dataarray.DataArray but I got a n error: index must be monotonic for resampling - If I convert the dataarray to pandas, which returns a pandas.core.series.Series I can then resample the same data without errors. it is in the last two cells of the following notebook: https://nbviewer.jupyter.org/gist/epifanio/8c028c50789ff5fee590eef3eb78e4ac
can be because of NaN ?
epifanio
@epifanio
epifanio
@epifanio

@all thanks for your help (here and on discourse) I am getting an error with combining xarray+geoviews+datashader -- it seems to be related to xarray version (installed by conda, v0.17.0), but I am not sure - the error log says: AttributeError: 'DataArray' object has no attribute '_file_obj' when atemting to use the following code:

# mask
# xarray.core.dataarray.DataArray
dataset = gv.Dataset(mask)
datashade(dataset.to(gv.Image, ["lon", "lat"])

is it somethiong we can discuss here, or should I post here on discourse? this the full 'error_log'

epifanio
@epifanio
I got the exact same error running the example from https://geoviews.org/user_guide/Resampling_Grids.html - I guess I should post on pyviz
James A. Bednar
@jbednar
Yes, thanks!
Andrei Berceanu
@berceanu
What function should I use to combine two DataArray objects which share a common coordinate into a Dataset?
They have different dimensionality though, one is 3D the other 1D.
What I want to get at is to sum them over the common coordinate. Right now I need to turn them into numpy arrays and use np.sum(A, B[np.newaxis, np.newaxis, :], axis=-1).
And i'd like something like xr.sum(A, B, axis="x").
Not sure how the broadcasting would work in that case, either.
Guen Prawiroatmodjo
@guenp

Hi all! I have a question about the visualization of datasets in a jupyter notebook. I am storing datasets with nested metadata dicts in dataset.attr and want to save them to netCDF format. however, it looks like netCDF perhaps does not support this, so I ended up just saving the nested dicts as strings and deserialize them later. However, now when I view the dataset in a jupyter notebook, the output is this massive string that takes up more than the entire page. I was going to see if it's possible to truncate these metadata/attr values when they exceed a certain length. Does anyone here have any tips or ideas on other ways to resolve this, before I dig into this further?

image.png

winash12
@winash12
I am getting this error when I am trying to read a CFS analysis file from here - https://nomads.ncep.noaa.gov/pub/data/nccf/com/cfs/prod/
Traceback (most recent call last): File "test.py", line 9, in <module> ds = xr.open_dataset('./cdas1.t00z.pgrbanl.grib2',engine='cfgrib',backend_kwargs={'errors': 'ignore'}) File "/usr/local/lib/python3.8/dist-packages/xarray/backends/api.py", line 526, in open_dataset store = backends.CfGribDataStore( File "/usr/local/lib/python3.8/dist-packages/xarray/backends/cfgrib_.py", line 43, in __init__ self.ds = cfgrib.open_file(filename, **backend_kwargs) File "/usr/local/lib/python3.8/dist-packages/cfgrib-0.9.8.5-py3.8.egg/cfgrib/dataset.py", line 651, in open_file File "/usr/local/lib/python3.8/dist-packages/cfgrib-0.9.8.5-py3.8.egg/cfgrib/dataset.py", line 596, in build_dataset_components cfgrib.dataset.DatasetBuildError: multiple values for unique key, try re-open the file with one of: filter_by_keys={'typeOfLevel': 'meanSea'} filter_by_keys={'typeOfLevel': 'isobaricInhPa'} filter_by_keys={'typeOfLevel': 'surface'} filter_by_keys={'typeOfLevel': 'atmosphereSingleLayer'} filter_by_keys={'typeOfLevel': 'heightAboveGroundLayer'} filter_by_keys={'typeOfLevel': 'tropopause'} filter_by_keys={'typeOfLevel': 'maxWind'} filter_by_keys={'typeOfLevel': 'heightAboveSea'} filter_by_keys={'typeOfLevel': 'isothermZero'} filter_by_keys={'typeOfLevel': 'highestTroposphericFreezing'} filter_by_keys={'typeOfLevel': 'pressureFromGroundLayer'} filter_by_keys={'typeOfLevel': 'sigmaLayer'} filter_by_keys={'typeOfLevel': 'sigma'} filter_by_keys={'typeOfLevel': 'potentialVorticity'}
winash12
@winash12
ds = xr.open_dataset('./cdas1.t00z.pgrbanl.grib2',engine='cfgrib',backend_kwargs={'errors': 'ignore'}) print(ds) That's how I am reading that file
and I have looked at this issue - ecmwf/cfgrib#66
Kai Mühlbauer
@kmuehlbauer
@winash12 You would need to add one of the mentioned kwargs to the backend_kwargs. Something along the lines:
`xr.open_dataset('./cdas1.t00z.pgrbanl.grib2',engine='cfgrib',backend_kwargs={'errors': 'ignore', 'filter_by_keys': {'typeOfLevel': 'potentialVorticity'} })
`
winash12
@winash12
@kmuehlbauer Thanks for that response. What if I want to filter by three keys such as meanSea and heightAboveSea and isobaricInhPa ?
I tried this ds = xr.open_dataset('./cdas1.t00z.pgrbanl.grib2',engine='cfgrib',backend_kwargs={'errors': 'ignore', 'filter_by_keys': {'typeOfLevel': 'meanSea'}, 'filter_by_keys': {'typeOfLevel': 'heightAboveSea'}})
but this prints only the variables of heightAboveSea i.e. the last filter by key
Kai Mühlbauer
@kmuehlbauer
@winash12 That won't work IIRC. You would need to open each of these datasets separately. but you can merge them afterwards.
winash12
@winash12
@kmuehlbauer That is what I thought thanks. In your opinion is this a bug ?
Kai Mühlbauer
@kmuehlbauer
@winash12 I do not think this is a bug, but a direct consequence of the GRIB format. The filtering is executed when indexing the file. You might get more detailed answers from the cfgrib-maintainers.
winash12
@winash12
@kmuehlbauer Final question on this topic
I did open three datasets for each of the three filter by keys
and then appended them. Then I tried to concatenate them but I am not sure of the argument to dim

`datasets = []

datasets.append(ds)
datasets.append(ds1)
datasets.append(ds2)

combined = xr.concat(datasets,dim='example')`

Kai Mühlbauer
@kmuehlbauer
I would use xr.merge instead.
winash12
@winash12
ok this is how I got it to go eventually - dsfinal = xr.merge([ds, ds1, ds2],compat='override')
Kai Mühlbauer
@kmuehlbauer
:+1: @winash12
winash12
@winash12
Hello. I am not sure if my question is fully answered. Here is the description of the problem. I have this index file here - https://nomads.ncep.noaa.gov/pub/data/nccf/com/cfs/prod/cfs/cfs.20210420/06/6hrly_grib_04/flxf2021042006.04.2021042006.grb2.idx and I am looking to extract three variables from this - UGRD, VGRID and TMP - u 10 m and v 10 m and temp 2 m. But when I filter by key (heightAboveGround) I get u10 and v10. So the question is why is the metadata variable output of xarray(print(ds)) different from the index file ? How will I know if these are the same variables ? Please let me know if this question is appropriate for this group or not. I asked NOAA support regarding this and they said this is question is beyond their purview.
winash12
@winash12
I am hesitant to write up a bug for such a trivial issue of reading data . Why ? I can read the data just fine using pygrib.
g2giovanni
@g2giovanni

Hello everyone. I'd like to compute zonal statistics between:

  • zones array has a spatial resolution of 10m
  • values array has a spatial resolution of 7km

but they refer to the same spatial location. To be able to compute zonal-stats (with xarray-spatial library) I have to upsample values array from 7km to 10m spatial resolution. What is the best way to upsample my xarray DataArray? Trying to interpolate with xarray.interp function I have issues (huge RAM usage) and anyway, the computation is not easily scalable to bigger areas. Any suggestion?

James A. Bednar
@jbednar
Datashader includes some types of upsampling using Canvas.raster or indirectly using HoloViews hv.Image or GeoViews gv.Image, and should be pretty fast. xesmf can also do regridding.
orttak
@orttak
stack_bands = xr.concat(input_imgs_list, dim='band', join='override' )
is it correct method to merge Sentinel-2 images to stack and expor as multiband raster ?
leonardojimenez1990
@leonardojimenez1990
Hello, I would like to know if it is correct to add DataArray in this way?
SumarDataArray.png
2 replies