Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Daniel Shapero
@danshapero
I have this ugly data repo that I ask people to get along with a glacier flow modeling package that I'm developing, but this way I should be able to get rid of it entirely
Leonardo Uieda
@leouieda
Hi @danshapero I'm glad it worked for you! Please let me know how it goes
Daniel Shapero
@danshapero
It would be nice if you could add a download progress bar, using say tqdm or python-progressbar
Is there a good way to add this with the current API or would it require modifications to pooch?
Justin Gosses
@JustinGOSSES
Just wondering if someone could confirm that https://github.com/fatiando/rockhound/blob/master/CONTRIBUTING.md instructions for installing the local copy of rockhound in develop mode work as written? I'm pretty sure I've seen that language in other project, so I don't know if they were maybe default from a cookie cutter?
When I follow those instructions to install and than try to import rockhound in a notebook, I get:
OSError                                   Traceback (most recent call last)
<ipython-input-4-cd6a10f5da64> in <module>()
----> 1 import rockhound as rh

~/Code/rockhound/rockhound/__init__.py in <module>()
      2 # Import functions/classes to make the public API
      3 from . import version
----> 4 from .registry import data_location
      5 from .etopo1 import fetch_etopo1
      6 from .prem import fetch_prem

~/Code/rockhound/rockhound/registry.py in <module>()
     10     path=pooch.os_cache("rockhound"), base_url="", env="ROCKHOUND_DATA_DIR"
     11 )
---> 12 REGISTRY.load_registry(os.path.join(os.path.dirname(__file__), "registry.txt"))
     13 
     14 

~/anaconda/lib/python3.5/site-packages/pooch/core.py in load_registry(self, fname)
    348                     raise IOError(
    349                         "Expected 2 or 3 elements in line {} but got {}.".format(
--> 350                             linenum, len(elements)
    351                         )
    352                     )

OSError: Expected 2 or 3 elements in line 9 but got 0.
Leonardo Uieda
@leouieda

Is there a good way to add this with the current API or would it require modifications to pooch?

@danshapero I was thinking that just yesterday. I think we would need modification to the current HTTPDownloader to make that work. It would be good to have a verbose=False flag so that current behaviour doesn't change and projects that want it can set verbose=True. Would you mind opening an issue for this? (Also, would you like to take a shot at implementing it?)

Leonardo Uieda
@leouieda

Just wondering if someone could confirm that https://github.com/fatiando/rockhound/blob/master/CONTRIBUTING.md instructions for installing the local copy of rockhound in develop mode work as written? I'm pretty sure I've seen that language in other project, so I don't know if they were maybe default from a cookie cutter?

@JustinGOSSES yep, all of those come from https://github.com/fatiando/contributing but they should work the same. I'm trying it here and everything seems to work properly. Did you make any modifications to the registry.txt file? That error message is kind of uninformative (opened fatiando/pooch#88 to change it).

Justin Gosses
@JustinGOSSES
Here's my fork, I'm working in McMurrayMess branch. I added information for one file to the registry.txt file, created a mcmurry_facies.py in rockhound basing it off of prem.py ,and added a line for from .mcmurray_facies import fetch_mcmurray_facies towards top of the init.py. I don't have the problem above when I import rockhound if I do a conda install of the verion on pypy instead of local install.
Leonardo Uieda
@leouieda
Ah, I see the problem. There is a blank line at the end of rockhound/registry.txt that shouldn't be there. This is a bit stupid of Pooch not to catch this error. For now, if you delete that it should be no problem. The module name also appears to be wrong in the __init__.py: mcmurray instead of mcmurry
Leonardo Uieda
@leouieda
OK, created fatiando/pooch#89 to request a fix to Pooch.load_registry so that it can be smarter about this sort of thing. If anyone wants to take a shot, this is a great way to get started with a contribution :wink:
Justin Gosses
@JustinGOSSES
I just found that too and came here to write it. Ugh. white spaces...
Daniel Shapero
@danshapero
@leouieda Just opened an issue for progress bars, I can write it
One thing I've encountered before is that you have to mess about with how the progress bar works depending on whether you're using python from a terminal or from a jupyter notebook
I think tqdm has some way to handle each but then it's up to you to choose
David Hoese
@djhoese
@leouieda I saw you had the GSHHG netcdf files on conda-forge. How did the conda-forge team feel about a package like that existing on their systems? I have a package that uses the shapefiles from GSHHG and it'd be really convenient for users if they could download them as part of conda/conda-forge.
Leonardo Uieda
@leouieda
@djhoese that was actually their idea, not mine. So I imagine they are fine with having data packages like that. Would you want to add the shapefiles to the current package or make a new one? If they are large, it might be worth having them separate to avoid unnecessary downloads.
@danshapero that's a good point. I don't know how they handle it or if it works with notebooks or jupyterlab. We might need to test it out and see what happens. Knowing if something is running on a notebook or not is not a trivial thing (almost impossible), as jupyter is designed in a way that the frontend shouldn't matter. I've struggled with this before and didn't find a decent solution.
David Hoese
@djhoese
@leouieda For some reason I remember the shape files being small, but just tried downloading the zip file and it was 142MB
Leonardo Uieda
@leouieda
@djhoese in that case it might be best to create a separate package for the shapefiles. That's a big download to do unnecessarily when installing other packages like GMT
David Hoese
@djhoese
agreed
thanks
Shib Ganguli
@dst_faculty_twitter
@leouieda Hi! I tried using the code by you in jupiter and got the following:

ImportErrorTraceback (most recent call last)

<ipython-input-6-2a44b6a72d52> in <module>()
----> 1 from mohoinv import TesseroidRelief, MohoGravityInvSpherical, make_mesh
2 from mohoinv import split_data, score_test_set, score_all, fit_all

ImportError: No module named mohoinv, while trying using the cell: from mohoinv import TesseroidRelief, MohoGravityInvSpherical, make_mesh
from mohoinv import split_data, score_test_set, score_all, fit_all

Leonardo Uieda
@leouieda
@dst_faculty_twitter can you please provide some more information. What exactly did you run, how did you install things, which notebook are you talking about? Make sure that the mohoinv.py file from this repository is in the same directory as the notebooks: https://github.com/pinga-lab/paper-moho-inversion-tesseroids
Shib Ganguli
@dst_faculty_twitter
@leouieda Many thanks for your reply. Have emailed you regarding this.
sylvan messenger
@sylvanmessnger_twitter
Hi, i wanted to use your seismic.wave psv code, but instead of the 3 padding areas + free surface at the top i want to have 4 padding areas. Therefore i tried to change the code and see how you did it. But i can not think of a way to see how the builtin function from fatiando.seismic._wavefd : _apply_damping() is written. Is there some way to look at the code of this function ?
Santiago Soler
@santisoler
Hi @sylvanmessnger_twitter. All Fatiando a Terra source code is available online on different GitHub repositories: https://github.com/fatiando/
They are all released under open-source licenses, so feel free to download them, study them, modify them and even share those modifications.
The code for fatiando.seisimic_wavefd is part of the deprecated fatiando package, although you can still use it if you want to. Specifically, the fatiando.seismic._apply_damping() functions is implemented on Cython in the following file: https://github.com/fatiando/fatiando/blob/master/fatiando/seismic/_wavefd.pyx
sylvan messenger
@sylvanmessnger_twitter
thank you @santisoler .
I looked a bit at github and in my fatiando folder and only found the _wavefd.pyd but wasn´t able to open it.
You mentioned it´s part of the deprecated fatiando package, is there another one ?
Santiago Soler
@santisoler
Does your local filename is _wavefd.pyd? The original one is named _wavefd.pyx. How are you opening it? Are you opening it from your File Manager or from a Text Editor/IDE?

You mentioned it´s part of the deprecated fatiando package, is there another one ?

@sylvanmessnger_twitter In fact there are four new packages (so far):

  • Verde
  • Pooch
  • Rockhound
  • Harmonica
Fatiando a Terra is currently undergoing through a big refactor, partially triggered by the end of life of Python 2.7. You can find more information about the current development of Fatiando a Terra on its website: https://www.fatiando.org .
@leouieda wrote a post on his personal blog explaining why we decided to subdivide the project into different packages.
sylvan messenger
@sylvanmessnger_twitter
yes my local filename is _wavefd.pyd. There is no .pyx file in there. I tried text editor(s) and inside my anaconda python prompt. It does import the package as it should be, but i just couldn´t find the source code (to modify it).
Thanks. i will look into the other packages.
Santiago Soler
@santisoler

@sylvanmessnger_twitter Ok. The .pyd files windows dll files created when the the Cython .pyx are compiled.
In order to modify the sources files you should clone the GitHub fatiando repository:

git clone https://github.com/fatiando/fatiando

Git will download the repository on your current directory. There you will find the latest version of the sources files.
Please read the documentation and the README.md.
Because the code for _wavefd.pyx is written in Cython, you must compile them after any modification. Otherwise you'll run the previous compiled version without any new modification.

Leonardo Uieda
@leouieda
@sylvanmessnger_twitter yep, @santisoler is right. The pyd file is a compiled binary file generated when we package fatiando for distribution. The Cython setup is a bit complex: the Cython code gets transformed into C which is compiled into a binary Python library which gets imported in wavefd. Changing the code would involve changing the Cython code, regenerating the C, and recompiling the library. Take a look at the wavefd branch on Github which has a new version of the code we were working on. It uses the numba library instead of Cython so the whole compilation step is taken out and done automatically: https://github.com/fatiando/fatiando/tree/wavefd/fatiando/seismic/wavefd It also has a much nicer class interface instead of the clumsy functions in the 0.5 implementation.
sylvan messenger
@sylvanmessnger_twitter
thank you very much. I will look into it this weekend.
Leonardo Uieda
@leouieda
🚨 Verde 1.2.0 is out! 🚨
Includes bug fixes and new features: cross-validated Spline, longitude handling, reading Surfer grids, new tutorials, and more. See what's new at https://www.fatiando.org/verde/latest/changes.html#version-1-2-0
Charles Rudder
@clrudder
Is there a straight forward way to force the Verde Spline interpolation to a buffered region? I have point data that interpolates well using the BlockReduce followed by Spline but generally ( due to the way the data are collected) never reaches the edges of the AOI im using and the interpolation doesnt fill the AOI polygon. i suppose I can modify the region and expand my interpolation area accordingly??
Leonardo Uieda
@leouieda
Hi @clrudder, let me see if I understand that correctly: you want to generate a grid that extends beyond the region where you have data. Is that correct? If so, the .grid method uses the region of the data by default but you can always pass in region=[west, east, south, north] to .grid to get interpolated values on any desired region (see here). See also the verde.pad_region (here) and verde.get_region (here) functions that are convenient in these situations.
Charles Rudder
@clrudder
thank you. Sounds like i was on the right path but I suspect the verde.pad_region is a better solution... Cheers
Leonardo Uieda
@leouieda
Glad to help :-) Let us know if there is anything you find missing in the library. We're always looking for ways to expand.
Daniel Shapero
@danshapero
Finally got around to adding a progress bar for pooch, just opened up the PR
Also you can add icepack: https://icepack.github.io as a project using a pooch
Leonardo Uieda
@leouieda
Thanks @danshapero! I'll have a look at the code as soon as I can. For reference this is fatiando/pooch#97
Leonardo Uieda
@leouieda
Hi @/all we're trying out a new Slack group that would replace our mailing list and Gitter. You can sign up for it through this invite link: https://join.slack.com/t/fatiando/shared_invite/enQtNzI5ODk4MDU3MzYzLTMyN2EwZjYwNTQzNWI4YzUzOGQyYjNkZTJlMmFiNDI2OThkYjMxNWQwNmZkNTc1ZDUxNmM2Mjg2Yjg4YTBlZjc
Later on we'll create a sign up form on the website.
Hope to see you there! :rocket: