Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Sep 05 01:47

    michaelaye on master

    improve configpath handling add more tests for pdstools ind… (compare)

  • Sep 05 01:46
    michaelaye edited #5
  • Sep 05 01:46
    michaelaye opened #5
  • Sep 05 01:37

    michaelaye on master

    Update pds_indices_db.toml Add… (compare)

  • Sep 04 18:09

    michaelaye on master

    Update pds_indices_db.toml Add… (compare)

  • Sep 04 18:06

    michaelaye on master

    Update pds_indices_db.toml add… (compare)

  • Aug 12 03:18

    michaelaye on master

    update travis to xenial (3.7 di… (compare)

  • Aug 12 03:12

    michaelaye on master

    improve pds index file handling… add get_remote_timestamp improve IndexDB functionality … (compare)

  • Aug 02 16:58

    michaelaye on master

    renamed indices_paths.toml to i… add last-modified time stamps t… rename again, to be better iden… (compare)

  • Jul 11 05:54

    michaelaye on 0.5.1

    (compare)

  • Jul 11 05:50

    michaelaye on v0.5.1

    (compare)

  • Jul 11 05:50

    michaelaye on master

    fixing old pypi upload Bump version: 0.5.0 -> 0.5.1 (compare)

  • Jul 11 05:47

    michaelaye on master

    changing name to planetarypy Bump version: 0.4.6 -> 0.5.0 (compare)

  • Jul 11 05:39

    michaelaye on master

    remove the rarely working lands… (compare)

  • Jul 11 05:38

    michaelaye on master

    change name (compare)

  • Jul 08 16:47

    michaelaye on 0.4.6

    (compare)

  • May 22 06:51

    michaelaye on master

    Delete planetpy.sublime-project (compare)

  • May 14 18:42

    michaelaye on master

    Adding ability to download a gi… (compare)

  • Mar 23 04:13

    michaelaye on master

    add importlib_resourtes to inst… Bump version: 0.4.5 -> 0.4.6 (compare)

  • Mar 23 04:12

    michaelaye on master

    add pdstools toml file to MANIF… Bump version: 0.4.4 -> 0.4.5 (compare)

J. McAuliffe
@johnnycakes79
and html5lib
and beautifulsoup
Victory!!!
OK! Hometime... thanks for the butt-kick to finish my move to Py3 and Conda.
K.-Michael Aye
@michaelaye
great!
@msbentley understand now how u were able to avoid pandas 0.16 install. i only asked for any pandas, not pandas 0.16. NOW i changed that. ;)
Mark S. Bentley
@msbentley
OK, now works great with a 2.7 virtualenv and all dependencies. As for 3.x I want to make the switch, but have no time to do the necessary reading and code changes for MIDAS planning/ops/analysis. As soon as Rosetta dies ;)
K.-Michael Aye
@michaelaye
with a py3.x virtualenv u could slowly get used to it. just don’t start any new projects with 2.7 if they don’t need anything old.
Mark S. Bentley
@msbentley
:thumbsup:
J. McAuliffe
@johnnycakes79
@msbentley are you using Anaconda/Conda?
K.-Michael Aye
@michaelaye
i forgot one file in last commit and amended it to it.
Mark S. Bentley
@msbentley

@johnnycakes79 Funnily enough I gave it a whirl today after the above discussion. So far I've just been using my distribution's packages, but they're always somewhat behind.

Is there a consensus that Anaconda is better than Enthought's Canopy? And how does conda's virtual environment implementation compare to virtualenv?

J. McAuliffe
@johnnycakes79
I hear it's better but I'm a little confused by it... I need an idiot's guide... I have built over the years a monster setup of virtualenv and cookiecutter
K.-Michael Aye
@michaelaye

So, I used Enthought for years, and the only reason to leave it for me was an ungodly complication if one wants to set it up for more than one user on a shared machine. @johnnycakes79 once it makes click with (ana)conda it’s so supereasy. here’s your idiot’s guide. ;)

  1. Decide if you want someone else at anaconda guard and guide your update cycles or you want that to do yourself. If you only want super-sanctioned updates that have been tested in a whole huge set of environment with a lot of other python packages, then install anaconda. If you feel like you have grown enough in terms of solving Python setup problems, then Miniconda is the way. The main differences: a) Anaconda is a distribution so a large set of packages that are distributed as a whole. Miniconda comes with bare essentials and then you install what you need with conda install package_name. b) Updating Anaconda only updates when someone went through the whole test-shebang for the Anaconda distribution. Miniconda packages updates when someone tested the packages itself only. My preferred choice: Miniconda, because when a package maintainer released a bugfix I don’t want to wait until Continuum.IO is releasing an update to the Anaconda dist, I just update the package itself.
  2. Now that you have realised that you really should install Miniconda, choose the base python version: I call the base python a python that is active if no virtual env is active. I am fine with this being py3.x so I installed the py3.x Miniconda. if you want your base python 2.x then install that one, but it’s absolutely no prob to install a virtual env with the other python on top of that.
  3. After installing your Miniconda of choice, these commands should install a good amount of your required science stack:
    conda install ipython-notebook scipy numpy pandas matplotlib basemap pip
    # if you want image analysis tools and some machine learning tools like clustering
    conda install scikit-image scikit-learn
  4. If you now need a virtual env of the other python:
    conda create -n py34 ipython-notebook python=3
    What this does is create a virtual environment called py34, and it start’s by installing all ipython-notebook packages, and you finish with conda install the_rest_from_above. In a terminal this virtual env is now activated by source activate py34 and deactivated by source deactivate.
  5. Whenever you now need to install a git repo into one of your envs, do like this:
    source activate py34
    python setup.py install (or develop)
    The same is true for pip installs (apart from old messed up beautifulsoup3). You first activate your env and then the pip is launched from inside that env so that things find their way correctly.

If you have any more questions, feel free to ping me.

For more advertisement: With conda I am able to completely reinstall my scientific python stack in under 15 minutes.
K.-Michael Aye
@michaelaye
The cool thing that a virtual environment for, let’s say a special version of pandas under python3 is super easy created:
$ conda create -n pandastest pandas=0.15 python=3
Fetching package metadata: ....
Solving package specifications: .............
Package plan for installation in environment /Users/klay6683/miniconda3/envs/pandastest:

The following NEW packages will be INSTALLED:

    numpy:           1.9.2-py34_0
    openssl:         1.0.1k-1
    pandas:          0.15.2-np19py34_1
    pip:             6.1.1-py34_0
    python:          3.4.3-0
    python-dateutil: 2.4.2-py34_0
    pytz:            2015.2-py34_0
    readline:        6.2-2
    setuptools:      15.0-py34_0
    six:             1.9.0-py34_0
    sqlite:          3.8.4.1-1
    tk:              8.5.18-0
    xz:              5.0.5-0
    zlib:            1.2.8-0

Proceed ([y]/n)? y

Linking packages ...
[      COMPLETE      ]|#########################################################################| 100%
#
# To activate this environment, use:
# $ source activate pandastest
#
# To deactivate this environment, use:
# $ source deactivate
and you list all installed environments with conda info -e
J. McAuliffe
@johnnycakes79
what about setting up an env with a cookiecutter template...? any ideas?
K.-Michael Aye
@michaelaye
i don’t understand the relation ship between a cookiecutter template (which is for a python package I presume) and a package manager? What do you want to do?
a new python package is or should be in principle independent from a package manager. so i guess what you want is:
  1. create new package with cookiecutter (which should have a setup.py if it’s a good template)
  2. create environment for testing and activate it
  3. go to new package and do python setup.py develop
K.-Michael Aye
@michaelaye
i had a look at it, it has grown a lot since the last time i checked them. i am interested in adding the auto-testing and sphinx stuff in this so will do that next.
J. McAuliffe
@johnnycakes79
virtualenv has hooks that can be run before (or after) a new virtualenv is created/deleted/whatever... I used to these and a python package cookiecutter template to start a new project with all the necessary bits to be easily pushed to pypi.
do you know if conda env creation has such hooks
K.-Michael Aye
@michaelaye
i don’t know. but maybe it still would work on top of it?
which cookiecutter template did u use? it looks like tox should take care of these environment creations for you, so that you don’t have to know this yourself.
J. McAuliffe
@johnnycakes79
:P yeah I read that diagonally but missed the conda skeleton part
K.-Michael Aye
@michaelaye
that article is still not putting me at ease though (see my comments)
J. McAuliffe
@johnnycakes79
What I miss when making the conda envs is hooks to prep a package directory.
K.-Michael Aye
@michaelaye
I even don't know what that means. :)
And I assume that u have to know this means that u r not using the path of least resistance but I might be wrong because I also want these auto tests to happen.
J. McAuliffe
@johnnycakes79
Huh? Path of least resistance? In what respect?
K.-Michael Aye
@michaelaye
I'm just hoping / thinking that one doesn't need to know about these hooks yourself. Feels hackish. As I said , I might be wrong
J. McAuliffe
@johnnycakes79
OK, after a bit more re-reading my own notes and bash scripts I've come to the conclusion what I need or perhaps what is needed is a virtualenvwrapper for conda envs ...
... I'll get started!
K.-Michael Aye
@michaelaye
@johnnycakes79 can u remind me what do u need these hooks for? After reading a bit myself, it seems that tox is redundant if we/you would use travis for your automatic checks? As I store my stuff on github all the time, I am actually fine with using travis only and travis can check multiple python versions as well, so no need for tox anymore.
Mark S. Bentley
@msbentley
Coming back to what one might want to include, an interface to the PDS/PSA via the Planetary Data Access Protocol (PDAP) would be great. It would be really nice to be able to pull data and work on it in e.g. an IPython Notebook for complete reproducibility...
K.-Michael Aye
@michaelaye
oh, didn't know about PDAP, i recently wrote a wrapper around the OPUS api used in the PDS planetary rings node. Works pretty easy, but they still have bugs in it from times to times.
Mark S. Bentley
@msbentley
I have to admit I've never tried it, but I really hate the java app that the PSA uses for queries and was looking for a better way to get data: http://psa.esac.esa.int:8000/aio/doc/
J. McAuliffe
@johnnycakes79
The new psa interface has an api. Are you aware of it? I don't know if it's public yet...
Mark S. Bentley
@msbentley
I'm certainly not aware of it - I only know the above. A new API would be great - that java client is a pile of dingo's kidneys ;)
J. McAuliffe
@johnnycakes79
Well, I just checked that the API is not public yet - still in beta/prototyping but I've seen it and it's promising...
BTW, we just finished the first Planetary GIS Workshop at ESAC and some great things are likely to come out of it... There'll be a report soon enough but one idea was the setting up of a github repo for tools - I mentioned this repo and suggested the setting up of a specific account instead, under which many repos could be accumulated.
I think having @msbentley and @michaelaye involved would be a good addition.
Mark S. Bentley
@msbentley
:thumbsup:
Mark S. Bentley
@msbentley
Just saw this - not sure if it's covered by other libraries or would be a good thing to consider including? (reads PDS images into numpy arrays etc.) https://pypi.python.org/pypi/planetaryimage/0.1.0
K.-Michael Aye
@michaelaye
it is a consolidation effort by the guys around the author of the Pysis ISIS interface. I’m not convinced this is the way to go because the gdal Interface is the far more complete GIS based raster reading library.
and it already could read most standard PDS images, just not the labels nicely.
Mark S. Bentley
@msbentley
Got it :thumbsup: