These are chat archives for dereneaton/ipyrad

12th
Sep 2016
leclearnm
@leclearnm
Sep 12 2016 19:31
Hi,
I just tried making a fresh installation of ipyrad using Miniconda and am getting the following error.
login1.ls5(26)$ ipyrad
Traceback (most recent call last):
  File "/home1/03530/nleclear/miniconda2/bin/ipyrad", line 11, in <module>
    load_entry_point('ipyrad==0.3.41', 'console_scripts', 'ipyrad')()
  File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 542, in load_entry_point
  File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 2569, in load_entry_point
  File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 2229, in load
  File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 2235, in resolve
  File "/home1/03530/nleclear/miniconda2/lib/python2.7/site-packages/ipyrad/__init__.py", line 22, in <module>
    from . import load
  File "/home1/03530/nleclear/miniconda2/lib/python2.7/site-packages/ipyrad/load/__init__.py", line 13, in <module>
    from .load import test_assembly
  File "/home1/03530/nleclear/miniconda2/lib/python2.7/site-packages/ipyrad/load/load.py", line 10, in <module>
    import pandas as pd
  File "/home1/03530/nleclear/miniconda2/lib/python2.7/site-packages/pandas/__init__.py", line 18, in <module>
    raise ImportError("Missing required dependencies {0}".format(missing_dependencies))
ImportError: Missing required dependencies ['numpy']
Deren Eaton
@dereneaton
Sep 12 2016 19:36
Hi @leclearnm, hmm, I'm trying to think of how ipyrad could be installed in your miniconda directory without numpy being there as well... seems odd. Can you run conda list and paste the output here.
Also conda info
leclearnm
@leclearnm
Sep 12 2016 19:38
login1.ls5(27)$ conda list
# packages in environment at /home1/03530/nleclear/miniconda2:
#
Using Anaconda API: https://api.anaconda.org
_nb_ext_conf              0.3.0                    py27_0  
alabaster                 0.7.8                    py27_0  
anaconda-client           1.5.1                    py27_0  
babel                     2.3.3                    py27_0  
backports                 1.0                      py27_0  
backports_abc             0.4                      py27_0  
bitarray                  0.8.1                    py27_0  
cairo                     1.12.18                       6  
clyent                    1.2.2                    py27_0  
conda                     4.1.12                   py27_0  
conda-env                 2.5.2                    py27_0  
configparser              3.5.0                    py27_0  
cython                    0.24.1                   py27_0  
decorator                 4.0.10                   py27_0  
docutils                  0.12                     py27_2  
entrypoints               0.2.2                    py27_0  
enum34                    1.1.6                    py27_0  
fontconfig                2.11.1                        6  
freetype                  2.5.5                         1  
funcsigs                  1.0.2                    py27_0  
functools32               3.2.3.2                  py27_0  
futures                   3.0.5                    py27_0  
get_terminal_size         1.0.0                    py27_0  
glib                      2.43.0                        1  
h5py                      2.6.0               np111py27_2  
harfbuzz                  0.9.39                        1  
hdf5                      1.8.17                        1  
imagesize                 0.7.1                    py27_0  
ipykernel                 4.5.0                    py27_0  
ipyparallel               5.2.0                    py27_0  
ipyrad                    0.3.41                        0    ipyrad
ipython                   5.1.0                    py27_0  
ipython_genutils          0.1.0                    py27_0  
ipywidgets                5.2.2                    py27_0  
jinja2                    2.8                      py27_1  
jsonschema                2.5.1                    py27_0  
jupyter                   1.0.0                    py27_3  
jupyter_client            4.4.0                    py27_0  
jupyter_console           5.0.0                    py27_0  
jupyter_core              4.1.1                    py27_0  
libffi                    3.2.1                         0  
libgfortran               3.0.0                         1  
libpng                    1.6.22                        0  
libsodium                 1.0.10                        0  
libxml2                   2.9.2                         0  
llvmlite                  0.13.0                   py27_0  
markupsafe                0.23                     py27_2  
mistune                   0.7.2                    py27_0  
mkl                       11.3.3                        0  
mpi4py                    2.0.0                    py27_1  
mpich2                    1.4.1p1                       0  
nb_anacondacloud          1.2.0                    py27_0  
nb_conda                  2.0.0                    py27_0  
nb_conda_kernels          2.0.0                    py27_0  
nbconvert                 4.2.0                    py27_0  
nbformat                  4.1.0                    py27_0  
nbpresent                 3.0.2                    py27_0  
notebook                  4.2.2                    py27_0  
numba                     0.28.1              np111py27_0  
numpy                     1.11.1                   py27_0  
openssl                   1.0.2h                        1  
pandas                    0.18.1              np111py27_0  
pango                     1.39.0                        1  
path.py                   8.2.1                    py27_0  
pathlib2                  2.1.0
login1.ls5(28)$ conda info
Current conda install:

             platform : linux-64
        conda version : 4.1.12
    conda-env version : 2.5.2
  conda-build version : not installed
       python version : 2.7.12.final.0
     requests version : 2.10.0
     root environment : /home1/03530/nleclear/miniconda2  (writable)
  default environment : /home1/03530/nleclear/miniconda2
     envs directories : /home1/03530/nleclear/miniconda2/envs
        package cache : /home1/03530/nleclear/miniconda2/pkgs
         channel URLs : https://repo.continuum.io/pkgs/free/linux-64/
                        https://repo.continuum.io/pkgs/free/noarch/
                        https://repo.continuum.io/pkgs/pro/linux-64/
                        https://repo.continuum.io/pkgs/pro/noarch/
          config file : None
         offline mode : False
    is foreign system : False
Deren Eaton
@dereneaton
Sep 12 2016 19:42
Hmm, well numpy is definitely installed. I'm not sure why ipyrad could not find it. I wonder if it has something to do with being on the login node. Though I'm not sure why that would matter. Try connecting to an interactive compute node for testing. Depending on your cluster, this can usually be done with a command like qsub -I.
Isaac Overcast
@isaacovercast
Sep 12 2016 19:44
The environment on the compute node could totally be different, that's a good thing to check.
Deren Eaton
@dereneaton
Sep 12 2016 19:49
@isaacovercast but the error message says that it's already looking in the miniconda dir for the software. Seems like it should be able to find it in there.
leclearnm
@leclearnm
Sep 12 2016 20:05
I tried it through the interactive development node and received the same error
Deren Eaton
@dereneaton
Sep 12 2016 20:05
Try typing ipython to open an interactive Python session and then copying what's below. This might help troubleshoot the problem.
## check your system path
import sys
sys.path

## check that numpy is installed
import numpy
print numpy.__version__

## check that ipyrad is installed
import ipyrad 
print ipyrad.__version__

exit
leclearnm
@leclearnm
Sep 12 2016 20:11
Python 2.7.12 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:42:40) 
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.
leclearnm
@leclearnm
Sep 12 2016 20:24
This message was deleted
In [1]: import sys

In [2]: sys.path
Out[2]: 
['',
 '/home1/03530/nleclear/miniconda2/bin',
 '/home1/03530/nleclear/miniconda2/lib/python27.zip',
 '/home1/03530/nleclear/miniconda2/lib/python2.7',
 '/home1/03530/nleclear/miniconda2/lib/python2.7/plat-linux2',
 '/home1/03530/nleclear/miniconda2/lib/python2.7/lib-tk',
 '/home1/03530/nleclear/miniconda2/lib/python2.7/lib-old',
 '/home1/03530/nleclear/miniconda2/lib/python2.7/lib-dynload',
 '/home1/03530/nleclear/.local/lib/python2.7/site-packages',
 '/home1/03530/nleclear/miniconda2/lib/python2.7/site-packages/Sphinx-1.4.1-py2.7.egg',
 '/home1/03530/nleclear/miniconda2/lib/python2.7/site-packages/setuptools-23.0.0-py2.7.egg',
 '/home1/03530/nleclear/miniconda2/lib/python2.7/site-packages',
 '/home1/03530/nleclear/miniconda2/lib/python2.7/site-packages/IPython/extensions',
 '/home1/03530/nleclear/.ipython']
In [3]: import numpy
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-3-5a0bd626bb1d> in <module>()
----> 1 import numpy

/home1/03530/nleclear/.local/lib/python2.7/site-packages/numpy/__init__.py in <module>()
    178         return loader(*packages, **options)
    179 
--> 180     from . import add_newdocs
    181     __all__ = ['add_newdocs',
    182                'ModuleDeprecationWarning',

/home1/03530/nleclear/.local/lib/python2.7/site-packages/numpy/add_newdocs.py in <module>()
     11 from __future__ import division, absolute_import, print_function
     12 
---> 13 from numpy.lib import add_newdoc
     14 
     15 ###############################################################################

/home1/03530/nleclear/.local/lib/python2.7/site-packages/numpy/lib/__init__.py in <module>()
      6 from numpy.version import version as __version__
      7 
----> 8 from .type_check import *
      9 from .index_tricks import *
     10 from .function_base import *

/home1/03530/nleclear/.local/lib/python2.7/site-packages/numpy/lib/type_check.py in <module>()
      9            'common_type']
     10 
---> 11 import numpy.core.numeric as _nx
     12 from numpy.core.numeric import asarray, asanyarray, array, isnan, \
     13                 obj2sctype, zeros

/home1/03530/nleclear/.local/lib/python2.7/site-packages/numpy/core/__init__.py in <module>()
     12         os.environ[envkey] = '1'
     13         env_added.append(envkey)
---> 14 from . import multiarray
     15 for envkey in env_added:
     16     del os.environ[envkey]

ImportError: /home1/03530/nleclear/.local/lib/python2.7/site-packages/numpy/core/multiarray.so: undefined symbol: PyUnicodeUCS2_AsUTF8String
In [5]: import ipyrad
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-5-e67bb9b9c1dd> in <module>()
----> 1 import ipyrad

/home1/03530/nleclear/miniconda2/lib/python2.7/site-packages/ipyrad/__init__.py in <module>()
     20 
     21 ## main ip.functions
---> 22 from . import load
     23 from . import assemble
     24 from .load import save_json

/home1/03530/nleclear/miniconda2/lib/python2.7/site-packages/ipyrad/load/__init__.py in <module>()
     11 """
     12 
---> 13 from .load import test_assembly
     14 from .load import save_json
     15 from .load import load_json

/home1/03530/nleclear/miniconda2/lib/python2.7/site-packages/ipyrad/load/load.py in <module>()
      8 import time
      9 import json
---> 10 import pandas as pd
     11 import ipyrad as ip
     12 import numpy as np

/home1/03530/nleclear/miniconda2/lib/python2.7/site-packages/pandas/__init__.py in <module>()
     16 
     17 if missing_dependencies:
---> 18     raise ImportError("Missing required dependencies {0}".format(missing_dependencies))
     19 
     20 

ImportError: Missing required dependencies ['numpy']
Deren Eaton
@dereneaton
Sep 12 2016 22:17
@leclearnm This appears to be a somewhat strange and old problem with conda. It arises due to a conflict in your system deciding whether to use the conda installation versus your system installation of Python. In particular, there may have been a Python package that was installed on your system at some time using the --user flag, which creates a .local installation of Python. This makes a /home1/03530/nleclear/.local/lib/python2.7/site-packages directory in your path and that confuses conda. There is big issue discussion about it here (conda/conda#448).
Deren Eaton
@dereneaton
Sep 12 2016 22:36
@leclearnm Ok, I think I understand the solution. You can tell conda to ignore the .local directory by setting the path variable PYTHONNOUSERSITE=True. This will tell your Python to ignore the user site dirs. You can run conda info -s to see what is listed under 'user site dirs'. To set this variable so it is remembered every time you start Python you can put it in your .bashrc file. Use a text editor like nano to open the file ~/.bashrc and add the line export PYTHONNOUSERSITE=True. I think it should probably go after the line in the file that has the miniconda path, so at the very bottom of the file.
@leclearnm sorry about this, conda is supposed to make these things really easy, not harder!
leclearnm
@leclearnm
Sep 12 2016 23:30
That worked! Thank you for helping me to sort that out.