Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Louis Jenkins
    @LouisJenkinsCS
    E       AssertionError: 'type[32 chars] of (Categorical, str, str_); got int instead' != 'type[32 chars] of (arkouda.categorical.Categorical, str, num[21 chars]tead'
    E       - type of argument "other" must be one of (Categorical, str, str_); got int instead
    E       + type of argument "other" must be one of (arkouda.categorical.Categorical, str, numpy.str_); got int instead
    Hmm
    ========================================================================================== FAILURES ==========================================================================================
    _________________________________________________________________________________ CategoricalTest.testBinop __________________________________________________________________________________
    
    self = <categorical_test.CategoricalTest testMethod=testBinop>
    
        def testBinop(self):
            cat = self._getCategorical()
            catDupe = self._getCategorical()
            catNonDupe = self._getRandomizedCategorical()
    
    
            self.assertTrue((cat._binop(catDupe,'==')).all())
            self.assertTrue((cat._binop(catNonDupe,'!=')).all())
    
            self.assertTrue((ak.array([True,True,True,True,True,True,True,
                                       True,True,True]) == cat._binop(catDupe,'==')).all())
    
            self.assertTrue((ak.array([False,False,False,False,False,False,
                                       False,False,False,False]) == cat._binop(catDupe,'!=')).all())
    
            self.assertTrue((ak.array([True,False,False,False,False,False,
                                       False,False,False,False]) ==
                                       cat._binop('string 1', '==')).all())
            self.assertTrue((ak.array([True,False,False,False,False,False,
                                       False,False,False,False]) ==
                                       cat._binop(np.str_('string 1'), '==')).all())
    
            self.assertTrue((ak.array([False,True,True,True,True,True,True,True,True,True]) ==
                       cat._binop('string 1', '!=')).all())
            self.assertTrue((ak.array([False,True,True,True,True,True,True,True,True,True]) ==
                       cat._binop(np.str_('string 1'), '!=')).all())
    
            with self.assertRaises(NotImplementedError):
                cat._binop('string 1', '===')
    
            with self.assertRaises(TypeError) as cm:
                cat._binop(1, '==')
    >       self.assertEqual(('type of argument "other" must be one of (Categorical, str, str_);' +
                              ' got int instead'),
                             cm.exception.args[0])
    E       AssertionError: 'type[32 chars] of (Categorical, str, str_); got int instead' != 'type[32 chars] of (arkouda.categorical.Categorical, str, num[21 chars]tead'
    E       - type of argument "other" must be one of (Categorical, str, str_); got int instead
    E       + type of argument "other" must be one of (arkouda.categorical.Categorical, str, numpy.str_); got int instead
    E       ?                                          ++++++++++++++++++++                  ++++++
    
    tests/categorical_test.py:129: AssertionError
    ================================================================================== short test summary info ===================================================================================
    7c180623e234acc691e04947c5a5faf3b0812faf
    Hash is above
    glitch
    @glitch
    What's your version of numpy?
    Louis Jenkins
    @LouisJenkinsCS
    Name: numpy
    Version: 1.20.1
    Summary: NumPy is the fundamental package for array computing with Python.
    Home-page: https://www.numpy.org
    Author: Travis E. Oliphant et al.
    Author-email: None
    License: BSD
    Location: /home/users/p02405/anaconda3/lib/python3.8/site-packages
    Requires: 
    Required-by: tifffile, tables, statsmodels, seaborn, scipy, scikit-learn, scikit-image, PyWavelets, pyerfa, patsy, pandas, numexpr, numba, mkl-random, mkl-fft, matplotlib, imageio, hdflow, h5py, Bottleneck, bokeh, bkcharts, astropy
    glitch
    @glitch
    Ah, ok, we're not compatible with 1.20.x yet (Bears-R-Us/arkouda#670), you'll need to downgrade to the 1.19.x line
    Louis Jenkins
    @LouisJenkinsCS
    D:
    I'd recommend using something like conda or python's venv to manage the development environment. It makes it much easier to control and test versioning of various dependencies.
    I know this isn't ideal, but on the bright side, I think we have a reasonable idea of why the tests are failing on your build.
    Louis Jenkins
    @LouisJenkinsCS
    I'll try to take another crack at it
    Louis Jenkins
    @LouisJenkinsCS
    See above for current error when trying to run the test
    Name: numpy
    Version: 1.19.3
    Summary: NumPy is the fundamental package for array computing with Python.
    Home-page: https://www.numpy.org
    Author: Travis E. Oliphant et al.
    Author-email: None
    License: BSD
    Location: /home/users/p02405/.local/lib/python3.8/site-packages
    Requires: 
    Required-by: tifffile, tables, statsmodels, seaborn, scipy, scikit-learn, scikit-image, PyWavelets, pyerfa, patsy, pandas, numexpr, numba, mkl-random, mkl-fft, matplotlib, imageio, hdflow, h5py, Bottleneck, bokeh, bkcharts, astropy
    __________________________ CategoricalTest.testBinop ___________________________
    
    self = <categorical_test.CategoricalTest testMethod=testBinop>
    
        def testBinop(self):
            cat = self._getCategorical()
            catDupe = self._getCategorical()
            catNonDupe = self._getRandomizedCategorical()
    
    
            self.assertTrue((cat._binop(catDupe,'==')).all())
            self.assertTrue((cat._binop(catNonDupe,'!=')).all())
    
            self.assertTrue((ak.array([True,True,True,True,True,True,True,
                                       True,True,True]) == cat._binop(catDupe,'==')).all())
    
            self.assertTrue((ak.array([False,False,False,False,False,False,
                                       False,False,False,False]) == cat._binop(catDupe,'!=')).all())
    
            self.assertTrue((ak.array([True,False,False,False,False,False,
                                       False,False,False,False]) ==
                                       cat._binop('string 1', '==')).all())
            self.assertTrue((ak.array([True,False,False,False,False,False,
                                       False,False,False,False]) ==
                                       cat._binop(np.str_('string 1'), '==')).all())
    
            self.assertTrue((ak.array([False,True,True,True,True,True,True,True,True,True]) ==
                       cat._binop('string 1', '!=')).all())
            self.assertTrue((ak.array([False,True,True,True,True,True,True,True,True,True]) ==
                       cat._binop(np.str_('string 1'), '!=')).all())
    
            with self.assertRaises(NotImplementedError):
                cat._binop('string 1', '===')
    
            with self.assertRaises(TypeError) as cm:
                cat._binop(1, '==')
    >       self.assertEqual(('type of argument "other" must be one of (Categorical, str, str_);' +
                              ' got int instead'),
                             cm.exception.args[0])
    E       AssertionError: 'type[32 chars] of (Categorical, str, str_); got int instead' != 'type[32 chars] of (arkouda.categorical.Categorical, str, num[21 chars]tead'
    E       - type of argument "other" must be one of (Categorical, str, str_); got int instead
    E       + type of argument "other" must be one of (arkouda.categorical.Categorical, str, numpy.str_); got int instead
    E       ?                                          ++++++++++++++++++++                  ++++++
    
    tests/categorical_test.py:129: AssertionError
    This is unmodified Arkouda
    I'm also using the chapel module on Swan (Cray-XC)
    glitch
    @glitch
    I filed an issue for this: Bears-R-Us/arkouda#955
    I think you are fine to comment out the failed assertion here until we fix it since the types are actually correct, just fully qualified versus the short name versions.
    Michael Merrill
    @mhmerrill
    We will NOT be having the Arkouda Weekly Call today because of a conflicting meeting.
    ATTENTION: We will NOT be having the Arkouda Weekly Call today because of a conflicting meeting.
    Louis Jenkins
    @LouisJenkinsCS
    Another error, this time in tests/categorical_test.py::CategoricalTest::testSaveAndLoadCategorical
    glitch
    @glitch
    Does h5ls exist on the machine you're running the test on? i.e. which h5ls ?
    Louis Jenkins
    @LouisJenkinsCS
    :o I don't have that
    Oh wait
    I didn't have module loaded
    Shall I try it again?
    For clarity: I am running on both a Cray-XC and an older infiniband machine
    glitch
    @glitch
    I would expect it to work as long as h5ls is installed and accessible
    Also because it's relevant I was considering taking on this ticket Bears-R-Us/arkouda#452 which would remove the need for h5ls installation
    Louis Jenkins
    @LouisJenkinsCS
    Cool, it worked!
    Thanks
    Louis Jenkins
    @LouisJenkinsCS
    It is unfortunate that the client does not have a way of backing out when the server is unavailable in between tests. It seems to happen at random times, but client is just hanging.
    glitch
    @glitch
    If you are running the unit tests on a multi-locale system the hangling client is generally indicative of the server dying for some reason. How many locales are you running on? (There are a few known issues with running unit tests with small amounts of data on systems with num_locales > num_data_elements and I'm wondering if that's what you're running into.
    I'd also advocate for running a specific test file / class with the -v flag on first to get an idea of what test is failing, then running it with the -s flag to capture the output.
    python3 -m pytest tests/categorical_test.py::CategoricalTest -v
    Louis Jenkins
    @LouisJenkinsCS
    The default number of locales which is 2
    I'll try the -s flag
    Louis Jenkins
    @LouisJenkinsCS
    Unfortunately I can't seem to drag-and-drop here... so here is a pastebin: https://pastebin.com/xvPa8ZHB
    It works fine down to testEquality
    Louis Jenkins
    @LouisJenkinsCS
    This is weird, I am having so many problems running on the supercomputers, but it works fine on my laptop. Now I can't even do a pip install to build the arkouda package...
    Building wheels for collected packages: arkouda
      Building wheel for arkouda (setup.py) ... error
      ERROR: Command errored out with exit status 1:
       command: /home/users/p02405/anaconda3/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/lus/scratch/tmp/pip-req-build-mra5diqu/setup.py'"'"'; __file__='"'"'/lus/scratch/tmp/pip-req-build-mra5diqu/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /lus/scratch/tmp/pip-wheel-2cq41ur_
           cwd: /lus/scratch/tmp/pip-req-build-mra5diqu/
      Complete output (30 lines):
      /home/users/p02405/anaconda3/lib/python3.8/site-packages/setuptools/dist.py:461: UserWarning: Normalizing 'v2021.08.20+35.g6f5616b.dirty' to '2021.8.20+35.g6f5616b.dirty'
        warnings.warn(tmpl.format(**locals()))
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib
      creating build/lib/arkouda
      copying arkouda/groupbyclass.py -> build/lib/arkouda
      copying arkouda/io_util.py -> build/lib/arkouda
      copying arkouda/pdarrayclass.py -> build/lib/arkouda
      copying arkouda/timeclass.py -> build/lib/arkouda
      copying arkouda/message.py -> build/lib/arkouda
      copying arkouda/sorting.py -> build/lib/arkouda
      copying arkouda/logger.py -> build/lib/arkouda
      copying arkouda/numeric.py -> build/lib/arkouda
      copying arkouda/infoclass.py -> build/lib/arkouda
      copying arkouda/strings.py -> build/lib/arkouda
      copying arkouda/client.py -> build/lib/arkouda
      copying arkouda/security.py -> build/lib/arkouda
      copying arkouda/pdarrayIO.py -> build/lib/arkouda
      copying arkouda/pdarraysetops.py -> build/lib/arkouda
      copying arkouda/_version.py -> build/lib/arkouda
      copying arkouda/pdarraycreation.py -> build/lib/arkouda
      copying arkouda/__init__.py -> build/lib/arkouda
      copying arkouda/dtypes.py -> build/lib/arkouda
      copying arkouda/categorical.py -> build/lib/arkouda
      copying arkouda/join.py -> build/lib/arkouda
      UPDATING build/lib/dist/arkouda/_version.py
      error: [Errno 2] No such file or directory: 'build/lib/dist/arkouda/_version.py'
      ----------------------------------------
      ERROR: Failed building wheel for arkouda
    This one is on a Cray-XC
    glitch
    @glitch

    Are you pip installing from pypi or are you building your own client package to install (https://github.com/Bears-R-Us/arkouda#install-ak).

    You shouldn't install from pypi as we have not updated that in quite some time and it will not work with the current server.

    Louis Jenkins
    @LouisJenkinsCS
    :o I see, I got it now, thanks
    Brett Eiffert
    @brettceiffert

    hi all - new to this project. i have just been playing with arkouda for the past month or so.

    i am having an issue with using a system where 256 threads are available (2 x 64 core processors with hyperthreading) and am only getting 128ish arkouda threads. how is the number of arkouda threads spun up decided for a given system? it seems there is a default value that applies to smaller machines but not this machine i am trying. Is there a max number of threads? Is there a flag to specify how many threads to spin up when starting the server? Thanks!

    Elliot Ronaghan
    @ronawho
    Arkouda is backed by Chapel, and by default Chapel uses physical cores only and not hyperthreads (which generally do not help performance of HPC Applications). You can control this value manually with CHPL_RT_NUM_THREADS_PER_LOCALE, but generally speaking using logical threads will not help performance and may in fact hurt. See https://chapel-lang.org/docs/usingchapel/tasks.html#controlling-the-number-of-threads for more info.
    Brett Eiffert
    @brettceiffert
    Thanks Elliot! that’s helpful
    Oliver Alvarado Rodriguez
    @alvaradoo
    @mhmerrill Hey Mike! Hope all is well. Where do we find the link with the weekly Arkouda meeting presentations on GitHub? I want to reference a student to one of the first ones you gave, but I am having trouble finding it on GitHub, thanks!
    Nevermind!