Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Louis Jenkins
    @LouisJenkinsCS
    Turns out the problem was the headnode
    glitch
    @glitch
    :thumbsup:
    Louis Jenkins
    @LouisJenkinsCS
    Thanks for the help!
    Louis Jenkins
    @LouisJenkinsCS
    This message was deleted
    tests/categorical_test.py .F......FFF... [ 5%]
    tests/check.py ................... [ 12%]
    tests/client_test.py ......... [ 15%]
    tests/coargsort_test.py ...... [ 18%]
    tests/compare_test.py ............... [ 23%]
    tests/datetime_test.py ............. [ 28%]
    tests/dtypes_tests.py ........... [ 32%]
    tests/groupby_test.py .............. [ 37%]
    tests/nan_test.py . [ 38%]
    tests/io_test.py F.F.FF.FFF..FF.FF.sF.. [ 46%]
    tests/io_util_test.py ... [ 47%]
    tests/join_test.py ....... [ 50%]
    tests/logger_test.py ....... [ 53%]
    tests/message_test.py .... [ 54%]
    tests/numeric_test.py .......F..... [ 59%]
    tests/operator_tests.py ............. [ 64%]
    tests/pdarray_creation_test.py .......F...F.F.F.F.. [ 71%]
    tests/regex_test.py ........ [ 74%]
    tests/registration_test.py ................ [ 80%]
    tests/security_test.py .... [ 82%]
    tests/setops_test.py ...... [ 84%]
    tests/sort_test.py ....
    So the "F" means failure? "s" means?
    glitch
    @glitch
    correct, s means skipped, currently there should only be 1 skipped test
    Louis Jenkins
    @LouisJenkinsCS
    Is each test different or are these trials?
    glitch
    @glitch
    they are all individual, you should have gotten a listing of the specific tests that failed
    Louis Jenkins
    @LouisJenkinsCS
    Gotcha, I'll wait until they finish then.
    glitch
    @glitch

    The make test target is really running pytest under the hood which you can invoke manually. i.e.

    python3 -m pytest tests/numeric_test.py

    You can run a specific one via

    python3 -m pytest tests/numeric_test.py::NumericTest::testHistogram

    Passing the -s option will also write all of the output to stdout

    You can see other optoins etc. in the pytest.ini configuration file
    Louis Jenkins
    @LouisJenkinsCS
    Gotcha. I ran make test-all but right now I'm waiting on the test suite to complete so I can see precisely which tests failed; its stalling right now
    I'm wondering, will it fail if it times out? Also, does it create/shutdown the server in each test? I.E. is it possible that it fails because it takes too long to start the server in each test?
    glitch
    @glitch
    From past experience if you're running in multi-locale mode and you think it stalled, do a ps -ef|grep arkouda_server to see if anything is running. Chances are the server process died out and the client in the test is waiting to reconnect ... which is never going to happen.
    Everything should pass and it looks like you have enough failures where I'd kill the make test-all process and then re-run just one of the test files manually with the -soption to see what's going on.
    Louis Jenkins
    @LouisJenkinsCS
    Weird its hanging but... ps -ef|grep arkouda_server yields a running server
    Categorial Test, which ran perfectly the first time around, hangs on the 4th test
    Oh nvm its still running even after the test ended
    OH NVM
    I was looking at the command itself; the server did indeed die
    How do I handle this issue? Is it killed due to default timeout?
    glitch
    @glitch
    No, the server likely seg-faulted during the test. I'd start with a single test class and single test and run it with the -s option so you can see the output
    Also you should probably re-compile Arkouda with the ARKOUDA_DEVELOPER flag turned on (i.e. export ARKOUDA_DEVELOPER=1)
    glitch
    @glitch
    This should display the name of the test running, which can help narrow down to a specific test that is hanging and then we can debug from there:
    python3 -m pytest tests/categorical_test.py::CategoricalTest -v
    Louis Jenkins
    @LouisJenkinsCS
    E       AssertionError: 'type[32 chars] of (Categorical, str, str_); got int instead' != 'type[32 chars] of (arkouda.categorical.Categorical, str, num[21 chars]tead'
    E       - type of argument "other" must be one of (Categorical, str, str_); got int instead
    E       + type of argument "other" must be one of (arkouda.categorical.Categorical, str, numpy.str_); got int instead
    Hmm
    ========================================================================================== FAILURES ==========================================================================================
    _________________________________________________________________________________ CategoricalTest.testBinop __________________________________________________________________________________
    
    self = <categorical_test.CategoricalTest testMethod=testBinop>
    
        def testBinop(self):
            cat = self._getCategorical()
            catDupe = self._getCategorical()
            catNonDupe = self._getRandomizedCategorical()
    
    
            self.assertTrue((cat._binop(catDupe,'==')).all())
            self.assertTrue((cat._binop(catNonDupe,'!=')).all())
    
            self.assertTrue((ak.array([True,True,True,True,True,True,True,
                                       True,True,True]) == cat._binop(catDupe,'==')).all())
    
            self.assertTrue((ak.array([False,False,False,False,False,False,
                                       False,False,False,False]) == cat._binop(catDupe,'!=')).all())
    
            self.assertTrue((ak.array([True,False,False,False,False,False,
                                       False,False,False,False]) ==
                                       cat._binop('string 1', '==')).all())
            self.assertTrue((ak.array([True,False,False,False,False,False,
                                       False,False,False,False]) ==
                                       cat._binop(np.str_('string 1'), '==')).all())
    
            self.assertTrue((ak.array([False,True,True,True,True,True,True,True,True,True]) ==
                       cat._binop('string 1', '!=')).all())
            self.assertTrue((ak.array([False,True,True,True,True,True,True,True,True,True]) ==
                       cat._binop(np.str_('string 1'), '!=')).all())
    
            with self.assertRaises(NotImplementedError):
                cat._binop('string 1', '===')
    
            with self.assertRaises(TypeError) as cm:
                cat._binop(1, '==')
    >       self.assertEqual(('type of argument "other" must be one of (Categorical, str, str_);' +
                              ' got int instead'),
                             cm.exception.args[0])
    E       AssertionError: 'type[32 chars] of (Categorical, str, str_); got int instead' != 'type[32 chars] of (arkouda.categorical.Categorical, str, num[21 chars]tead'
    E       - type of argument "other" must be one of (Categorical, str, str_); got int instead
    E       + type of argument "other" must be one of (arkouda.categorical.Categorical, str, numpy.str_); got int instead
    E       ?                                          ++++++++++++++++++++                  ++++++
    
    tests/categorical_test.py:129: AssertionError
    ================================================================================== short test summary info ===================================================================================
    7c180623e234acc691e04947c5a5faf3b0812faf
    Hash is above
    glitch
    @glitch
    What's your version of numpy?
    Louis Jenkins
    @LouisJenkinsCS
    Name: numpy
    Version: 1.20.1
    Summary: NumPy is the fundamental package for array computing with Python.
    Home-page: https://www.numpy.org
    Author: Travis E. Oliphant et al.
    Author-email: None
    License: BSD
    Location: /home/users/p02405/anaconda3/lib/python3.8/site-packages
    Requires: 
    Required-by: tifffile, tables, statsmodels, seaborn, scipy, scikit-learn, scikit-image, PyWavelets, pyerfa, patsy, pandas, numexpr, numba, mkl-random, mkl-fft, matplotlib, imageio, hdflow, h5py, Bottleneck, bokeh, bkcharts, astropy
    glitch
    @glitch
    Ah, ok, we're not compatible with 1.20.x yet (Bears-R-Us/arkouda#670), you'll need to downgrade to the 1.19.x line
    Louis Jenkins
    @LouisJenkinsCS
    D:
    I'd recommend using something like conda or python's venv to manage the development environment. It makes it much easier to control and test versioning of various dependencies.
    I know this isn't ideal, but on the bright side, I think we have a reasonable idea of why the tests are failing on your build.
    Louis Jenkins
    @LouisJenkinsCS
    I'll try to take another crack at it
    Louis Jenkins
    @LouisJenkinsCS
    See above for current error when trying to run the test
    Name: numpy
    Version: 1.19.3
    Summary: NumPy is the fundamental package for array computing with Python.
    Home-page: https://www.numpy.org
    Author: Travis E. Oliphant et al.
    Author-email: None
    License: BSD
    Location: /home/users/p02405/.local/lib/python3.8/site-packages
    Requires: 
    Required-by: tifffile, tables, statsmodels, seaborn, scipy, scikit-learn, scikit-image, PyWavelets, pyerfa, patsy, pandas, numexpr, numba, mkl-random, mkl-fft, matplotlib, imageio, hdflow, h5py, Bottleneck, bokeh, bkcharts, astropy
    __________________________ CategoricalTest.testBinop ___________________________
    
    self = <categorical_test.CategoricalTest testMethod=testBinop>
    
        def testBinop(self):
            cat = self._getCategorical()
            catDupe = self._getCategorical()
            catNonDupe = self._getRandomizedCategorical()
    
    
            self.assertTrue((cat._binop(catDupe,'==')).all())
            self.assertTrue((cat._binop(catNonDupe,'!=')).all())
    
            self.assertTrue((ak.array([True,True,True,True,True,True,True,
                                       True,True,True]) == cat._binop(catDupe,'==')).all())
    
            self.assertTrue((ak.array([False,False,False,False,False,False,
                                       False,False,False,False]) == cat._binop(catDupe,'!=')).all())
    
            self.assertTrue((ak.array([True,False,False,False,False,False,
                                       False,False,False,False]) ==
                                       cat._binop('string 1', '==')).all())
            self.assertTrue((ak.array([True,False,False,False,False,False,
                                       False,False,False,False]) ==
                                       cat._binop(np.str_('string 1'), '==')).all())
    
            self.assertTrue((ak.array([False,True,True,True,True,True,True,True,True,True]) ==
                       cat._binop('string 1', '!=')).all())
            self.assertTrue((ak.array([False,True,True,True,True,True,True,True,True,True]) ==
                       cat._binop(np.str_('string 1'), '!=')).all())
    
            with self.assertRaises(NotImplementedError):
                cat._binop('string 1', '===')
    
            with self.assertRaises(TypeError) as cm:
                cat._binop(1, '==')
    >       self.assertEqual(('type of argument "other" must be one of (Categorical, str, str_);' +
                              ' got int instead'),
                             cm.exception.args[0])
    E       AssertionError: 'type[32 chars] of (Categorical, str, str_); got int instead' != 'type[32 chars] of (arkouda.categorical.Categorical, str, num[21 chars]tead'
    E       - type of argument "other" must be one of (Categorical, str, str_); got int instead
    E       + type of argument "other" must be one of (arkouda.categorical.Categorical, str, numpy.str_); got int instead
    E       ?                                          ++++++++++++++++++++                  ++++++
    
    tests/categorical_test.py:129: AssertionError
    This is unmodified Arkouda
    I'm also using the chapel module on Swan (Cray-XC)
    glitch
    @glitch
    I filed an issue for this: Bears-R-Us/arkouda#955
    I think you are fine to comment out the failed assertion here until we fix it since the types are actually correct, just fully qualified versus the short name versions.
    Michael Merrill
    @mhmerrill
    We will NOT be having the Arkouda Weekly Call today because of a conflicting meeting.
    ATTENTION: We will NOT be having the Arkouda Weekly Call today because of a conflicting meeting.
    Louis Jenkins
    @LouisJenkinsCS
    Another error, this time in tests/categorical_test.py::CategoricalTest::testSaveAndLoadCategorical
    glitch
    @glitch
    Does h5ls exist on the machine you're running the test on? i.e. which h5ls ?