by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jul 28 05:09
    li-dennis commented #384
  • Jul 24 06:02
    avikchaudhuri commented #386
  • Jul 22 18:36
    codecov[bot] commented #392
  • Jul 22 18:36
    codecov[bot] commented #392
  • Jul 22 18:35
    codecov[bot] commented #392
  • Jul 22 18:35
    codecov[bot] commented #392
  • Jul 22 18:35
    codecov[bot] commented #392
  • Jul 22 18:34
    codecov[bot] commented #392
  • Jul 22 18:34
    codecov[bot] commented #392
  • Jul 22 18:34
    codecov[bot] commented #392
  • Jul 22 18:33
    twoertwein opened #392
  • Jul 22 02:54
    twoertwein commented #366
  • Jul 20 17:17
    twoertwein commented #366
  • Jul 20 15:17
    ogrisel commented #385
  • Jul 20 15:13
    ogrisel commented #385
  • Jul 20 02:49
    twoertwein closed #372
  • Jul 15 09:08
    ogrisel commented #384
  • Jul 15 08:51
    ogrisel commented #391
  • Jul 08 21:57
    kinghuang edited #391
  • Jul 08 21:53
    kinghuang edited #391
Olivier Grisel
@ogrisel
I will put that in the CHANGES.md file
Kyle Kelley
@rgbkrk
They changed my flight so I didn't miss my connection, due to delays.
Olivier Grisel
@ogrisel
I pushed a new CHANGES.md to master
Kyle Kelley
@rgbkrk
Yeah I wondered if I should put a CHANGELOG together as I was doing the release
Kyle Kelley
@rgbkrk
I'm currently tinkering with using IPython kernels as the fresh "remote" namespace
km, client = IPython.kernel.manager.start_new_kernel()
Olivier Grisel
@ogrisel
that might be a good idea to write such (optional) integration tests!
Kyle Kelley
@rgbkrk
The alternative is to run these (optional) integration tests with IPython parallel.
Olivier Grisel
@ogrisel
I like it better to use km, client = IPython.kernel.manager.start_new_kernel()
Kyle Kelley
@rgbkrk
I have a simple thing that works, not sure what to test on the other end
Could repickle the result
Olivier Grisel
@ogrisel
yes
Kyle Kelley
@rgbkrk
and then test that locally
ok
Olivier Grisel
@ogrisel
"pickle echo"
Kyle Kelley
@rgbkrk
lol
"How's the brine?"
Steven De Gryze
@sdegryze
still futzing around with travis not being triggered on my ufunc branch cloudpipe/cloudpickle#20
yml looks okay and parses well
just reverted to a previous state of the .yml and even then is the travis build not triggered
any other ideas?
This message was deleted
Kyle Kelley
@rgbkrk
Before I got on flight I meant to tell you to try opening the branch on the main github repo
in case its just not picking up the travis file
Travis uses the merge commit, but it always parses the travis config first
Ok, fairly silly.
There's got to be better machinery in IPython for this
try:
    from queue import Empty  # Python 3
except ImportError:
    from Queue import Empty  # Python 2

import cloudpickle
import IPython

def remote_pickle(obj):
    km, client = IPython.kernel.manager.start_new_kernel()
    client.execute("import pickle")
    client.execute("import cloudpickle")
    dump_id = client.execute("cloudpickle.dumps(pickle.loads({}))".format(cloudpickle.dumps(obj)))

    winner = None

    try:
        while(True):
            message = client.get_iopub_msg(timeout=0.1)
            if(message['msg_type'] == 'execute_result' and message['parent_header']['msg_id'] == dump_id):
                winner = message['content']['data']['text/plain']

    except Empty:
        pass
    km.shutdown_kernel(now=True)
    return eval(winner)
Olivier Grisel
@ogrisel
@sdegryze maybe try to open a new PR
@rgbkrk looks good
not pep8 80 columns though ;)
Kyle Kelley
@rgbkrk
Punishments I get for coding in the notebook
;)
Olivier Grisel
@ogrisel
@rgbkrk have a look at #31
it's good news: cloudpickle works well!
well the PySpark tests pass and they should do pretty much the same (indirectly though)
Kyle Kelley
@rgbkrk
Neat
Might as well adapt that to a series of test cases that rely on a configurable PickleEcho
so it runs the same set in different environments
Olivier Grisel
@ogrisel
yes I agree, I need to understand what's wrong in travis, I cannot reproduce locally
Kyle Kelley
@rgbkrk
I made a branch that relies on yours to do testing with IPython kernels
I'm in the road to Texas so I'll post later
Olivier Grisel
@ogrisel
ok I still have issues on Python 3.4 on travis
I cannot reproduce
I will stop rebasing / amending my branch then :)
Kyle Kelley
@rgbkrk
Oh it's ok. I just wanted your test setup so I didn't have to rewrite it.
Olivier Grisel
@ogrisel
the test failure I had in #31 was caused by a bug in subprocess in Python 3.4.2 (which is the version used on travis) but fixed in Python 3.4.3. I skip that test on that version
Olivier Grisel
@ogrisel
now thinking how to refactor that code to introduce PickleEcho base classes and subclasses.
Lingepumpe
@Lingepumpe
not sure if this room is alive, but I would appreciate input and/or help with my issue to cloudpickle classes decorated with @attrs - I created testcases here: cloudpipe/cloudpickle#320 - especially I am wondering if this should be fixed in attr or in cloudpickle, and how :)