Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Paul m. p. P.
    @pmp-p
    the easy way is probably create a shared queuse beetween pyodide and host and pass your event with json
    Paul m. p. P.
    @pmp-p
    for canvases 2D/3D you can also use emscripten interfaces via a native library like Panda3D does https://rdb.name/panda3d-webgl/editor.html
    Junas Orionsky
    @NayaStudios99_twitter
    I was talking about that bit of code.
    from js import document
    x = document.getElementById("myElement")
    I cant seem to find a relevant example for this, while using events
    Paul m. p. P.
    @pmp-p
    you would probably have to define a function that can build an event handler which fills a queue
    you could probably find inspiration in some existing java<->python proxy
    demonstrating this would be like proposing a stdlib for browser which is way too much effort ( for now )
    Andreas Klostermann
    @akloster
    I got bokeh working with pyodide, should I make a pull request with the packages?
    Andreas Klostermann
    @akloster
    It is somewhat tricky to actually use it in a useful way, though. I'm still figuring that part out. Displaying figures works fine, updating figures I have yet to figure out.
    Paul m. p. P.
    @pmp-p
    looking at pyodide/pyodide#1590 : the libyaml problem sounds a lot like the libglpk i've got recently . It feels like the build scripts should integrate two extra folders , one sysroot for missing native libraries and one for the target (wasm, possibly translated from the other when building the replay).
    not all packages have a broken setup.py i guess a lot of them can support cross compilation fine
    Alex
    @aadereiko

    Hey, guys
    I've faced some issue, related to pyodide and plotly.express.

    The problem is when I run a code snippet and console is opened, it works perfectly.

    import pandas
    import plotly.express as px
    
    df = px.data.iris()
    fig = px.scatter(
    df, x="sepal_width", y="sepal_length",
    color="species")
    
    fig

    On other hand, when I run this code and console is closed, I get the error:

    alt

    FYI: I open a console after an error appeared
    Hood Chatham
    @hoodmane
    Thanks for the report @aadereiko. It looks like this is pyodide/pyodide#1507 -- in particular the innermost frame matches:
    at __static_1 (<anonymous>:wasm-function[174]:0x17748)
    Alex
    @aadereiko

    @hoodmane
    it hasn't helped actually :(
    I was trying to add at the beginning

      await loadPyodide({
        indexURL: "https://cdn.jsdelivr.net/pyodide/dev/full/",
      });
      await pyodide.loadPackage(["numpy", "pandas", "plotly"]);
         ...
     <script src="https://cdn.jsdelivr.net/pyodide/dev/full/pyodide.js"></script>

    and after that was trying to change the version of loaded pyodide to dev one and both ways don't work neither. my observation tells me that the problem is on px.scatter...

    Itay Dafna
    @ibdafna

    Hey all,

    I've been using Pyodide quite a lot at work and recently open sourced a research project from earlier this year, which relies on Pyodide. In case you find it interesting:

    Blog post: https://polyglot.codes/posts/webdash
    Github: https://github.com/ibdafna/webdash

    :smile_cat:

    Nathaniel Kofalt
    @kofalt

    I'm digging through the documentation, and having trouble figuring out the full process for building my own pure python package from pip.

    So far, I'm in the pyodide-env container and I've ran python -m pyodide_build mkpkg pydicom from within the pyodide-build folder.

    This has created pyodide-build/packages/pydicom/meta.yaml and it looks fine, but I'm not sure what the next step is? The rest of the page describes the YAML structure, what to do if the package has C extensions, and then what looks to be low-level details that I'm given to understand mkpkg automates.

    Roman Yurchak
    @rthy:matrix.org
    [m]
    That's a great project @ibdafna ! Having used Dash in the past, not having to run a flask server would certainly make it much easier to deploy. Would you like to add it to https://pyodide.org/en/latest/project/related-projects.html? maybe under a new section "Visualization" or something similar.
    @kofalt: If it's a pure Python package, you can create a regular Python wheel then load it via micropip.install(<url of the wheel>).

    and it looks fine, but I'm not sure what the next step is?

    If you go the route of packaging it as part of the Pyodide build system (which is unnecessary here), the next step would be to run,

    PYODIDE_PACKAGES="<your package>" make

    or just make but that would rebuild everything

    Itay Dafna
    @ibdafna

    That's a great project @ibdafna ! Having used Dash in the past, not having to run a flask server would certainly make it much easier to deploy. Would you like to add it to https://pyodide.org/en/latest/project/related-projects.html? maybe under a new section "Visualization" or something similar.

    Hey @rthy:matrix.org! Thanks for checking it out! I'll add the it to the page :)

    Nathaniel Kofalt
    @kofalt

    @rthy:matrix.org Thanks - hahaha, silly of me to not realize that.

    When I ran PYODIDE_PACKAGES="pydicom" make that appeared to succeed, though I was surprised to get an .egg file instead of a wheel. I converted this with wheel convert ./pydicom-2.1.2-py3.8.egg, but let me know if there was some other method I should have used.

    It looks like my converted wheel can be loaded by await micropip.install just fine, but actually importing the package causes a ModuleNotFoundError: No module named 'pkg_resources' error to be thrown. I can also reproduce this by skipping all the local work and just trying await micropip.install('pydicom').

    I did not see any similar reports, so I've filed a ticket with a full stack trace:
    pyodide/pyodide#1611

    Caleb Kaiser
    @caleb-kaiser

    Hey all!

    First, thank you so much for your work on Pyodide. The project really is incredible. I'm hitting a weird error I can't debug involving classes, and I was hoping someone here might have had similar experiences/could point me in the right direction.

    I'm trying to extend a Javascript class accessible from the global scope. In JS, I'm able to do something like:

    class SpecificChart extends Base.Chart {

    I know the class itself is accessible to pyodide via the js module, as I've tested it. However, when I try to extend the class in Python like so:

    class SpecificChart(js.Base.Chart):

    I get this error: `TypeError: cannot create 'pyodide.JsProxy' instances'

    Looking through the docs, I can't find many references to the class keyword specifically, so I'm curious as to whether the conversion there isn't fully supported. Or, more likely, if I'm missing something extremely obvious. Appreciate any help!

    Hood Chatham
    @hoodmane
    @caleb-kaiser In Javascript it's impossible to determine whether an object is a class or not and there are huge differences between the Javascript object model and the Python object model. So I don't think we could support this. Not sure what the best alternative is for your use case.
    Caleb Kaiser
    @caleb-kaiser
    @hoodmane Completely understand—I think I found a workaround by being a bit hack-y with the prototype chain. Appreciate you taking the time to help out!
    James Tsang
    @wtzeng1
    Hi, could any one tell me the pronunciation of "pyodide"? I search it on Google but found no valuable result.
    Roman Yurchak
    @rthy:matrix.org
    [m]
    If you enter "Pyodide" into https://ttsmp3.com/ for US English it sounds right I think. You can also listen to https://talkpython.fm/episodes/show/212/python-in-web-assembly-with-pyodide podcast where the pronunciation of Pyodide is included.
    Hood Chatham
    @hoodmane
    the IPA for Pyodide is ˈpɑɪ.ə.ˌdɑɪd if that helps
    Correct-Syntax
    @Correct-Syntax

    Hi, this is a neat project. I had some general questions about Python + webassembly (but they do relate to pyodide):

    • What is the current state of python webassembly? Are there hurdles that must be overcome/can't be overcome? (I find many articles with slightly differing information, so I don't have a clear picture).
    • Is it (pyodide) production ready yet? If so, would it be recommended to use (e.g: pyodide) to create a desktop application -or is it likely to be slow?
    • I have heard that pyodide is resource hungry (is that true?) If so, why does python webassembly have that problem, but it doesn't seem like a problem for compiled languages? Is it the interpreter overhead? The fact that webassemblycan't be faster than the original language?
    • What is required to compile a C module for use in webassembly, pyodide, etc

    (I am just trying to get a clear picture of how something like pyodide could work for a project I'd like to try.)
    Thanks a lot for your time and work on this project.

    Hood Chatham
    @hoodmane

    A lot of good questions. They are somewhat open ended. Here are a few of my opinions:

    1. The biggest existing limitations are lack of support for synchronous i/o and threading. ctypes support is also an important missing feature. I don't think any of these things are insurmountable. Another big issue is that some WASM runtimes have a call stack that is too small for programs that use much recursion to run well (e.g., in some cases we can only support a recursion depth of ~125 without seeing a segfault).

    2. It's a bit adventurous to use Pyodide in production but people are using it, particularly in CS education. I think the lack of synchronous I/O is a major limitation for educational purposes. A few people are using pyodide for electron applications and stuff like that but we haven't worked in support for it that much so you are likely to need to change some stuff in Pyodide to get it to work. Any contributions to make this easier are appreciated.

    3. Python itself is large and slow and we are running it inside of a separate browser VM. Web assembly actually achieves quite good performance in many cases but you pay a significant price for the process isolation / sandbox. Compiled languages are much more efficient than Python in native environments and they are much more efficient in WASM too. Another aspect of Pyodide that you should remember is that it effectively installs a new copy of Python on your computer every time you visit the webpage.

    4. It depends a lot on what the C module does, in some cases it's trivial. I haven't tried to do this much myself but I think the biggest issue is that some system calls that the C module might use are missing and so you may need to patch them. I think it would be helpful to look at the patches we apply to the various existing C modules. Some C modules work almost without any changes, others require complex adjustments.

    Roman Yurchak
    @rthy:matrix.org
    [m]

    +1 for the above answer.

    Maybe to add information on point 3. Say for scientific computing Python works well because most of the performance critical parts will be written in C / Cython / Numba or some other complied language anyway. Performance wise we are between comparable and 3-5x slower depending on the benchmark, and there is some indications that it might be improved further. Then there is indeed the cost of starting the WASM VM, but I wouldn't really say that it's "resources hungry"

    For 1 see also https://pyodide.org/en/stable/project/roadmap.html

    Correct-Syntax
    @Correct-Syntax

    @hoodmane and @rthy:matrix.org thanks so much for the detailed answers! It helped me to understand things a bit better.

    Yes, I asked open-ended questions because I am trying to get a "big picture" of where python web assembly stands currently. :)

    James Tsang
    @wtzeng1
    @rthy:matrix.org @hoodmane Thanks a lot, you solve my problem about the pronunciation which troubled me for several days.
    Alex
    @aadereiko

    Hey, hope you're doing well guys :)

    I have some small questions about pyodide:

    1) the first one is about code running process.
    The problem is while python code is running (i run the code with runPythonAsync), the main thread is blocked and user can not interact with UI.
    The solution for this case is to write webworkers, but there is an issue with them: I can not make matplotlit library work in worker, because it requires document and window objects that a worker doesn't have access to.
    so, is there a way to run libraries that require js document and window objects in worker?

    2) the second question is about packages importing.
    There is micropip to load "external" py pure packages. Is it possible to import seaborn package? It's a pretty common statistics library

    Hood Chatham
    @hoodmane
    @aadereiko seaborn can be imported but there is a bug that must be worked around: pyodide/pyodide#1619
    Hood Chatham
    @hoodmane

    I have been investigating how to access the DOM from a webworker using something like Comlink.

    I think in the case of Matplotlib, I think all of the updates are asynchronous (well more specifically they are done by callbacks that don't ever check for the result) so it shouldn't be too hard to do this. I have a demo of something like this in pyodide/pyodide#1535 but for some reason it seems not to be working anymore...

    Hood Chatham
    @hoodmane
    we are very interested in improving UI from webworkers, I have a strategy that works but it is a bit complex and there might be a better alternative involving emscripten pthreads.
    Anyways in the current version of Pyodide you would have to patch the matplotlib backend.
    Alex
    @aadereiko

    @hoodmane

    thanks a lot! i appreciate your answers.
    i'm trying to implement ComLink library to work with document and window in workers, it means to run a matplotlib in workers.
    It helped to solve the issue with from js import document, window. Thanks!

    But currently I'm facing an issue AttributeError:'JsException' object has no attribute 'createElement'.

    my current implementation looks like:

    1. I use a wrap method of ComLink for worker.
    2. From worker I expose a function, which takes windowProxy and documentProxy as arguments and initialize self by them.

    It looks like:

    // worker.js
    const runCodeSnippet = (codeSnippet, windowProxy, documentProxy) => {
      self.window = windowProxy;
      self.document = documentProxy;
    
      self.pyodide
        .runPythonAsync(codeSnippet)
        .then(...)
        .catch(...);
    };
    // main.js
    runCodeSnippet(
          event.data.pyCode,
          ComLink.proxy(window),
          ComLink.proxy(document)
     );

    it doesn't help actually to run the example completely :(
    It breaks after the line with subplots.

    import matplotlib.pyplot as plt
    import numpy as np
    t = np.arange(0.0, 2.0, 0.01)
    s = 1 + np.sin(2 * np.pi * t)
    fig, ax = plt.subplots()

    Don't you have any ideas how to solve this?

    Alex
    @aadereiko
    @hoodmane
    don't you have any updates?
    Hood Chatham
    @hoodmane

    oh I do, sorry I meant to respond but I forgot about this. Could you send an example that demonstrates the behavior that's causing trouble?

    So for one thing, you can't currently directly import a Comlink proxy into pyodide because the proxy really badly screws up the reflection that happens here:
    https://github.com/pyodide/pyodide/blob/d1a2778088231dd859443eff53a2adfeef893c13/src/core/jsproxy.c#L1637
    you need to first detect whether something is a Comlink proxy and if so goto the end of the feature detection.

    It breaks after the line with subplots

    Could you tell me more about how it breaks?

    does it throw an exception?
    oh yeah okay
    I think the problem is exactly what I said
    Hood Chatham
    @hoodmane
    But I think even with a patch to allow import of comlink proxies
    hmmm
    actually that should fix the trouble I guess
    I think this branch might make your code work:
    pyodide/pyodide#1642