Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 04:16

    hoodmane on main

    MAINT Add support for testing e… (compare)

  • May 22 04:11

    hoodmane on main

    MAINT Improvements to micropip … (compare)

  • May 21 20:35

    hoodmane on main

    chore: complete more incomplete… (compare)

  • May 20 06:27

    rth on main

    refactor: use .path & type buil… (compare)

  • May 19 18:02

    hoodmane on main

    Export PATH and ERRNO_CODES fro… (compare)

  • May 19 06:09

    ryanking13 on main

    MAINT Remove outdated commands … (compare)

  • May 19 05:19

    hoodmane on main

    FIX Fix feature detection when … (compare)

  • May 19 02:04

    hoodmane on main

    Remove uneeded numpy install in… (compare)

  • May 19 01:27

    hoodmane on main

    Late addition of ryanking13's e… (compare)

  • May 19 01:22

    hoodmane on main

    Improve docs on running package… (compare)

  • May 18 21:37

    hoodmane on main

    ENH Use `dist-info` + `importli… (compare)

  • May 18 05:04

    hoodmane on main

    Add pyodide_build create_xbuild… (compare)

  • May 18 04:57

    hoodmane on main

    PKG Bump scikit-learn to 1.1.0 … (compare)

  • May 18 04:14

    ryanking13 on main

    BLD Install libraries to centra… (compare)

  • May 17 19:34

    hoodmane on main

    micropip Fix async race conditi… (compare)

  • May 17 19:31

    hoodmane on main

    PKG Add msprime and deps includ… (compare)

  • May 17 18:07

    hoodmane on main

    Add FAQ when micropip fails to … (compare)

  • May 17 02:58

    hoodmane on main

    MAINT micropip Move wheel-relat… (compare)

  • May 17 02:12

    hoodmane on main

    Fix ordering in micropip add_re… (compare)

  • May 17 01:32

    hoodmane on main

    MAINT Move testing compatibilit… (compare)

Antonis Christofides
@aptiko
Hey people, I'm trying to run pyodide (I'm a beginner) on a Vue application (also a beginner). I can't figure out how (and whether) I can use pyodide with node/webpack. I tried, for example, import { languagePluginLoader } from 'pyodide' and various other things, but nothing I've tried works.
dalcde
@dexter:hailstorm.adm.srcf.net
[m]
I don't think it works at the moment, but some people are looking into it
Antonis Christofides
@aptiko
Is iodide-project/pyodide#14 what I'm looking for?
Antonis Christofides
@aptiko
So, I've had some success with this:
const languagePluginLoader = require('pyodide');
languagePluginLoader.then(() => { console.log(self.pyodide.runPython('import sys; sys.version')); });
Antonis Christofides
@aptiko

So, when Python, numpy and pandas are all loaded, executing pyodide.runPython('import numpy') takes about 5 seconds on my system; import pandas takes about 10 seconds. (These are with Firefox; Chrome times are different but similar.)

I noticed that even in CPython the times needed to import these are non-negligible; import pandas takes 0.5 to 1 second, depending on the status of disk cache. import numpy is faster but clearly not instant (I didn't measure it).

But why the long times in pyodide?

3 replies
dalcde
@dexter:hailstorm.adm.srcf.net
[m]
One possibility is that we lz4 compress all files, and they are decompressed on the fly when you use them
Antonis Christofides
@aptiko
Antonis Christofides
@aptiko
Today I'm downloading https://pyodide-cdn2.iodide.io/v0.15.0/full/pyodide.asm.wasm at something like 10 kB/s. Is it just me?
rthy
@rthy:matrix.org
[m]
@aptiko: Please use the JsDelivr endpoint (cf docs)
Antonis Christofides
@aptiko

Can I ask a question about Firefox vs. Chrome? I wrote (as a rough benchmark) this program that calculates pi using the formula pi/4 = 1 - 1/3 + 1/5 - 1/7 ... by looping 10 million times, and tells you how much it took:

#include <sys/time.h>
#include <stdio.h>
#include <stdlib.h>

int main (void) {
    double pi = 4;
    double denom = 3;
    double sign = -1;
    struct timeval tv1, tv2;
    gettimeofday(&tv1, NULL);
    while (denom < 1e8) {
        pi = pi + sign * 4.0 / denom;
        denom += 2.0;
        sign = sign * -1.0;
    }
    gettimeofday(&tv2, NULL);
    time_t secs = tv2.tv_sec - tv1.tv_sec;
    suseconds_t usecs = tv2.tv_usec - tv1.tv_usec;
    printf("%f %f\n", pi, secs * 1000 + usecs / 1000.0);
    exit(0);
}

On my computer, I compile it with gcc and I execute it, and it takes about 300 ms. I compile it with emcc and run it with node, again it takes about 300 ms (I'm impressed). I run it with Chrome, it takes about 300 ms. I run it with Firefox, it takes about 1000 ms, which is about as much as a JS version of the same program takes.

Am I doing something wrong?

BTW, the WASM engine is part of the JS engine? The answer to "which WASM engine does Firefox have" is SpiderMonkey?

dalcde
@dexter:hailstorm.adm.srcf.net
[m]
How exactly do you time it? I would generally expect firefox to have faster wasm than chrome
Antonis Christofides
@aptiko
@dalcde The program times itself. It calls gettimeofday() just before and after the loop and subtracts the two values.
dalcde
@dexter:hailstorm.adm.srcf.net
[m]
I get the same speed on firefox, unless i keep the
console open
It seems like it is the console that somehow slows it down
Antonis Christofides
@aptiko
@dalcde That's right, thanks! I hadn't noticed!
Darwin Johnson
@schwaaweb_gitlab
Hello,
Does anyone have resources or suggestions for running pyiodide on Deno?
Thanks!
Hood Chatham
@hoodmane
@schwaaweb_gitlab There has been a decent amount of discussion about running this in Node, which probably faces a similar set of issues. I think the most recent discussion happened on iodide-project/pyodide#869. The others who are more knowledgeable about this (rth, dalcde) seemed to say that it will be easier to do once they finish merging iodide-project/pyodide#1102
Altar
@altar31
Hello,
Did you know if it possible to create Iodide private notebook?
Thanks in advance for your answers!
wlach
@wlach:mozilla.org
[m]
@altar31: not on alpha.iodide.io
Altar
@altar31
@wlach: okay thanks for your answer. Sorry for my "newbies" question, what is the way to create private iodide notebook outside alpha.iodide.io ?
wlach
@wlach:mozilla.org
[m]
@altar31: you can spin up your own copy of the server on heroku or elsewhere, though note that iodide is no longer being actively developed or maintained
Altar
@altar31
@wlach:mozilla.org : okay, i'm sorry to hear that for the project development. I 'm very enthusiastic about the spirit of this kind of project... Simple curiosity, Is it because of the "rise in power" of Jupyter project notebook "add on" (like Voilà) or things like that ?
wlach
@wlach:mozilla.org
[m]
@altar31: it's a pretty long story, the short version is that Mozilla was funding development for 2018-2019 but it got de-prioritized. I agree that this approach has a lot of potential and it seems like there are a bunch of projects following in iodide's footsteps
Altar
@altar31
@wlach:mozilla.org Alright, yes i agree this kind of approach has a lot of potential...Did you hear about similar project ?
wlach
@wlach:mozilla.org
[m]
@altar31: I know of starboard (https://starboard.gg), there's also various smaller efforts I've seen go by on this channel
dmondev
@dmondev
Hi, I'm trying to both build pyodide from source, using the Docker image, and also trying to package plotly into it, but with no success so far. Is this the best place to ask for help ?
Hood Chatham
@hoodmane
@dmondev I think here is good for quick questions, if you have a more detailed question it might be better to open an issue on github (you are welcome to open an issue to ask a specific question in any case). As a third option, you could start a thread here:
https://github.com/iodide-project/pyodide/discussions
dmondev
@dmondev
Thanks, @hoodmane !! I ended up creating an issue in github.
Paul m. p. P.
@pmp-p
same for the shebang of pyodide/emsdk/emsdk/upstream/emscripten/tools/file_packager.py problem is there https://github.com/iodide-project/pyodide/blob/master/Makefile#L159
Paul m. p. P.
@pmp-p
package building should be incremental, it's annoying in CI to have the whole set fail for only one pkg that maybe not a dependancy for others
@rth it starting to work with system python in github CI with a few tweaks https://github.com/pmp-p/pyodide/blob/master/build-system-3.8.diff
also regarding wheels names emscripten-core/emscripten#13440
rthy
@rthy:matrix.org
[m]

package building should be incremental, it's annoying in CI to have the whole set fail for only one pkg that maybe not a dependancy for others

That's why it's preferable to always build only the packages you need (dependencies will be resolved) https://pyodide.readthedocs.io/en/latest/development/building-from-sources.html#partial-builds

Unless you mean that build should go on even if a package fails to build? We could add it as a flag, equivalent to --keep-going in emerge but I think in most cases one would want to fail on first error..
Paul m. p. P.
@pmp-p
@rthy:matrix.org nvm, i used a bash loop and i came to the conclusion that a system python built with gcc+special optim instead of clang is not the best choice for pywasmcross emsdk(clang) replay and some packages.
rthy
@rthy:matrix.org
[m]
@pmp-p It's true that it would have been better/simpler if the default amd64 compiler was clang and not gcc for replaying package compilations. We are compensating for some of it in pyodide_build/pywasmcross.py and so far hadn't seen too much issues about it. What is your "gcc+special optim" setup? Could you provide more information in iodide-project/pyodide#884 ?
Paul m. p. P.
@pmp-p
maybe it would be simpler to have a ready made clang replay ( because wasm build are reproducible ) and use them is host python is found to be gcc built
Roman Yurchak
@rthy:matrix.org
[m]

yeah, but the way it's used in the Makefile PYMINOR=3.8 apparently. We never actually need the .8 by itself I think. I agree the naming might not be ideal.

For pyodide related questions please rather comment in https://gitter.im/pyodide/community

ragreenburg
@ragreenburg
Hey, I was hoping to find a way to launch Iodide on a personal server at my house and just use it locally, would anyone happen to know how to do that?
2 replies
wlach
@wlach:mozilla.org
[m]
@ragreenburg: you mean the iodide notebook itself? aside from copying/pasting, no ☹️ . there's an issue filed about this here (unlikely to be resolved at this point): iodide-project/iodide#2551
ragreenburg
@ragreenburg
@wlach:mozilla.org Shoot, thanks for the information though! What would be required to make this possible?
wlach
@wlach:mozilla.org
[m]
@ragreenburg: a bunch of work on the iodide client code and server -- not sure how much but as the README states it's no longer under development so a bit unlikely to happen
ragreenburg
@ragreenburg
Right, okay. I was just thinking if maybe it was something I could work on myself but if it a big project then probably not possible by myself.
2 replies
Adrian Gschwend
@ktk
ah really :)
cool