Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Hugo Granström
    @HugoGranstrom
    Haha cluttering the namespace with several houndred thousands of lines of code, what could go wrong 😂🙈
    A solution is needed indeed
    No idea actually if it works 😬 we'll see I guess. What other people are there out there that you know of?
    Vindaar
    @Vindaar
    regarding the imports I think it should be solvable by having all major submodules one might want to import in the src/ directory directly. So like src/tensor.nim etc. and arraymancer.nim is also sitting there, which just imports those submodules and exports them.
    I think nimble will complain about "bad project structure" or whatever it is, but well
    Other people? Hm, from the top of my head maybe @bluenote10 and @brentp. Aside from that not sure right now
    Brent Pedersen
    @brentp
    hi, guys. didn
    t know about this channel til now..
    Hugo Granström
    @HugoGranstrom
    Hi Brent! Would've been surprised if you did... 😅 Not much has happened here 'til today
    (cross channel pinging seems to work at least)
    Vindaar
    @Vindaar
    haha, that's established :D
    matrixbot
    @matrixbot
    BarrOff Hello Vindaar and Brent! Great to see some new faces on here. 😀
    Mamy Ratsimbazafy
    @mratsim
    Hey there
    yes cross channel pings seem to work
    I learned that nimble can handle monorepo: for example https://github.com/nim-lang/graveyard
    so that's one way of having multiple modules
    If we do a monorepo I don't recomment using the src approach as long as nimble eats the directory, @Vindaar and me had troubles with that
    otherwise we don't need an actual monorepo, just an umbrella organization
    and we put all the stuff under it, it could be SciNim for example
    NumNim is a bit hard to pronounce :p
    doesn't roll off the tongue
    Hugo Granström
    @HugoGranstrom
    Haha yeah it has the risk of turning into Nomnom if you say it often enough 😂 We could go with the slogan "The tastiest Numerical package" then 🤪
    Oh monorepos was new to me
    Hmm an umbrella org could work as well, it depends on how we want to do it I guess. Could perhaps get a bit cluttered with issues if the repo contains 5 different project....
    Mamy Ratsimbazafy
    @mratsim
    more like drowned, arraymancer has a lot of issues opened since I use them as a brain dump
    Vindaar
    @Vindaar
    I think it might be a good idea to do both? An umbrella org is useful regardless, because even if one decides to have one large package / mono repo there's bound to be stuff that doesn't quite belong in there, but's still relevant for scientific stuff
    I would personally try to start with one module (theoretically also a mono repo, but that kinda depends on how easy those are actually to use with nimble!) and see how far one can get
    especially regarding compilation times. Need to make sure that it's possible to import a subset that really is a subset
    also I totally forgot about @jcosborn and @jxy of https://github.com/jcosborn/qex (not sure if either of them is using gitter)
    Vindaar
    @Vindaar
    but for a start I wouldn't mind just trying to grow arraymancer by adding the functionality of other modules (numericalnim, seqmath etc.).
    Would just be nice to have a single module that defines linspace instead of 3 :P
    Mamy Ratsimbazafy
    @mratsim
    I added arange AFAIK not sure under what name though
    Hugo Granström
    @HugoGranstrom
    Haha you can't get enough linspace and arange procs 😂
    It feels like there are a lot of stuff happening in Arraymancer atm (Laser and new ptr+length) so perhaps this isn't the best time to start adding even more in it right now? Or am I just to picky?
    A separate Arraymancer-addons package or something in the meanwhile as a dependency of Arraymancer perhaps? So when you install Arraymancer it automatically install this as well
    Hugo Granström
    @HugoGranstrom
    But this may not feel as neat as having it all in the same package... 🤔
    Mamy Ratsimbazafy
    @mratsim
    na, nothing much. For now I've finished on the low-level stuff
    implementing the Lux compiler would take lots of time. I've been very impressed by the result of a new project by ETH Zurich https://github.com/spcl/dace
    but doing it in Nim requires a graph library that works at compile-time
    however it neatly addresses some problems that I may be stuck with for distributed computing
    on a more impactful thing, I was otherwise thinking on implementing at least a simple random forest and/or some OneHotEncoding so that we can at least do the "Titanic" playground competition from Kaggle fully in Arraymancer and obtain OK results
    Hugo Granström
    @HugoGranstrom
    Alright then! 👍 As there are no obstacles it should just be a matter of starting to implement things 😅
    Once Lux is done it will be really cool!
    I'm so amazed by how much you can abstract away from the user and still make the code run so much faster
    Machine learning algorithms are cool as well and one of Arraymancers primary focuses if I understand it correctly?
    Is there any point in implementing it all from scratch (apart from that you actually learn how it works) or is porting for example SciKit's random forest implementation acceptable?😅
    Mamy Ratsimbazafy
    @mratsim
    porting is OK
    yes ML algo are a primary focus because I don't know much about physics (besides being trained as a mechanical engineer so I do know about DE and used multiple FE simulations, including Nastran, Patran, Ansys Fluent for airplane design and many other computational dynamics tool for melted iron and steel but that was only as a user)
    Mamy Ratsimbazafy
    @mratsim
    one thing I will definitely not implement from scratch is gradient boosted trees given the work that is being done in XGBoost and LightGBM
    another reason of learning from scratch is that you can be much faster than Python if the low-level tooling match the problem https://github.com/mratsim/Arraymancer/pull/384#issuecomment-536682906.
    note that I didn't reimplement from scratch, just reused linear algebra components from Lapack