Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Jul 01 09:43
    dmvianna added as member
  • Jun 15 04:55

    ocramz on gh-pages

    Add pcg-random (compare)

  • Jun 14 16:08
    ocramz labeled #42
  • Jun 14 16:08
    ocramz labeled #42
  • Jun 14 16:08
    ocramz labeled #42
  • Jun 14 16:08
    ocramz labeled #42
  • Jun 14 16:08
    ocramz labeled #42
  • Jun 14 16:08
    ocramz labeled #42
  • Jun 14 16:08
    ocramz opened #42
  • Jun 14 16:08
    ocramz opened #42
  • Jun 06 18:21

    ocramz on gh-pages

    Fix graphite link Merge pull request #41 from alx… (compare)

  • Jun 06 18:21
    ocramz closed #41
  • Jun 06 18:21
    ocramz closed #41
  • Jun 06 17:32
    alx741 opened #41
  • Jun 06 17:32
    alx741 opened #41
  • Jun 06 16:46

    ocramz on gh-pages

    Add graphite Merge pull request #40 from alx… (compare)

  • Jun 06 16:46
    ocramz closed #40
  • Jun 06 16:46
    ocramz closed #40
  • Jun 06 14:43
    alx741 opened #40
  • Jun 06 14:43
    alx741 opened #40
Manuel Bärenz
@turion
How do compact-regions and https://github.com/esoeylemez/foreign-store compare?
Kevin Brubeck Unhammer
@unhammer
I've used foreign-store with ghci when developing, but it feels a bit hacky, and for some reason I haven't bothered investigating I get double the memory usage. How big is a "large" dataset @o1lo01ol1o ?
(There should be a word like "large" defined to mean DoesNotComfortablyFitInRAM)
Manuel Bärenz
@turion
And how big is your RAM ;)
:+1:
Tim Pierson
@o1lo01ol1o
@Drezil ah yea, compact regions might simplify things. @unhammer "large" in this case is about 100gb.
Tim Pierson
@o1lo01ol1o
This means that the info tables of the receiving process must be laid out in exactly the same way as from the original process; in practice, this means using static linking, using the exact same binary and turning off ASLR.
Looks like compact might not be a fit; I have one binary that does the processing-dumping and other binaries that do the reading. Maybe I'll just roll serialize instances for all the data . . .
Manuel Bärenz
@turion
@o1lo01ol1o serialize instances should usually be doable with Generics, right? The only trouble is that you can't change the definition of the type between two runs
Tim Pierson
@o1lo01ol1o
@turion Yup
Marco Zocca
@ocramz
oh hai everyone, looks like I have lots of catching up to do.
Doug Burke
@DougBurke
I recently added a tutorial to hvega (an API to create Vega-Lite visualizations) - based on the Elm tutorial - and would be interested on feedback. You can find it on hackage at http://hackage.haskell.org/package/hvega-0.4.0.0/docs/Graphics-Vega-Tutorials-VegaLite.html
Aleksey Khudyakov
@Shimuuar
Hi! I skimmed hvega tutorial and understood nothing but got impression that one have to specify manually everything
Tim Pierson
@o1lo01ol1o
@DougBurke I didn't find @Shimuuar 's observation to be a problem. However, if someone were to make some carbon-copies of known graph paradigms with a simple api, it might help newcomers.
Adam Conner-Sax
@adamConnerSax
@o1lo01ol1o @DougBurke I’ve worked at that some, esp using Frames to add some types to the data. I’ll post a github link later. But once you get used to it, most charts are pretty sui generis and the templates for them ended up being as limiting as they were useful. The examples are the most useful thing, I think.
Tim Pierson
@o1lo01ol1o
:plus1:
Doug Burke
@DougBurke
@Shimuuar - an aim of Vega Lite (I believe) is that you should be able to "compactly" specify a visualization without having to define everything. That is, it is rather declarative - "plot up this column versus this column" and it will decide on axis ranges, titles, symbols etc. You can then add in extra "options" to change the default choices. Now, whether hvega manages to capture "compactly specify" is a different question. I would appreciate any comments on the existing documentation and how they could be improved.
I should add that as well as the frames interface by @adamConnerSax there is also http://hackage.haskell.org/package/rigel-viz by @ocramz (which I haven't looked at)
Aleksey Khudyakov
@Shimuuar

@DougBurke I just wanted to build very boring scatterplot gave up and went back to Charts.

For tutorial it probably would be good idea to very simple and minimal plot and then expand it: axes labels, title, multiple plots, log scale etc.

Adam Conner-Sax
@adamConnerSax
Didn’t know about Rigel-viz! Cool.
Doug Burke
@DougBurke
I only found out about rigel-viz when I was looking for a category for http://hackage.haskell.org/package/vega-view - which is a possibly-pointless web server to let you view on-disk vega/vegalite visualizations (aka I could have just installed https://github.com/vega/vega-desktop but decided to write omething in Haskell instead)
Aleksey Khudyakov
@Shimuuar

@DougBurke I finally managed to plot a parabola using hvega. I wouldn't call it compact:

let enc = encoding
        . position X [ PName "X", PmType Quantitative ]
        . position Y [ PName "Y", PmType Quantitative ]
let dat = dataFromColumns []
        . dataColumn "X" (Numbers [1 .. 10])
        . dataColumn "Y" (Numbers $ map (^2) [1 .. 10])
toVegaLite
    [ dat []
    , mark Line []
    , enc []
    ]

I'd say it's completely unsuitable for interactive plotting. Why should I specify name of column both in data and axis specification? Surely it could be inferred from data type [(Double,Double)]

Adam Conner-Sax
@adamConnerSax
@Shimuuar, @DougBurke can reply for himself, but hvega is intended to lightly wrap the json building.
So you can then wrap that how you like with much more inference. I do some of that with Frames, etc.
But it requires some use, for me anyway, to see what I would like inferred and what not. For publication ready charts you often want to fiddle with the details and even just for data exploration I’ve found that I rarely use much simplification except the Frames part. Perhaps a different question is what would a truly “haskelly” grammar-of-graphics API look like? Maybe we could work towards that and then use hvega (and other things!) as back-ends.
But some of that arbitrariness that comes from the “stringiness” of the column names can def be handled using a typed data frame (like Frames) and doing some wrapping. YMMV, of course.
Aleksey Khudyakov
@Shimuuar

I suspect I'll end up doing something along these line. One thing is designing good APIs is difficult. Especially ones intended for interactive use

Generally I think one shouldn't need do much beyond plot someData on one hand. On other hand sometimes plots will be tweaked to death and this shouldn't be difficult either

Tim Pierson
@o1lo01ol1o
@Shimuuar On another topic, I've been using monoid-statistics and find myself needing windowed statistics. There exists some mentions of monoidal-rolling statistics , ekmett has a semi-cryptic tweet here which unfortunately isn't enough to derive an implementation from (that I can see anyway) and this paper that also uses an okasaki queue and some pointer swapping that seems less pleasant to implement. Have you looked at any approaches?
Aleksey Khudyakov
@Shimuuar
@o1lo01ol1o I've never heard about such approach. I skimmed through paper and I think I'm starting to undesrtand how it works (tweet is cryptic but describes same approach)
I'll think more about it. Seems useful
Tim Pierson
@o1lo01ol1o
Yeah, super useful. The issue with the paper is that the flip method needs to do some juggling that one would need to roll their own queue for. This of course is possible, but the dequeue package looks pretty solid and I'd rather not reinvent the wheel.
So I keep hoping kmett will pop up in IRC so that I can pester him about it.
Doug Burke
@DougBurke
@Shimuuar as @adamConnerSax says, the point of hvega is to provide some way to create a Vega-Lite specification. It is based on the Elm API because that required little thought to implement. I would love there to be a "higher level" API, but whether there really will be one API to rule them all, or a range of them, is to be seen. I don't think I am the person best placed to design such an API, but we already have some examples (with http://hackage.haskell.org/package/rigel-viz and https://github.com/adamConnerSax/Frames-hvega).
Some extra complications for the vega/vega-lite design space at this time: the underlying JSON specifications are moving pretty fast, so while I would appreciate a more type-safe API than hvega provides, it is not obvious to me how to do this and be responsive to changes in the specs.
Adam Conner-Sax
@adamConnerSax
@DougBurke @Shimuuar I’m pretty happy with hvega as is but I would also love a more strongly-typed interface. I’ve thought some about that but I’m not quite sure I have a clear handle on that API design. Partly, I am starting to see how useful the g-o-g ideas are but don’t yet understand it well enough to see what those ideas look like mathematically. I like that hvega is tracking the vega-lite spec and I think a higher-level g-o-g API would be great, esp if it had an hvega and diagrams back-end.
@DougBurke Anyway, thanks so much for the work! You can see some in action here: https://blueripple.github.io/research/preference-model/p1/main.html
Doug Burke
@DougBurke
:thumbsup:
(I should say that Jo Wood, who created the Elm version that hvega is based on, really deserves the credit, I'm just along for the ride!)
Aleksey Khudyakov
@Shimuuar
Yes. Design of high level API is open problem. I'd say we are in "let hundred flowers bloom" phase. I'm personally tempted to explore design where high level API is created first and vega-lite is treated as implementation detail.
Somewhat unrelated. Plotting APIs tend to create tons of names. On one hand you don't want to dump them into unqualified namespace on the other haskell (and ML-family) syntax is outright hostile to qualified names: Plot.axes . Plot.coordX . Plot.label .~ "Cool label" I wish for something like
open Plot
  All names from Plot could be used unqualified here
idontgetoutmuch
@idontgetoutmuch
I am thinking of setting up an azure vm with cuda - any views on what would be a good entry level machine? https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes-gpu
Tim Pierson
@o1lo01ol1o
Are you tied to Azure?
idontgetoutmuch
@idontgetoutmuch
Sort of - I have a few VMs set up including a nixos VM
I set up an account about 6 years ago but only started really using it about 6 months ago
Tim Pierson
@o1lo01ol1o
Right, I was going to suggest paperspace for cuda machines as their pricing and capabilities are nice
idontgetoutmuch
@idontgetoutmuch
I know very little about this space - perhaps I can use the image I created for azure for paperspace also?
Tim Pierson
@o1lo01ol1o
I don't have much experience with those tesla models, but I've read throughput can vary depending on what you're doing, so I'd maybe google
You can just use an ubuntu machine on paperspace and install nix
"just"