Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 31 2019 16:26
    kmuehlbauer commented #1568
  • Jan 31 2019 16:15
    kmuehlbauer commented #1568
  • Jan 31 2019 15:53
    djhoese commented #1568
  • Jan 31 2019 15:46
    kmuehlbauer edited #1568
  • Jan 31 2019 15:44
    kmuehlbauer opened #1568
  • Jan 31 2019 15:32
    djhoese commented #1566
  • Jan 31 2019 15:30
    djhoese commented #1567
  • Jan 31 2019 14:18
    GuillaumeFavelier commented #1561
  • Jan 31 2019 14:15
    GuillaumeFavelier commented #1561
  • Jan 31 2019 12:46
    opekar synchronize #1130
  • Jan 31 2019 09:54
    kmuehlbauer commented #1567
  • Jan 31 2019 09:53
    kmuehlbauer synchronize #1567
  • Jan 31 2019 09:37
    kmuehlbauer labeled #1567
  • Jan 31 2019 09:37
    kmuehlbauer labeled #1567
  • Jan 31 2019 09:37
    kmuehlbauer labeled #1563
  • Jan 31 2019 09:36
    kmuehlbauer labeled #1563
  • Jan 31 2019 09:28
    kmuehlbauer commented #1567
  • Jan 31 2019 09:27
    kmuehlbauer synchronize #1567
  • Jan 31 2019 09:19
    kmuehlbauer assigned #1563
  • Jan 31 2019 09:17
    kmuehlbauer opened #1567
Philip Winston
@pwinston
But the other way is TiledImage is Visual that internally manages these textures, and whose draw() draws those textures in the grid pattern.
The "one big Visual" way seems like it might be more efficient. But I suspect the SceneGraph way will be pretty decent. And simpler maybe. So I'm thinking start there and if it's the weak link in performance consider something fancier.
David Hoese
@djhoese
@pwinston It is your time so :+1:. Go with what you feel. That said, as the maintainer of vispy, I can tell you that we have users complain about performance issues when they have 6+ Visuals in a single Scene. I'm sure napari has done a lot more than that and been fine. The nice thing about the tile atlas idea is that you don't have to create or remove any objects from the GPU. You don't even need to recompile the shader code. It is only updating the vertex buffer, the texture coordinate buffer, and the texture data itself. Another nice thing about it is that you don't need to have one image, you tell the code "write this tile to this subset of the texture" and it can control everything else.
Philip Winston
@pwinston
I'm not really seeing how a texture atlas would work here. As you pan and zoom the tiles are constantly changing. You might draw these 6 tiles one frame but then 6 different tiles the next frame. So are you suggesting dynamically building a texture atlas potentially every frame? And then rendering out of that atlas?
A texture atlas seems ideal if you have all the textures up front. Since we are rendering out of an octree, the set of textures being rendered will be varying a lot.
But yes if it can only support 6 visuals, that will be a huge problem. Why is slowing it down at that point? Is it iterating through the graph, changing shaders? What's really the slowness from?
I'm used to games and simulations with pretty huge scene graphs and still running at 60Hz. Maybe a vispy node is not a node in the way I'm thinking, or it's just Python vs. C++ type of thing.
Certainly, I want a solution that works well with vispy, so yeah I want something that will perform well.
Philip Winston
@pwinston
One thing I did notice is drawing a new texture for the first time seemed to take like 5-10ms. That seemed super long. I'm not sure the # of textures from ms that we loaded but I think it was many textures in 1ms. But that was C++.
Philip Winston
@pwinston
We can certainly do it all inside one visual if we have to, whether we use an atlas or not. Separate visuals seemed cleaner, but not if it will be a bottleneck.
Philip Winston
@pwinston
I did a tiled renderer "Raveller" at Janelia and that was Python + OpenGL and it performed amazingly well. That was just a for loop in Python drawing quads, no shaders, no scene graph. That was 8-9 years ago too so the hardware was a lot slower.
David Hoese
@djhoese

@pwinston The way I've always understood it is that OpenGL is drawing each GL Program one at a time. Each Visual in vispy is a GL Program so having to draw a lot of visuals means the GPU has to draw visual 1, then visual 2, then visual 3, and so on. The more visuals you add the longer it takes for vispy (and the GPU) to get back to the first visual for redrawing.

The number 6 was just something I remember hearing one user complaining about. I don't remember what they were actually doing. It could be more than that and obviously it depends on the hardware running it.

I am honestly also surprised by this performance but haven't had time to really figure out if this can be expected. That is, is it vispy's python that is slowing things down (the GLIR conversion, etc) or is it actually the GPU taking time to draw things. Because of my lack of time, it is usually easiest to tell users looking for frames per second to combine their visuals into as few visuals as possible. Typically they are doing things like making a ton of SphereVisuals, then they could just make a single MeshVisual, or something like that. So the optimizations is more obvious.

Oh and the tile atlas can be more thought of as a tile cache. Some outer logic is filling it with data based on camera events (pan/zoom) and filling it with the necessary data. The TiledImageVisual would just be told "the tiles were updated, here's the data, here's the coordinates, go". And of course as a "cache", it could hold more tiles than are being displayed and the TiledImageVisual is only told to display some of the tiles.

While this may require logic every frame, how would you do it with separate ImageVisuals? You'd still have to do the logic of what tile is shown and which one isn't, right?

Philip Winston
@pwinston
Yes either way you are "swapping" in tiles. I say potentially every frame, but realistically you'd expect some temporal coherence in the tileset. Swap it in, then draw some number of frames, then swap again. So I think there are two major costs, the cost of the swap and then the cost of the draws with a fixed set of tiles.
A tile atlas/cache seems like a perfectly good approach. But it's doing a lot from scratch that a scene graph and stand-alone ImageVisuals should be getting you. I'm totally not adverse to doing it that way if it's necessary. We want it be fast.
But yeah the number "6" stands out as very odd. It's like a phone that can only hold 6 songs, or an electric car that can only go 6 miles on a charge. It's at least two order of magnitude smaller than I'd expect, maybe three.
One thing most scene graphs do is sort nodes by shader or other state, to minimize the number of state transitions. For example set up the "tree shader" then draw 10,000 trees. If vispy does not at all do that, then the it's incurring an unnecessary setup/teardown cost in many cases. But even then I think 6 is absurdly low. So yeah I wonder if that was just a bogus report, or a mis-understood situation.
Philip Winston
@pwinston
At any rate, having a scene graph but then basically being unable to use it, and asking users to create giant stand-alone visuals that internally manage their own resources and do their own rendering, is an unfortunate situation an engine that's supposed to be "fast, scalable, and easy to use."
Anyway I might start with separate ImageVisuals just to get something drawing quickly and work out the mechanics. But I'm fine with going the texture cache route. Even if the scene graph was fast, that would likely be even faster. So I don't see any real downside once it's working.
Philip Winston
@pwinston
Thanks for ideas. Certainly was good to get the heads up because I was 99% expecting the separate ImageVisuals to be totally fine.
David Hoese
@djhoese

@pwinston Vispy was rapidly developed by a group of scientists (mostly) many years ago. Almost none of them have time for the project any more. What you describe for SceneCanvas optimization is likely something they had planned, but is not currently implemented. For the last couple years, I've just been keeping vispy alive with minimal actual features being added (by me at least).

We are always accepting of pull requests so if you spot something while you're doing your work, please let us know as either a github issue or even as a pull request.

I think in general vispy tried to be as flexible and generic as possible. This may make it difficult to do things like sorting Visuals by shader or opacity. There are some features borrowed from the glumpy library that would perform better for some of this stuff (called "collections") that haven't gotten the full Visual/SceneCanvas treatment yet. This would likely help some users.

Try out the multiple ImageVisuals and see how it goes. It will likely work just fine and possibly perform just fine.

Philip Winston
@pwinston
Yes, great, it sounds like a plan. The FUD that "more than 5 nodes is too slow" would be good to dispel. Maybe the users in question were using 100% software rendering for a example, and even a modest card does way better?
That said, a single TiledVisual should be wicked fast. I see zero downside once it's written and debugged. It would be a neat bit of technology.
Philip Winston
@pwinston
It's just that I'd hate for there to be this self-perpetuating myth that the scene graph simply is not usable, and so everyone must make giant custom visuals. If in fact it's not actually that dire.
BTW, in napari we integrated some performance tools that spit out traces which chrome://tracing can view. The nice thing is we have a config file and we can monkey patch in timers into vispy without modifying the source. This might come in handy looking at performance issues.
See documentation with some images: https://napari.org/docs/dev/events/perfmon.html
But then more extensive images where Kevin was using the tool to look into perf issues:
Philip Winston
@pwinston
So that might prove useful.
Philip Winston
@pwinston
Also, I realize one of my standard test cases has 16 "layers" which in napari means 16 ImageVisuals. And it renders at 16Hz. So yeah that user must have been had a certain specific situation that was closing them problem. Either the code or their machine, etc.
I looked at this some. Using a tiny 16x16 texture or even commenting out the GLIR 'DATA' command, and calling set_data on an ImageVisual is still like 15-20m. So it's doing something significant that doesn't seem to be related to the texture.
About 4ms of that is ModularProgram.build. That's a pretty big problem. But it's the actual run-time execution that's even slower. Something it's doing even with 16x16 texture or no texture at all.
Another thought I had about a custom TileVisual is we'd have to duplicate any features of ImageVisual that we needed. Like clim, gamma, etc. I'm not sure which we need.
Philip Winston
@pwinston
So those seem like the two options still. Make ImageVisual much faster, realizing it seems to not be the texture that's a problem. Or if we create a new visual make sure we duplicate what ImageVisual does that we neeed.
I'll have to look into it more later.
David Hoese
@djhoese
@pwinston subclassing the ImageVisual would require overriding like two methods.
David Hoese
@djhoese
That assumes clim and gamma are the same for all tiles
Philip Winston
@pwinston
I think your suggestion for one new TiledVisual which internally manages a cache of tiles like an atlas is definitely our best bet. It'd be nice to make rendering many ImageVisuals faster. But I think that might be challenging, it would involve changing vispy itself, and in the end, it would still not beat your one-visual idea for speed. This way we are just writing one new thing, and changing nothing. Thanks for the idea and the pointer to the SIFT code, both were very helpful.
David Hoese
@djhoese
if you are making this, please call it TiledImageVisual and please make a PR for it in vispy. Thanks for talking it out. It was good to think about all the ways this could be done.
Robin White
@robintwhite
Hi all, I am trying to see about getting VisPy up and running on Google Colab. I have run into issues essentially right away. I'm not sure how to activate in notebook extension or if this is actually possible. I ran through Cyrille's example from the cookbook for jupyter notebook, and get no errors but no plot either...has anyone tried and succeeded?
David Hoese
@djhoese

@robintwhite What cookbook are you referring to?

As the vispy installation instructions say the jupyter widget is considered experimental and in general should not be used for "operational" uses. It is more of a proof of concept. It works but suffers from some major performance issues for anything interactive or anything that updates over time quickly.

Given that Cyrille hasn't made a commit on vispy in years, I would recommend starting with the current installation instructions for getting anything to show up in a jupyter lab/notebook session.

Robin White
@robintwhite
Thanks @djhoese I was referring to this one: https://github.com/ipython-books/cookbook-code/blob/master/notebooks/chapter06_viz/06_vispy.ipynb I do see some newer examples now posted by yourself which I will investigate further first. I understand the limitation, I am just curious at where I can be using it
David Hoese
@djhoese
@robintwhite that cookbook might work but I would remove the app.run() part at the end
if it still doesn't work (maybe try it in a regular jupyter environment first) then I would try putting all the code in one cell and see if it behaves better
Juan Nunez-Iglesias
@jni
Does anyone here know about missing freetype dlls on Windows? Some napari users are running into this: https://forum.image.sc/t/error-freetype-library-not-found-during-adding-points/41742/7
David Hoese
@djhoese
@jni The freetype-py (or whatever the name of the package is) no longer distributes the binaries with 32-bit Windows if I recall correctly. In our vispy azure builds we had to add the environment variable FREETYPEPY_BUNDLE_FT=1 to tell pip to install freetype-py from source and include the binaries of freetype. Otherwise, python 2.7 and 32-bit Windows was dropped in the last release: https://github.com/vispy/vispy/pull/1886#issuecomment-656992425
(and that's just the wheels for 32-bit)
Juan Nunez-Iglesias
@jni
@djhoese thanks for the answer! I'm only unclear about what you're saying about FREETYPE_BUNDLE_FT=1 — does that mean that the freetype binary should be included in the vispy wheel? Or is it only a setting that you recommend for installing freetype on Windows 32?
David Hoese
@djhoese
@jni That is a setting I recommend when installing freetype-py on Windows 32. It tells freetype-py that it shouldn't try to use a system-installed freetype C library and should instead include its own copy. Since vispy depends on freetype-py then this would be needed when installing vispy too (but is not part of the vispy installation process necessarily). I could add it to the installation instructions for vispy.
Juan Nunez-Iglesias
@jni
Great, thanks!