These are chat archives for AvaloniaUI/Avalonia

27th
Nov 2018
John Källén
@uxmal
Nov 27 2018 00:03
Using a listbox to bind those same 10000 items also eats up hundreds of megabytes.
John Källén
@uxmal
Nov 27 2018 00:12
...and I just found VirtualizationMode on Listbox, but apparently not on TreeView. Is this something that is in the works?
Nikita Tsukanov
@kekekeks
Nov 27 2018 05:25
@battlebottle
There are two renderers: ImmediateRenderer and DeferredRenderer.
  • ImmediateRenderer always renders the entire window on Paint event (WM_PAINT, basically) by traversing the visual tree and calling Render on visuals.
  • DeferredRenderer instead provides visuals with a "virtual" DrawingContext that records drawing commands to produce a scene graph. Said scene graph is usually rendered from background thread, but when window receives Paint event, it renders the scene on UI thread
Due to the fact that we have a scene graph, DeferredRenderer can associate layers with some subtrees of the scene graph
So animations that change opacity and render transformations only change layer properties
And we don't have to redraw render contents
@uxmal our TreeView doesn't currently support virtualization
Brian Hoary
@battlebottle
Nov 27 2018 12:31
@kekekeks That’s interesting. Am I understanding right? Both the UI thread and a background thread (render thread) can both render the window? Does that not create issues with access to the actual renderer (Direct2D)?
Nikita Tsukanov
@kekekeks
Nov 27 2018 12:45
Most window systems allow rendering from arbitrary thread
The only one which we have issues with is GTK
But we are getting rid of it
In favor of XLib and libwayland layter
Brian Hoary
@battlebottle
Nov 27 2018 12:55
@kekekeks That’s interesting. Does AvaloniaUI support anything like asynchronous rendering? This is supported in cocoa in OS X and iOS where drawing commands can be rendered to a layer from a background thread. My understanding is that if rendering a asynchronous layer takes a long time, it shouldn’t block the rendering of other layers in the window, in principle.
Nikita Tsukanov
@kekekeks
Nov 27 2018 12:58
Cocoa doesn't support it anymore, I think
I'm not sure if it's still benefitial
UI should be utilizing GPU instead of CPU
Oh, it's reenabled
Nikita Tsukanov
@kekekeks
Nov 27 2018 13:03
We might want to allow widgets to draw directly on render thread, I think
So they won't depend on UI thread being blocked
That will need a rework of DeferredRenderer though
ahopper
@ahopper
Nov 27 2018 13:04
yep I definitely have a use for that.
Nikita Tsukanov
@kekekeks
Nov 27 2018 13:05
That would be useful for video output, manually drawn stuff, etc
Brian Hoary
@battlebottle
Nov 27 2018 13:09
@kekekeks Direct access to the render thread would be wonderful to me.
@kekekeks I’m also fond of what android does (did?) for animations. I read that a display list is passed from UI thread to the render thread, but the display list is aware of what animations are happening to different draw elements, and if the UI thread is ever blocked for a frame, the render thread can continue correctly rendering animations.
Roman Minyaylov
@roman-minyaylov
Nov 27 2018 13:35
@kekekeks For video output adding native control is very useful, most of frameworks like DirectShow or GStreamer by default have very fast renderers that that render to any native control that have handle
So, some way to add native control (panel for example) is what really required for video output
I'm using Image and WriteableBitmap now as bad workaround
DirectShow have video renderers with several video streams mixing, OSD, etc., that should be done manually because no ways to add native control in Avalonia (But possible in GTK, MonoMac, WPF/WinForms)
Benedikt Schroeder
@Gillibald
Nov 27 2018 13:43
Your so called native control is just a child window embedded into the application and that is very much possible to implement.
Nikita Tsukanov
@kekekeks
Nov 27 2018 13:51

any native control that have handle

That would be a problem

1) we won't be able to overlay our own UI on top of said native window
2) Wayland doesn't allow to have subsurfaces created by other clients
Roman Minyaylov
@roman-minyaylov
Nov 27 2018 13:53
@Gillibald yes, that what I need
Nikita Tsukanov
@kekekeks
Nov 27 2018 13:53
The second one actually breaks SMPlayer and other mplayer GUIs on Wayland
Roman Minyaylov
@roman-minyaylov
Nov 27 2018 13:54
@kekekeks 1. Correct 2. Sorry, have no ideas how it's worked
Nikita Tsukanov
@kekekeks
Nov 27 2018 13:55
That all could be solved if we could ask for data from the render thread
Roman Minyaylov
@roman-minyaylov
Nov 27 2018 13:55
We can't ask for data
In Windows video rendering, video streams mixing, etc. done using Direct3D and worked really fast
So, most of operations performed on GPU
And that's how all worked for 20+ years already
Currently I just grab the frames and draw on Image control, but all features like video mixing or OSD are lost
Nikita Tsukanov
@kekekeks
Nov 27 2018 13:58
Is it possible to draw to DirectX 9/11 texture?
Instead of HWND
We could work with DirectX texture
Even with Skia+OpenGL backed via ANGLE
Roman Minyaylov
@roman-minyaylov
Nov 27 2018 13:59
I don't think so. We can pass HWND to video renderer, and that's the same for DirectShow and renderer used in GStreamer in Windows. So, all renderers use HWND.
Nikita Tsukanov
@kekekeks
Nov 27 2018 13:59
Windows 8 has introduced DirectComposition API
Roman Minyaylov
@roman-minyaylov
Nov 27 2018 13:59
GStreamer can use OpenGL on Linux and Windows
Nikita Tsukanov
@kekekeks
Nov 27 2018 14:00
That allows to render video stream to a DirectComposition layer instead of HWND
Roman Minyaylov
@roman-minyaylov
Nov 27 2018 14:00
OK, I know a lot of ways including making everything on CPU but that's a lot of changes in code.
Let me check DirectComposition
Nikita Tsukanov
@kekekeks
Nov 27 2018 14:01
We had plans for using DirectComposition for our layers
But it's not compatible with Win7, so I don't think that will happen anytime soon
Roman Minyaylov
@roman-minyaylov
Nov 27 2018 14:02
OK, but also it's Windows only, not Mac, Linux, etc.
Roman Minyaylov
@roman-minyaylov
Nov 27 2018 14:02
Native window (control) resolve all problems and no require any code writting in all video frameworks
Nikita Tsukanov
@kekekeks
Nov 27 2018 14:02
Media Foundation seems to be supporting textures
Roman Minyaylov
@roman-minyaylov
Nov 27 2018 14:03
Media Foundation is very limited, DirectShow still actual
Nikita Tsukanov
@kekekeks
Nov 27 2018 14:03
1) "native" window embedding won't work with Wayland which replaces X11 on Linux
So it won't solve any problems on Linux, unfortunately
In case of OSX we could use CoreAnimation layers for composition
Nikita Tsukanov
@kekekeks
Nov 27 2018 14:04
CoreVideo definetely supports CA layers
bad-plugins
Roman Minyaylov
@roman-minyaylov
Nov 27 2018 14:05
As I told I can made all even on CPU but all of this already done in GStreamer for example. Your way it's a lot of pain
Nikita Tsukanov
@kekekeks
Nov 27 2018 14:05
I don't see where I can specify wl_surface for that plugin
Again, that does not work on Wayland
Roman Minyaylov
@roman-minyaylov
Nov 27 2018 14:06
bad-plugins doesn't mean really bad, just can be not production ready or something like it
OK, will work everywhere except wayland for now
Nikita Tsukanov
@kekekeks
Nov 27 2018 14:07

for now

That's the problem

The reason it doesn't work for wayland
Is not some not-yet-implemented feature
Roman Minyaylov
@roman-minyaylov
Nov 27 2018 14:07
"
The application can supply the surface with
gst_video_overlay_set_window_handle(), it must also create a context
using gst_wayland_display_handle_context_new() and set that context on
the pipeline with gst_element_set_context(). "
Nikita Tsukanov
@kekekeks
Nov 27 2018 14:08
Does it support subsurfaces?
Roman Minyaylov
@roman-minyaylov
Nov 27 2018 14:12
I don't know
You can get surface from GDK window
Anyway, Wayland not a problem as I see, will work using some way, GStreamer have support already
And same API used to set Wayland surface instead HWND
Jeremy Koritzinsky
@jkoritzinsky
Nov 27 2018 17:35
If you look at AvaloniaAV there's code there for rendering video from MediaFoundation to Avalonia bitmaps via the GPU. The CPU path currently doesn't work, but the GPU one works perfectly.
MediaFoundation actually works pretty well imo. Would work better if we had native layers with DComposition though.
Roman Minyaylov
@roman-minyaylov
Nov 27 2018 18:15
@jkoritzinsky I'm using RGBA images from CPU and it's worked perfectly. Do you think that AvaloniaAV way is crossplatform? Also, how it can help me with things that I described like video mixing of several video streams or OSD that already available in DirectShow, for example? That's only basic rendering, without possible usage of existing renderers. Not interesting. Same as Image control usage.
Basic rendering is not a problem. Problem that I can't use existing video renderers in GStreamer or DirectShow.
Nikita Tsukanov
@kekekeks
Nov 27 2018 19:13
gst-plugins-gl should be able to render video to texture on demand
Jeremy Koritzinsky
@jkoritzinsky
Nov 27 2018 20:05
Another option other than gstreamer would be the ffmpeg apis
The core of AvaloniaAV should be crossplat
Roman Minyaylov
@roman-minyaylov
Nov 27 2018 20:06
@jkoritzinsky GStreamer uses ffmpeg libraries inside for decoding/encoding, have pipeline of elements and many other things. FFMPEG is not made for this.
AvaloniaAV based on Media Foundation that windows-only
Jeremy Koritzinsky
@jkoritzinsky
Nov 27 2018 21:04
Yep the MediaFoundation part is Windows only