These are chat archives for AvaloniaUI/Avalonia

13th
Jan 2017
danwalmsley
@danwalmsley
Jan 13 2017 00:08
@grokys you there?
I have a quick question
Richard Simpson
@RichiCoder1
Jan 13 2017 00:17
Interesting project that just popped up on my reddit feed: http://www.ammyui.com/2017/01/04/ammy-modern-ui-language-for-xaml-platforms/
Reminds me of previous Steven had about UI langauge
Steven Kirk
@grokys
Jan 13 2017 00:20
hey yes i'm here
@RichiCoder1 oh man yes that looks almost exactly like the UI markup language i'd like
Richard Simpson
@RichiCoder1
Jan 13 2017 00:22
It apparently compiles down to Xaml, so in theory it wouldn't be hard to port to Avalonia once it's in a more stable state.
danwalmsley
@danwalmsley
Jan 13 2017 00:24
@grokys I'm trying to bind image source like this...
<Image Source={Binding ImageUri} />
then in viewmodel I'm returning string... like "resm: a.valid.uri.to.image.png"
the binding works. but I get
in trace messages
Binding Error, unable to convert 'resm:xxx.xxx.png' to IBitmap
so I'm thinking no default converter ?
how can I specify the converter to use?
I can see there is a BitmapTypeConverter in Avalonia already
Steven Kirk
@grokys
Jan 13 2017 00:29
yeah i don't think we handle URLs in the bitmap converter yet
we should
danwalmsley
@danwalmsley
Jan 13 2017 00:29
well in XAML I can code it manually like
Steven Kirk
@grokys
Jan 13 2017 00:29
we also need to work out our story for loading bitmaps from network urls
danwalmsley
@danwalmsley
Jan 13 2017 00:30
<Image source="resm:My.Namespace.assets.image.png" />
and it works
Steven Kirk
@grokys
Jan 13 2017 00:30
oh yeah
danwalmsley
@danwalmsley
Jan 13 2017 00:30
but if I bind the string
Steven Kirk
@grokys
Jan 13 2017 00:30
hmm that is strange
danwalmsley
@danwalmsley
Jan 13 2017 00:30
and pass from viewmodel it doesn't use the converter
Steven Kirk
@grokys
Jan 13 2017 00:30
hummm
danwalmsley
@danwalmsley
Jan 13 2017 00:30
its coming from embedded resource
Steven Kirk
@grokys
Jan 13 2017 00:31
i'm a bit busy right now and can't investigate but i can confirm that that should work
do default converters not get used from bindings? that might be the problem
if they don't, do they in WPF?
and if they don't in WPF, what should we do?
these are my questions
danwalmsley
@danwalmsley
Jan 13 2017 00:32
looks like in wpf this works without specifying a converter
Steven Kirk
@grokys
Jan 13 2017 00:32
yeah because bitmaps in WPF know about URLs and networking and everything
which didn't seem right to me
danwalmsley
@danwalmsley
Jan 13 2017 00:33
ok,
Steven Kirk
@grokys
Jan 13 2017 00:33
it might be that they should though
danwalmsley
@danwalmsley
Jan 13 2017 00:33
well I guess then I'm just after the syntax to specify the bitmaptype converter in avalonia already
but couldn't find an example already in avalonia of {Binding xxx, Converter={static MyTypeConverter}
i'll create an issue and we can look later ;)
Steven Kirk
@grokys
Jan 13 2017 00:35
hmm yeah, there isn't a decent syntax for that either
danwalmsley
@danwalmsley
Jan 13 2017 00:37
as a workaround maybe you could quickly point me to the code I need to load an embedded resource image to an IBitmap
?
danwalmsley
@danwalmsley
Jan 13 2017 00:51
@grokys don't worry found a workaround for now
and created an issue
;) thanks
danwalmsley
@danwalmsley
Jan 13 2017 01:20
@RichiCoder1 that new ammy ui language looks really nice!
we should definitely support that!
danwalmsley
@danwalmsley
Jan 13 2017 01:28
@ionoy hey! We would love to support Ammy here! :)
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 03:56
@grokys Ive been looking at DirectComposition and Windows.UI.Composition and been thinking about retained mode renderers. Would you be interesting in having APIsupport for retained mode renderers? These renderers have concepts of a visual tree and Visuals that we could hook into to basically have everything youre doing in the scenegraph branch be handled in the underlying framework.
We'd obviously still support immediate mode renderers, but this would allow us to get more performance for free (and it might make some interop scenarios easier)
Mihhail Maslakov
@ionoy
Jan 13 2017 06:29
hey, guys
ammy + avalonia would be neat
Steven Kirk
@grokys
Jan 13 2017 06:32
@ionoy yeah it'd be so cool. i'd love an ammy parser for avalonia, so it'd not have to even be transpiled to xaml
it looks so much less verbose than xaml
(i'm not a massive xaml fan)
@jkoritzinsky yeah i thought about that - it suits our model pretty well i think. the reason i didn't go down that path was that it's not x-plat
but a new directcomposition renderer could easily be created that uses it
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 06:34
I can investigate what other retaining mode renderers exist on other platforms. It isn't just a windows thing (though there are two APIs on windows)
I'll keep you posted 😊
Steven Kirk
@grokys
Jan 13 2017 06:34
yeah, please do!
sorry i've not had chance to check out your PR yet - i'm in SF for work so not had so much free time. hopefully at the weekend though
Mihhail Maslakov
@ionoy
Jan 13 2017 06:35
i'm thinking of creating some kind of IL. this way we could make ammy/avalon binding as a separate component.
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 06:36
Totally fine!
Steven Kirk
@grokys
Jan 13 2017 06:36
@ionoy well our XAML support is intended to be completely separate from the rest of the framework so in theory it could be plugged in reasonably easily
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 06:36
Btw you'll find this funny. The second result on google for "Linux retained mode 2d renderer" gets me #61.
Kinda creepy if you ask me
Mihhail Maslakov
@ionoy
Jan 13 2017 06:37
@grokys alright. how do you load components in runtime?
Steven Kirk
@grokys
Jan 13 2017 06:38
haha that reminds me of work when i search something related to a VS plugin and the only mention you can find is the project i work on ;)
@ionoy we use OmniXaml which handles that stuff for us
but all our xaml stuff is limited to this single assembly: https://github.com/AvaloniaUI/Avalonia/tree/master/src/Markup/Avalonia.Markup.Xaml
Mihhail Maslakov
@ionoy
Jan 13 2017 06:41
@grokys do you have a sample project? i want to see what's happening on initalization
Mihhail Maslakov
@ionoy
Jan 13 2017 07:07
@grokys what is paml? is it you version of baml?
Steven Kirk
@grokys
Jan 13 2017 07:08
just a different file extension for xaml files
Mihhail Maslakov
@ionoy
Jan 13 2017 07:10
would something break if you replaceAvaloniaLocator.Current in runtime?
the thing is, we need to load xaml from memory when updating in runtime
or maybe it would be better to just add a abstraction layer that would do that :)
danwalmsley
@danwalmsley
Jan 13 2017 08:34
@ionoy hey great to see you here
We currently parse the xaml at run time
So no baml
In avalonia
All the XAML files are embedded resources
So I guess yes we are pretty well set up to load at runtime
Mihhail Maslakov
@ionoy
Jan 13 2017 08:37
yeah, it all looks similar to how WPF does things, so transition shouldn't be difficult
i mean, absence of baml makes it even easier
danwalmsley
@danwalmsley
Jan 13 2017 10:00
@ionoy I guess its possible to mix XAML and ammy within the same project?
Mihhail Maslakov
@ionoy
Jan 13 2017 10:03
yeah, sure
Nikita Tsukanov
@kekekeks
Jan 13 2017 10:35

Interesting project that just popped up on my reddit feed: http://www.ammyui.com/2017/01/04/ammy-modern-ui-language-for-xaml-platforms/

I think I heard about that guy from Nitra devs a year ago. The main issue with the project is that it is commercial.

BTW, it uses Nitra for compilation
yeah, it's that guy
So it won't be available for free
Oh, @ionoy you are here. Heard about you from Vlad at CLRium
Nikita Tsukanov
@kekekeks
Jan 13 2017 10:40
Is your project still free for non-commercial use only?
Nikita Tsukanov
@kekekeks
Jan 13 2017 10:47
Regarding loading XAML from memory
You don't need to reconfigure locator for that
Just use AvaloniaXamlLoader directly
Mihhail Maslakov
@ionoy
Jan 13 2017 10:49
yeah, Ammy is free for non-commercial only. at least at the moment
Nikita Tsukanov
@kekekeks
Jan 13 2017 10:49
If you are planning to use our previewer, just grab Avalonia.Designer.exe
From the main repository
Just make sure to rename it
To avoid conflicts with our current vs extension
Mihhail Maslakov
@ionoy
Jan 13 2017 10:50
alright, i'll take a look at it
current vs extension?
Nikita Tsukanov
@kekekeks
Jan 13 2017 10:51
BTW, extension itself is GPL-licensed, so you can't copy-paste code from there. You probably don't need it, since you are using Nitra anyway
Yep, previewer is used by AvaloniaVS
So you need to use different assembly name to avoid conflicts
Oh, extension was relicened
so nevermind, feel free to copy-paste code
Mihhail Maslakov
@ionoy
Jan 13 2017 10:52
well, my idea was to work with Avalon the same way i do with WPF
sit side by side, and compile everything that WPF (Avalon) needs
Nikita Tsukanov
@kekekeks
Jan 13 2017 10:52
That's probably a bad idea, since our xaml syntax is far from being stable
Mihhail Maslakov
@ionoy
Jan 13 2017 10:53
i was thinking of creating some kind of Ammy IL. this way you could maintain your own AmmyIL2Avalon project as a plugin to Ammy
Nikita Tsukanov
@kekekeks
Jan 13 2017 10:54
There would be issues with previewer through
Mihhail Maslakov
@ionoy
Jan 13 2017 10:54
why? in the end it would be still using same avalon "xaml" files
Nikita Tsukanov
@kekekeks
Jan 13 2017 10:55
You see, our previewer spawns a separate process
That loads everything
And basically starts the whole application
Without triggering app lifetime events
And then just loads XAML and embeds loaded control to visual studio window
If you are generating IL, you need to either load your compilation toolchain directly to that proccess
Or supply compiled assemblies
And those won't be ever unloaded, because that's how AppDomains work in .NET
Mihhail Maslakov
@ionoy
Jan 13 2017 10:58
hm, ok. we can start without a designer anyway
with live update it's not really important
Nikita Tsukanov
@kekekeks
Jan 13 2017 10:58
BTW, what's the current status of Nitra? I was planning to use it for XAML parser, but suddenly run completely out of free time
Mihhail Maslakov
@ionoy
Jan 13 2017 10:58
and when everything works as expected we can look at designer
Nitra was in pretty good shape for a long time now
Vlad took a long time to create VS Extension generator, but otherwise you can use Nitra without fear
especially since you can always contact Vlad directly (as I did. sometimes a lot :D )
Nikita Tsukanov
@kekekeks
Jan 13 2017 11:01
Yeah, that's probably what stops english speakers from using Nitra. No proper support from Vlad)
Without his ultimate help it would be hard to figure out how nitra works
So, regarding for loading XAML from memory
That's also probably a bad idea
Since we rely on the "base url " of xaml file
To look up resources like images
So it would probably better to have some kind of MSBuild task
That converts ammy files to xaml and save them as embedded resources
So our XAML loader will find them where expected
Mihhail Maslakov
@ionoy
Jan 13 2017 11:05
this is being done with WPF right now, but to update UI in runtime you would still need loading from memory. good thing is - base uri can be cached and since resource files shouldn't be moved while debugging it's going to work fine
Nikita Tsukanov
@kekekeks
Jan 13 2017 11:09
Wait, are you reloading the whole XAML file for UI updates at runtime?
Oh, you are using it as a live preview
Mihhail Maslakov
@ionoy
Jan 13 2017 11:11
Ammy can actually update UI in runtime, not just as a preview
watch the Demonstration video at the bottom of the page
Nikita Tsukanov
@kekekeks
Jan 13 2017 11:13
Well, updating the whole xaml in released app may lead to all sorts of awful things
Mihhail Maslakov
@ionoy
Jan 13 2017 11:14
it's not enabled in non-debug builds or when NO_AMMY_UPDATE compilation symbol is defined
danwalmsley
@danwalmsley
Jan 13 2017 12:29
I think the live update is brilliant idea, it would beat our previewer hands down!
Nikita Tsukanov
@kekekeks
Jan 13 2017 14:14
Well, that's actually would be easy to implement
danwalmsley
@danwalmsley
Jan 13 2017 14:16
like with Avalon studio, I could run it
navigate to project, load stuff
then edit the XAML and see what happens
combine that with the visual tree dev tool (F12)
and you have pretty powerful development tool
Nikita Tsukanov
@kekekeks
Jan 13 2017 14:21
You see, it's quite easy to implement file watcher
That will track changes in source xaml files
So it will provide change events if supported
well, there is one problem that I see through
If you are updating existing window
Default values won't be restored after xaml reload
I wonder how @ionoy deals with that
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 15:30
@grokys I found retained mode APIs for each platform except Android. Each one interfaces with an underlying 2d rendering API to render on each visual/layer:
Win32: DirectComposition -> Direct3D/2D
UWP: Windows.UI.Composition -> Direct3D/2D
iOS/Android: CoreAnimation -> Quartz CoreGraphics
Linux: Clutter -> Cairo
Steven Kirk
@grokys
Jan 13 2017 15:34
ok, would be interesting to see how well our concepts map across
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 15:38
I'll start working on a doc that maps everything at the level of a Visual in Avalonia to each underlying platform so we can see how well the mapping holds up.
Nikita Tsukanov
@kekekeks
Jan 13 2017 16:34
Are you planning to drop Skia in favor of using native toolkits?
BTW, DirectComposition isn't available in Windows 7
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 16:40
Are we still supporting Windows 7?
The changes in the scenegraph branch that I spun out to #850 need to call some functions only available in Direct3D11.1 and I'm not sure if they're supported in the Windows 7 Platfrom Update.
Nikita Tsukanov
@kekekeks
Jan 13 2017 16:42
Well, it still has something like 30% of desktop user base
And it won't go lower anytime soon
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 16:43
I was thinking we continue to support immediate mode APIs but also add support for retained mode APIs
Nikita Tsukanov
@kekekeks
Jan 13 2017 16:43
Or create our own compositor
As a fallback
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 16:43
The work in the scenegraph branch does enough of that to be our fallback.
At least for the features we're using
Nikita Tsukanov
@kekekeks
Jan 13 2017 16:44
BTW, Android doesn't have proper API for retained mode like iOS does with CALayer
The best you can do is to map each visual to SurfaceView
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 16:44
That's what I thought but I wasn't positive.
Isn't Android support broken right now anyway?
Nikita Tsukanov
@kekekeks
Jan 13 2017 16:44
But that's far from optimal from performance point of view
Yeah, but we kinda want to support it in the future
Or so I thought
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 16:45
I agree
Nikita Tsukanov
@kekekeks
Jan 13 2017 16:45
BTW, if you map each visual to a CALayer
You'll run out of memory before showing a hello word
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 16:46
You sure?
Nikita Tsukanov
@kekekeks
Jan 13 2017 16:46
Well, you see, people tend to create complex hierarchies
Of those visuals
If you look at our visual tree in control catalog
You'll realize that there are hundreds of them
We just can't afford to keep a surface for each visual
Most of them don't even draw anything
Have you read that Chrome rendering architecture document?
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 16:49
Aren't we already adding that same memory pressure with the new lower level scene graph with the scene nodes anyway?
Nikita Tsukanov
@kekekeks
Jan 13 2017 16:49
They have very specific conditions for introducing a new GPU-composited layer
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 16:49
I haven't read the chrome rendering architecture document. Can you point me to it?
Nikita Tsukanov
@kekekeks
Jan 13 2017 16:49
Well, the thing is that CALayer have a offscreen bitmap associated with it
Which retains in memory even when you aren't drawing anything right now
In our current implementation visuals are very cheap
Since Visual is just an AvaloniaObject with some properties and child list
So it occupies a dozen of kilobytes of memory at most
But if you have an associated CALayer
It will need to store bitmap data somewhere
Which is width*height*4 bytes
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 16:53
One thing we could do is try to consolidate some of our visuals together into one layer.
It looks like that's what chrome does.
Nikita Tsukanov
@kekekeks
Jan 13 2017 16:53
Yeah, exactly
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 16:54
We just need to figure out what huristics we want to use given our architecture.
I'm all for doing that.
Nikita Tsukanov
@kekekeks
Jan 13 2017 16:54
Chrome allows you to have hints in HTML markup, I think
That ask Chrome for introducing a layer
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 16:55
One of the places I was thinking was splitting between Panels and their children.
And trying to consolidate a TemplatedControl with its control template.
Nikita Tsukanov
@kekekeks
Jan 13 2017 16:56
Well, the problem is that you need to analyze the scene as the whole
With simple control-level heuristics it will run out of control very quickly
What else gets its own layer? Chrome’s heuristics here have evolved over time and continue to, but currently any of the following trigger layer creation:
  • 3D or perspective transform CSS properties
  • <video> elements using accelerated video decoding
  • <canvas> elements with a 3D (WebGL) context or accelerated 2D context
  • Composited plugins (i.e. Flash)
  • Elements with CSS animation for their opacity or using an animated transform
  • Elements with accelerated CSS filters
  • Element has a descendant that has a compositing layer (in other words if the element has a child element that’s in its own layer)
  • Element has a sibling with a lower z-index which has a compositing layer (in other words the it’s rendered on top of a composited layer)
Something like that
You can enable Layers tab in dev tools
And look at a typical web page
To see where Chrome uses layers
BTW, I think they don't use hardware acceleration for rendering, only for compositing and webgl
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:03
For example, this page https://github.com/AvaloniaUI/Avalonia is represented by 1 (one) layer
And hover your mouse over one of the boxes
You'll see that once you do that Chrome creates a couple of layers
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:07
Ok..
Do you think it's worth the effort to add support for these APIs for the performance benefit we'd get?
If the scenegraph branch issues are fixed and its merged in.
See? Most of the time there are no layers on the page
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:10
I see.
So do you think it's worth the energy and time to code in support or not?
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:10
Well, I'm afraid, as it is, we'll probably lose performance
Because layer composition isn't free
Especially if compositor is GPU and rendering is in CPU
And on mobile platforms we want it to be done by CPU
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:11
Why on CPU on mobile?
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:11
Well, you see, there are no proper way of rendering fonts on GPU
Skia prerenders fonts using CPU and uses font atlas as a texture
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:12
Oh, that's a good point.
Didn't know that
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:12
And if you rotate text
It has to render each glyph in each angle that you use
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:12
Also one more question: Are we aiming to eventually drop support for all rendering subsystems outside of Skia?
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:12
Because that's how smoothing works
We've got serious performance degradation on iOS
With our previous skia backend
When I've tried to render animated rotated text on GPU
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:13
Since (at least sometimes) that seems like the direction you're talking.
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:13
Basically the app just got frozen every 2 seconds
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:13
Damn. Does CoreText have the same problem?
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:14
You see, iOS doesn't use GPU acceleration most of the time
Sure, CoreAnimation is GPU-accelerated
But it's generally used for CALayer composition
May be they've changed something in the recent version
But last time I've checked it was the case
iOS also creates a CALayer for each view
It works fine there, because iOS's native UI toolkit is similar to Windows Forms
Controls just occupy some space on the screen and manually re-render themselves when asked
So there is no much overhead from having a backing bitmap for each of them
Since apps are fullscreen and you need an active CALayer only for onscreen controls
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:19
Unfortunately, because of our complex visual trees we can't do that
So, in my opinion for significant performance gains 2 things could be done:
1) move rendering to a separate thread by recording rendering commands and replaying them
So main thread won't have to spend CPU time on rasterizing and/or waiting for GPU to respond
2) track changes in visual graph, so only changed parts would be updated
And only when that's done, we need to think about layers
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:22
Ok.
Another idea building off of the scenegraph branch (which implements option 2)
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:22
Because they are needed to optimize cases, when you need to update the whole scene no matter what
For example if you have some complex UI and semi-transparent overlay above it
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:23
Associate a CALayer/SpriteVisual/DCompositionVisual etc with a RenderLayer object
That seem to make sense?
At least to a point?
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:23
Well, you see, we probably won't need to use native compositor for that
Since both Direct2D and Skia can be used for composition
And we'll avoid tons of work and bugs for very little performance price
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:24
Ok.
How about we put this discussion on hold until we see what the new perf characteristics of Avalonia are after the scenegraph branch is merged? Since we need to see how it all works after the merge happens to make an informed decision.
I do have some other questions for you about interop if you have some time:
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:26
Well, unfortunately I have no idea what you were doing in that branch
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:26
I'm not working on the scenegraph branch. Thats @grokys
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:26
Same here
I'm kinda off track nowadays
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:27
I just pulled out the Direct3D11 initialization into a seperate branch since I need that for AvaloniaAV (I'm working on implementing streaming video controls)
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:28
So you want to have streaming video inside a scene graph instead of just embedding a native view
That would have a negative performance impact
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:29
Yep. I'm planning on streaming from a MediaFoundation MediaEngine into a surface which is shared with a d2d1bitmap
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:29
For example, if you embed a flash plugin inside a web page
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:29
I've only started work on windows so far
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:30
You have to options:
1) embed it as a native widget, in this case it works fast, but you can't place anything on top of it
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:30
I wanted to support the full spectrum of Avalonia's transforms on platforms that I can.
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:30
2) offload the image to browser's rendering pipeline
And it's generally recommended to use the first way
Because of that performance impact
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:31
The MediaEngine class in MediaFoundation already offloads to the browser's rendering pipeline to render to the surface.
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:32
Yeah, you can do the same on Linux with gstreamer
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:32
Then I was going to share the surface with a d2d1 bitmap (at least for Direct2D support) which I would then render.
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:32
But the point is that you are introducing an extra composition step
And when you are using a native widget, it will create an actual native window
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:33
I have to introduce another step no matter what unless we have composition built in. I don't really have much of another choice.
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:33
That can also utilize some DWM magic
to render directly on the screen
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:33
I'm trying to avoid using the "render to HWND" options. I was thinking about using DirectComposition, but to do that we need to add the composition engine support.
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:33
Even bypassing the system compositor
Well, you have two options
1) have access to rich features
2) have good performance
Have you noticed that sometimes you can't take a screenshot from the video player?
That's the DWM magic I'm talking about
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:35
That's never been something I've ever actually tried to do.
I know that DirectComposition uses some DWM magic internally to improve performance.
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:38
Yep, but to do that, you basically need to have everything inside a separate layer (which won't work because of enormous memory requirements)
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:38
My plan was to fall back to embedding a native control only when I need to (like on Windows 7 since I was having issues creating a custom EVR presenter)
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:38
or, alternatively, draw everything beneath and above your video in separate layers
And I have no idea how to properly do that
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:39
This is actually one of the places that Chrome splits different render layers
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:39
Well, I recommend to leave an option to force that native control
If one wants performance
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:39
<video> elements using accelerated video decoding
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:39
Yeah
Well, I might be overthinking and these optimizations aren't needed anymore on the modern hardware
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:40
I'll test things out and see how they end up working.
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:40
For now I recommend to just have a shared Direct2D/video-decoder texture
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:41
That's the plan
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:41
That's the simplest way, I think
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:41
My general plan is the following:
MediaEngine is initialized with a dxgi surface
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:42
But I'm afraid, that without proper layer infrastructure that would be slow
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:42
All frames the engine renders go to that surface
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:42
Since you need to trigger whole window update for each frame
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:43
There might be lag because of our current "repaint everything" design, but once Steven finishes up the lower level scene graph we shouldn't have quite as many issues.
Nikita Tsukanov
@kekekeks
Jan 13 2017 17:43
yeah
Jeremy Koritzinsky
@jkoritzinsky
Jan 13 2017 17:43
But at that point it's not the architecture of AvaloinaAV, but of core Avalonia.
I can say that I've already seen the lag when I made an Avaloina backend for OxyPlot.
Steven Kirk
@grokys
Jan 13 2017 18:14
hey guys
so yeah the scenegraph branch on the UI thread renders everything to an immutable low-level scenegraph of objects
then in a seprate render thread it renders those
a new layer is currently created only if a control has opacity
but i want to allow a control to be tagged as starting a new layer
when a control is invalidated, a copy of the immutable scene graph is made, using the same nodes and only the invalidated node is replaced
Steven Kirk
@grokys
Jan 13 2017 18:22
So basically it does what @kekekeks was suggesting
the WIP PR is here: AvaloniaUI/Avalonia#827
it pretty much works except for the missing things listed in the PR
and it also needs documentation
Weston
@ronnyek
Jan 13 2017 21:13
didnt someone have a gui designer for avalonia?