These are chat archives for bjz/gfx-rs

4th
Jul 2014
Corey Richardson
@cmr
Jul 04 2014 00:02
Anyone mind me porting to cargo?
Brendan Zabarauskas
@brendanzab
Jul 04 2014 00:27
no, not at all
Corey Richardson
@cmr
Jul 04 2014 00:27
coolio
I'm going to spend my spare time working on channel perf.
I'm not sure if there are any wins
but I at least want to benchmark against go etc
Brendan Zabarauskas
@brendanzab
Jul 04 2014 00:27
how do you feel about rust-empty? I dunno how it really works
Corey Richardson
@cmr
Jul 04 2014 00:28
rust-empty is fine
it's just a makefile with some boilerplate
don't really need it though
Brendan Zabarauskas
@brendanzab
Jul 04 2014 00:28
mmk
Corey Richardson
@cmr
Jul 04 2014 00:28
I aliased cargo to c
brendanzab @bjz would love to be free of make
Corey Richardson
@cmr
Jul 04 2014 00:28
it's a lot nicer that way :P
c build
etc
Brendan Zabarauskas
@brendanzab
Jul 04 2014 00:29
I wonder if we could make a build system syntax extension :P
(probably would result in something horrific)
(insert relevant xkcd here)
Dzmitry Malyshau
@kvark
Jul 04 2014 01:02
@bjz I kinda like make... dunno why you are so against it
even when cargo is integrated, we could still automate some tasks with Makefile
Dzmitry Malyshau
@kvark
Jul 04 2014 02:03
@csherratt BTW, if we are profiling it, we need to build it with -O
Coraline Sherratt
@removed~csherratt
Jul 04 2014 02:03
yeah, one of the concerns I have with using cargo
Brendan Zabarauskas
@brendanzab
Jul 04 2014 02:03
Almost killed the render task!
removed~csherratt @csherratt hit the random rustc bug lottery
Dzmitry Malyshau
@kvark
Jul 04 2014 02:05
@bjz I hope you implement it as an alternative, and share as much code as possible
Brendan Zabarauskas
@brendanzab
Jul 04 2014 02:17
@kvark I was just turning everything into methods on Server
same API exposed
just without the task in the middle
@kvark would you prefer to keep the messaging API too?
I can keep an update method and just dispatch out to those methods
wait - no not an update method
I mean a method that takes a render::Request and preforms the corresponding method
Dzmitry Malyshau
@kvark
Jul 04 2014 02:21
@bjz It'd be nice to provide both client versions with the same interface - with task and without
@bjz you can start a PR now so that we look into iit
Brendan Zabarauskas
@brendanzab
Jul 04 2014 02:22
sure thing
Dzmitry Malyshau
@kvark
Jul 04 2014 02:53
#57 is closed? I was hoping for it to merge
Brendan Zabarauskas
@brendanzab
Jul 04 2014 02:54
@csherratt ?
@kvark argh runtime errors are the best
Dzmitry Malyshau
@kvark
Jul 04 2014 02:56
@bjz what have you got?
Brendan Zabarauskas
@brendanzab
Jul 04 2014 02:56
bit hard to explian - I will figure it out
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:01
\o/
Dzmitry Malyshau
@kvark
Jul 04 2014 03:06
we have a winner? :)
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:07
crap
pushed to master >_<
quick, revert!
Dzmitry Malyshau
@kvark
Jul 04 2014 03:08
oh
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:09
I just killed some kittens
@kvark #64
kvark @kvark is looking
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:11
puts the responsibility of making a separate task with messages in the hands of the client
we might want to make it easier in the future, but for now I find it much easier to wrap my head around
Dzmitry Malyshau
@kvark
Jul 04 2014 03:12
so you want to kill it then
I see...
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:12
feel free to disagree though
Dzmitry Malyshau
@kvark
Jul 04 2014 03:12
I'm for simplicity, at least until we find the limits of it
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:12
I didn't see much point to it right now - but you might have future plans
Dzmitry Malyshau
@kvark
Jul 04 2014 03:13
nah, no plans so far
so much code deleted...
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:13
I know!
:D
Dzmitry Malyshau
@kvark
Jul 04 2014 03:14

Perfection is attained, not when there is nothing left to add, but when there is nothing left to take away.

-Antoine de Saint-Exupery

Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:14
the one issue I had was the Renderer::common_array_buffer
I have to lazily load it, otherwise it would block
Dzmitry Malyshau
@kvark
Jul 04 2014 03:15
hmm
why would it block?
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:16
because Renderer::new is not called on a separate task now
Dzmitry Malyshau
@kvark
Jul 04 2014 03:16
hmm, well, we are going to have more than just a common VAO, so we need some kind of initialization routine
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:17
We can make that prettier in the future though - I think the gains in simplicity are bigger the other way
ahh ok
hmm
kvark @kvark is gonna need some heavy merging after that PR
Dzmitry Malyshau
@kvark
Jul 04 2014 03:18
@bjz I'm glad you took that task. My hand would be too resistant to remove so much code, especially if it was written by me :)
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:18
well the initial structure was mine, so its my baby to kill
Dzmitry Malyshau
@kvark
Jul 04 2014 03:19
true :clap:
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:19
I think when I initially laid it out my intention was to have higher level stuff on the render task - like shadows and culling and stuff, but I think we have refocused on the original goals laid out in the README
Dzmitry Malyshau
@kvark
Jul 04 2014 03:20
@bjz I see. It does sound a bit weird though (shadows and culling for something that is just a lightweight graphics device manager?)
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:21
agreed - I think I forgot my own readme :P
I am much happier with this - less monolithic
Dzmitry Malyshau
@kvark
Jul 04 2014 03:21
I'm ready to merge. Do you want more eyes on this before we do it?
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:22
@photex @cmr @csherratt #64
lets wait a little longer
sorry - I know this is underneath your current stuff
Dzmitry Malyshau
@kvark
Jul 04 2014 03:25
@bjz how do you know what I'm doing? :)
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:25
@kvark is gonna need some heavy merging after that PR
;)
Dzmitry Malyshau
@kvark
Jul 04 2014 03:25
I'm working on FBO support
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:26
ooh
so, still haven't got my head around FBOs. what do they do again, and why are they important?
Corey Richardson
@cmr
Jul 04 2014 03:27
they let you render into a texture
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:27
why are they called 'frame buffer objects'?
Dzmitry Malyshau
@kvark
Jul 04 2014 03:28
@bjz they encapsulate state of several hard-coded plane attachments
as in color0, color1, ..., depth, stencil
Corey Richardson
@cmr
Jul 04 2014 03:28
Everything under the sun is an object, and the thing you render into is called a framebuffer -- framebuffers themselves can be the source of a texture
under the sun in opengl..
Dzmitry Malyshau
@kvark
Jul 04 2014 03:28
Pretty much the same as VAO for vertex attributes, FBO is for target surfaces
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:29
is this common to most APIs?
Dzmitry Malyshau
@kvark
Jul 04 2014 03:29
@bjz rendering to side rendertargets is. My plan is to hide FBO internals from the user
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:29
ooh, nice
'rendertarget' seems a lot more comprehensible
Dzmitry Malyshau
@kvark
Jul 04 2014 03:30
@bjz the first approximation is - keep a common FBO and change it according to what targets user requests
pretty much the same as VAO is doing, except that for FBO we actually need to unbind stuff that is not needed
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:32
awesome!
this is exciting
you are amazing @kvark
sorry for being such a noob :P
Dzmitry Malyshau
@kvark
Jul 04 2014 03:33
:smile: sorry if references to acronyms are confusing. I can describe it in detail if you want, but there are tons of materials out there to read
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:34
oh no, that is fine
@cmr what do you think of #64?
Corey Richardson
@cmr
Jul 04 2014 03:36
it seems like throwing out the render task entirely isn't necessary to make not using the render task possible.
but I'm fine with it
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:37
do you think it could constrain us in the future?
Corey Richardson
@cmr
Jul 04 2014 03:37
the render task? sure. it should be possible to use it without a render task, but I think tossing out the entire feature for that is extra work
Dzmitry Malyshau
@kvark
Jul 04 2014 03:39
@cmr I guess the problem is that we don't know (yet) what the render task is for... As @bjz said, it was intended for high-level requests (shadows, culling), but the way we got it working was far from it. There was simply no benefit in it.
We need to figure out how gfx can be used on the highest level. A layer between gfx and the application needs to be in its own crate, since it will have tons of dependencies (math lib, at least)
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:42
@kvark is that similar to what @csherratt is doing?
Dzmitry Malyshau
@kvark
Jul 04 2014 03:42
I was hoping that @csherratt would guide us in regards to high-level requirements, since his snowmew is amazingly high-level
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:43
yeah
I mean, we will have our own requirements, re. procedural generation
Corey Richardson
@cmr
Jul 04 2014 03:43
I'm convinced
kill it with fire
Dzmitry Malyshau
@kvark
Jul 04 2014 03:44
it's gonna live forever in our hearts and git histories...
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:44
^_^
Dzmitry Malyshau
@kvark
Jul 04 2014 03:44
I'm not good with high-level stuff. Implementations that I happened to deal with always stunk with being too limiting. I'm hoping to learn a lot from this project.
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:49
yeah I am glad to have your experience.
urr
Corey Richardson
@cmr
Jul 04 2014 03:49
^
Dzmitry Malyshau
@kvark
Jul 04 2014 03:50
:laughing:
Brendan Zabarauskas
@brendanzab
Jul 04 2014 03:50
hopefully we will start to have something in a decent enough state to show off to glenw at some point
and h3r3tic
they might be able to give us some pointers too
Coraline Sherratt
@removed~csherratt
Jul 04 2014 03:58
@bjz I love that dog
Brendan Zabarauskas
@brendanzab
Jul 04 2014 04:00

Don't forget his pal:

ehehe

@csherratt good to merge?
Coraline Sherratt
@removed~csherratt
Jul 04 2014 04:00
Yeah it looks good.
Brendan Zabarauskas
@brendanzab
Jul 04 2014 04:00
@kvark reading #65 now
@csherratt you can do the honors
Dzmitry Malyshau
@kvark
Jul 04 2014 04:01
@bjz I'm gonna leave it for now, since I don't have time to merge. Will resolve tomorrow, you can have a look meanwhile.
Brendan Zabarauskas
@brendanzab
Jul 04 2014 04:01
@kvark np
Coraline Sherratt
@removed~csherratt
Jul 04 2014 04:02
#64 This change is actually good for a number of reasons. If you are doing lots of procedural generation you will probably want direct buffer access since it saves a buffering step. Removing a layer of server indirection can help with that goal.
I imagine mmap in gfx is going to be tricky.
Brendan Zabarauskas
@brendanzab
Jul 04 2014 04:03
mmap?
Dzmitry Malyshau
@kvark
Jul 04 2014 04:07
@csherratt buffer map? I'd rather not provide it at all.
Brendan Zabarauskas
@brendanzab
Jul 04 2014 04:08
@kvark why would you not provide it?
Dzmitry Malyshau
@kvark
Jul 04 2014 04:11
@bjz because it's not GPU-idiomatic. GPUs are most effective as a one-way street. In very rare examples you may want to read something back, preferably nothing at all. Without reading, buffer map is replaced by buffer update, which we already have in its very basic form.
Brendan Zabarauskas
@brendanzab
Jul 04 2014 04:12
makes sense
I think that is a reasonable goal for gfx-rs - to be a one-way street
Dzmitry Malyshau
@kvark
Jul 04 2014 04:13
exactly, that's amplified by the fact our device lives in it's own thread
Coraline Sherratt
@removed~csherratt
Jul 04 2014 04:18
I worry about making to many copies of data.
Dzmitry Malyshau
@kvark
Jul 04 2014 04:19
@csherratt copies are better than GPU stalls ;) what do you want to map?
Coraline Sherratt
@removed~csherratt
Jul 04 2014 04:20
the worst case a buffer could be copied 3 times. User -> GFX, GFX -> OpenGL Driver -> Driver to GPU memory. That would probably be in the case of something like buffersubdata which we actually don't support right now.
Dzmitry Malyshau
@kvark
Jul 04 2014 04:21
how much data are you talking about?
Coraline Sherratt
@removed~csherratt
Jul 04 2014 04:24
I don't know, that's the problem. Snowmew does not really do to much dynamic updating of vertex data right now. But if it is initial loading the buffers tend to be pretty big, a vertex buffer could be upwards of 100MB for a large scene.
Dzmitry Malyshau
@kvark
Jul 04 2014 04:25
@csherratt so you think mapping it and reading into the mapped memory would be less resource consuming?
Coraline Sherratt
@removed~csherratt
Jul 04 2014 04:27
It should be, you are saving a memory copy and allocation.
but the case I worry for more is actually @bjz's I think for dynamically creating assets having direct access to the vertex buffers to write to is useful.
If the terrain is being generated every few frames based on where the player is standing. That's quite a bit of data to buffer, having the generator functions write directly into gpu memory seems useful to me.
but you are right, in a pipeline architecture like gfx it is going to be tricky to know when it is safe to modify a buffer, and when you can do so without stalling the pipeline.
Dzmitry Malyshau
@kvark
Jul 04 2014 04:47
@bjz ok, #65 should be ready now. time to sleep... @csherratt sorry, I'll get back to that discussion tomorrow
BTW, we are getting tons of warnings now with the new Rust (see Travis log)
Coraline Sherratt
@removed~csherratt
Jul 04 2014 04:49
DuplexStream is being removed?
Brendan Zabarauskas
@brendanzab
Jul 04 2014 04:51
it is
I forgot to mention it before (I was in the meeting when it was discussed!)
Dzmitry Malyshau
@kvark
Jul 04 2014 12:34
@cmr glad to have you on board!
Dzmitry Malyshau
@kvark
Jul 04 2014 13:34

@csherratt Interesting discussion here: http://www.gamedev.net/topic/658256-using-vbos-for-dynamic-geometry/ . In particular, that message:

If vertex data changes every frame (i.e. you're streaming), you have four options:

  1. one buffer, three times the size, persistently mapped (GL_MAP_WRITE_BIT|GL_MAP_PERSISTENT_BIT|GL_MAP_COHERENT_BIT)
  2. one buffer, three times the size, mapped with GL_MAP_UNSYNCHRONIZED
  3. one buffer, three times the size, using glBufferSubData
  4. one buffer, invalidated using glBufferData(..., 0), and replaced by another one (again, glBufferData)

(1) is the fastest, but requires ARB_buffer_storage and setting fences, a third of the buffer you write to, the second third is in transfer, the last third is being drawn from
(2) avoids a CPU-GPU sync, but causes a client-server thread sync, requires GL 3.x and setting fences
(3) same as (2) but surprisingly faster on most IHVs, no fence needed
(4) compatibility mode, should probably even work with GL 1.5/2.0, somewhat slower but not that bad

If that's true, then basically there is no memory gain from using buffer map. Data still needs to be copied multiple times to not block GPU, so even reducing one copy (despite the fact that the driver may do more copies to compensate for that, and I'm sure it does for (4)) on our side doesn't help it too much. Notice how (3) (glBufferSubData) considered the best supported and faster than anything else. I think we can go with it safely.
There is also a decent article in OpenGL insights (that I'm sure I have read, just need to refresh my memory :))
Dzmitry Malyshau
@kvark
Jul 04 2014 15:28
@csherratt @bjz The article got the best overall results in mapping with GL_MAP_UNSYNCHRONIZED, and I have all means to believe in the quality of work these guys did. I suggest us to just passively keep looking for ways to use map but don't block on it, since the classical glBuffer(Sub)Data is only slower by about 15%, and it's actually faster on integrated chips. I don't believe this is worth re-architecturing anything now, looks more like a premature optimization effort ;)
Coraline Sherratt
@removed~csherratt
Jul 04 2014 16:31
@kvark ok. I have ideas for how it could be implemented but generally I agree with you that feature wise these are not the most pressing issues.
Brendan Zabarauskas
@brendanzab
Jul 04 2014 16:55
Sounds good
:+1:
Definitely something to keep in mind though.
Dzmitry Malyshau
@kvark
Jul 04 2014 18:05
This message was deleted
@bjz why not getting PRs reviewed?
Brendan Zabarauskas
@brendanzab
Jul 04 2014 18:06
Yeah that was naughty
brendanzab @bjz heads to the naughty corner
Brendan Zabarauskas
@brendanzab
Jul 04 2014 18:10
@kvark comment on #68 if you want me to change stuff - sorry
Dzmitry Malyshau
@kvark
Jul 04 2014 18:12
np. If you are concerned about getting something in quicky, you can at least ask here if anyone is available for review. I am, most of the time.
Brendan Zabarauskas
@brendanzab
Jul 04 2014 18:28
yeah, np
what are you up to now?
I might be heading out to brunch soon
Dzmitry Malyshau
@kvark
Jul 04 2014 18:29
@bjz thinking about shader parameters, really have no plans for anything in particular. Will probably just add textures support if find no solution for the shader params
Brendan Zabarauskas
@brendanzab
Jul 04 2014 18:30
sounds good
I'm guessing @photex might be out doing stuff with family today
excited about the handle stuff though, when he gets time
Dzmitry Malyshau
@kvark
Jul 04 2014 18:31
thanks to our simplified architecture, working on gfx is going to be much more lightweight now
have you guys ported your game on gfx-rs yet?
Brendan Zabarauskas
@brendanzab
Jul 04 2014 18:32
not yet!
ok, heading out!
Dzmitry Malyshau
@kvark
Jul 04 2014 18:32
cheers!
Corey Richardson
@cmr
Jul 04 2014 18:55
I've been looking a lot at deferred techniques
Also if you go to http://next.gitter.im/ you can join the beta
it has "message bursting" now, doesn't repeat username/avatar for adjacent messages from same person
Dzmitry Malyshau
@kvark
Jul 04 2014 18:58
@photex wow nice! gitter keeps surprising me in a good way
Corey Richardson
@cmr
Jul 04 2014 18:58
@bjz you keep closing and opening issues :fax:
Dzmitry Malyshau
@kvark
Jul 04 2014 19:00
This is muuuuch better now. More text visible here. @cmr anything in particular you want to try out (in regards to deferred), or just looking around?
Corey Richardson
@cmr
Jul 04 2014 19:01
I'm just looking around,
there's some good stuff in GPU Gems
Dzmitry Malyshau
@kvark
Jul 04 2014 19:02
oh that... that's pretty old. It does experiment with various Gbuffer formats, but overall follows a very basic pipeline.
I'd like to adopt clustered forward or forward+ for GL-3.3 hardware somehow....
Somehow, I dislike the idea of storing variable-length lists on the GPU.
Corey Richardson
@cmr
Jul 04 2014 19:06
Yeah I've always found that somewhat odd, but do you know a technique to avoid it in lighting?
Dzmitry Malyshau
@kvark
Jul 04 2014 19:10
Well, not really, and that's the problem... AFAIK, there is classical forward, deferred (both are GL-2.1 compatible), and there is forward+ and clustered (GL-4+), but nothing really covers GL-3.3 nicely. Something is clearly not yet being researched ;)
I did investigate different approximations to have one strong light plus many weak ones, but hasn't got anything useful yet.
Corey Richardson
@cmr
Jul 04 2014 20:12
What are forward+ and clustered?
Dzmitry Malyshau
@kvark
Jul 04 2014 20:17
@cmr forward+ is storing a list of lights per pixel or tile. Clustered is per cell in 3D texture (using depth). Can't say anything beyond it since I haven't tried them myself.
Corey Richardson
@cmr
Jul 04 2014 20:19
Ah ok
I had seen the tiled family of techniques, didn't recognize the names you used.
kvark @kvark need to read a bit more about the whole family to figure out the proper naming of things
Dzmitry Malyshau
@kvark
Jul 04 2014 20:23
AFAIK, clustered is good because it doesn't require depth pre-pass.
Corey Richardson
@cmr
Jul 04 2014 22:28
Anyone have any particularly good reading on rendering techniques etc?
Brendan Zabarauskas
@brendanzab
Jul 04 2014 22:42
We should put some of this stuff in the research.md
Dzmitry Malyshau
@kvark
Jul 04 2014 23:11
@bjz I'd like gfx-rs to remain what it is now - a lightweight device abstraction, a generic convenient interface to draw stuff. Rendering techniques belong to the layer above gfx-rs, whatever this is going to be.
Brendan Zabarauskas
@brendanzab
Jul 04 2014 23:11
Yeah, true
scratch that then :P
Corey Richardson
@cmr
Jul 04 2014 23:12
Yeah it's not super relevant to gfx-rs but there's a bunch of smart people in here ;)
This seems very promising too, http://jcgt.org/published/0002/02/04/
Corey Richardson
@cmr
Jul 04 2014 23:16
Holy shit that first screenshot
I've never seen realtime look like that
Coraline Sherratt
@removed~csherratt
Jul 04 2014 23:19
it's very impressive
also I think it will kill most pcs
Corey Richardson
@cmr
Jul 04 2014 23:20
yeah
especially at the resolutions we expect to play games at :laughing:
Coraline Sherratt
@removed~csherratt
Jul 04 2014 23:21
:/ I want to throw links at you but the best articles were in books
Corey Richardson
@cmr
Jul 04 2014 23:22
Which books?
Coraline Sherratt
@removed~csherratt
Jul 04 2014 23:33
I enjoyed GPU Pro 4, it had a few interesting chapters that were worth the price of admission. I need to read the entire series, (@kvark was published in GPU Pro 3). OpenGL insights has some interesting chapters, but I found it a bit hit and miss. The GPU gems books are free online, and they are also very good.
Corey Richardson
@cmr
Jul 04 2014 23:34
yeah I've been reading through gpu gems
Coraline Sherratt
@removed~csherratt
Jul 04 2014 23:35
I want to list chapters, but I have everything on kindle and not a machine at work that can display them.
Brendan Zabarauskas
@brendanzab
Jul 04 2014 23:36
@cmr how are you feeling about bjz/gl-rs#95?
Corey Richardson
@cmr
Jul 04 2014 23:41
Let's do it.
You may have the honors