by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • May 11 21:31
    adamnv opened #33
  • Mar 06 13:19
    yeuhibd opened #32
  • Feb 13 23:56
    janwilmans edited #31
  • Feb 13 12:11
    janwilmans opened #31
  • Jul 01 2019 14:53
    jlyga3 commented #26
  • Jun 17 2019 14:08
    travnick opened #30
  • Apr 02 2019 03:45
    Justin-Randall commented #26
  • Apr 02 2019 03:43
    Justin-Randall closed #27
  • Apr 02 2019 03:43
    Justin-Randall commented #27
  • Apr 01 2019 06:42
    Justin-Randall closed #29
  • Apr 01 2019 06:42
    Justin-Randall commented #29
  • Apr 01 2019 04:16
    damian123 edited #29
  • Apr 01 2019 04:16
    damian123 edited #29
  • Apr 01 2019 04:15
    damian123 opened #29
  • Mar 06 2019 16:26
    mmatrosov commented #28
  • Mar 06 2019 03:22
    Justin-Randall commented #28
  • Mar 05 2019 16:55
    mmatrosov commented #28
  • Mar 02 2019 03:29
    Justin-Randall commented #28
  • Mar 01 2019 20:37
    mmatrosov commented #28
  • Mar 01 2019 20:36
    mmatrosov commented #28
Jean-Sebastien Mouret
@jsmouret
you can reuse the same link to activate the other agents
midinerd
@midinerd_twitter
Fantastic. Thanks for the help
Jean-Sebastien Mouret
@jsmouret
the trial license can connect up to 42 seats, after that you'll need to get a license with the proper number of seats
midinerd
@midinerd_twitter
When it gets distributed to another PC, via Incredibuild, that destination machine also has to have identical filepaths set up as well? I believe this is true based on my test just now, just want to clarify.
As we have some production machines that are virtualized and practically identical, while also there are developer boxes in our Incredibuild pools that are probably gonna be more customized filesystem-wise :)

If I disable IB and re-run I get cache hits. If I re-enable it, I think I see myself getting misses and I presume it's based on developer machines handling the distributed request, and their filesystem not being the same.

I hoped the IB 'containerization' method xoreax uses would shield from this issue.

If this is not the case - and I maybe have IB set up wrong somehow, please let me know.

Otherwise I will draft this (filesystem congruence for machines with distributed seating) into our project requirements per stashed's requirements.

Jean-Sebastien Mouret
@jsmouret
wow, that's nice to add as a requirement :)
some teams are using 'subst' drives to have the same paths everywhere
about incredibuild, when the build is run on a remote agent, as it is containerized, stashed does not use the local stashed service on the agent machine, but calls back to the issuing machine
(not sure if that makes sense)
midinerd
@midinerd_twitter
I think I follow you
Jean-Sebastien Mouret
@jsmouret
basically, the agent receiving a build with incredibuild, runs it inside some kind of virtual filesystem that mirrors the original machine filesystem
so when the stashed wrapper accesses the filesystem, it sees the files from the original machine
to conclude, it should work :)
midinerd
@midinerd_twitter
gotcha - :) I follow you, thanks. haha
Alrighty - I re-enabled incredibuild and I'm seeing job hits happening.
Thanks a ton for helping me along here.
Jean-Sebastien Mouret
@jsmouret
thanks for testing stashed :)
midinerd
@midinerd_twitter
How would I reach out to you via e-mail?
Jean-Sebastien Mouret
@jsmouret
maybe support@playscale.io would be more reliable if I'm on holidays :)
midinerd
@midinerd_twitter
Gotcha :) Definitely noted
midinerd
@midinerd_twitter
Hey Jean - is there a log I can check any details on the hit/misses?
I'm trying to figure out a moving target in our build system that might cause 'misses' from a filename changing, but the content being the same (i.e. this modifies the cmdline/pathing)
I'm fairly certain this is the reason, but if I could get some feedback from stashed to confirm it I'll definitely leverage that.
Jean-Sebastien Mouret
@jsmouret
sadly, if nothing is showing in the logs tab in the dashboard, there are no other options for now
we can plan to add an option to log more info (like a debug switch)
akaraivanov
@akaraivanov

Hi. I am testing stashed. I think cache hit is not very good. I did number of rebuild and clear cycles with stashed. Also increased disk cache size. Before the last rebuild I cleared logs, and statistics. Then I get in statistics:
Job hits: 1940
Job miss: 2
Blob hit: 10,531
Blob miss: 2,146

In logs, I get 340 warnings of the same type having reason: "/out:something\foo.rc.res.dep.obj", that appear to be related to compilation of resources.

Q1: how can I diagnose where are blob misses coming from?
Q2: is it possible to disable temporarily stashed to have more builds without it?

akaraivanov
@akaraivanov
On the next day, when my vm was restarted, I do not get Blob miss any more... Strange
Jean-Sebastien Mouret
@jsmouret
sorry, for the late reply
Q1: for common cases like using PDB+PCH at the same time, it just increases the `unsupported job' counter. for other cases, it should output an error/warning telling why it bailed in the Logs tab on the dashboard.
Q2: yes, you can disable all frontends at once in the dashboard
Looking at the stats above, very few jobs misses and plenty of blob misses is a symptom of a small cache size. stashed knows about the compilation request (job hit) but cannot find the matching obj files (blob miss), most likely because they got removed from the cache.
akaraivanov
@akaraivanov
Thanks for the answer! I will try increasing the cache size
bpaberg
@bpaberg
Hi, I'm also testing stashed. looks really cool. A question:
There is an old problem compile problem with building in parallel (/MP) and using #import so we do not have /MP on some projects.
With stashed this problem started happening again. Does stashed build in parallel even if /MP is not on?
I'm thinking that since I saw this:
"In this scenario, caching is disabled but compilation will still benefit from automatic multi-threading and the correct output will be generated."
here: https://github.com/playscale/stashed.io/wiki/Stats#job-unsupported
bpaberg
@bpaberg
Also, I tried to put the #import in StdAfx.h since that one should be compiled before everything else but then I got cache miss on all jobs in the project.
Jean-Sebastien Mouret
@jsmouret
as far as I know, stashed simply discards /MP
stashed splits the command line in multiple compilation requests that are indeed run in parallel whether /MP is on or not
(googling the #import problem, we don't do anything special about it...)
Jean-Sebastien Mouret
@jsmouret
looking at https://msdn.microsoft.com/en-us/library/bb385193.aspx it says that /MP is not compatible with #import, but I'd like to know why to understand how we can workaround that...
bpaberg
@bpaberg
yea, well /MP is not compatible with #import since compiling a cpp with import will generate a tlh and tli files so if multiple cpp files have the same #import and are compiled in parallell then they will clash on disk, getting a permissin denied error
tis is a limitation in cl and I guess that modern code does not use #import anymore
there is also a workaround that Microsoft mention, to make a specific pass (target) that have the #imports and will generate the tlh and tli files, then in another pass (target) use the generated files.
so I would guess that stashed should not really have to do anything about this cl-flaw apart from maybe mention it and the workaround in the faq
bpaberg
@bpaberg
btw, with stashed, there is a job miss on cpp-files that use #import, maybe it should be "job unsupported" instead?
Jean-Sebastien Mouret
@jsmouret
is there anything on the dashboard logs when that file is compiled?
bpaberg
@bpaberg
no, just that I get job miss for it in Stats
bpaberg
@bpaberg
there is a blob hit for it though, how can that be if is is a job miss?
Jean-Sebastien Mouret
@jsmouret
the job itself is stored as a blob, which is fetched ok, but it might be missing dependencies so it counts a job miss. we must be doing something wrong when finding dependencies/outputs for cpp files with #import.
bpaberg
@bpaberg
mm, i guess the tlb file is a dependency and tlh tli files are extra outputs