playscale/stashed.io

Let's talk about build acceleration

Ben Peck

@bpeck

ie everyone has their workspace mapped to a common base path on a z: drive. But they are mapping from an arbitrary physical drive like c: or d:

Jean-Sebastien Mouret

@jsmouret

Interesting... I don't think we have tried this. Stashed will try to use the local hard drive to cache first (which can be confusing with the mapped drive) but it should fallback on others. We need some time to investigate and we'll get back to you

KonanM

@KonanM

Ok interesting. It seems like there is excatly the same issue for my company. We also use virtual drives and stashed.io doesn't work...

willjoseph

@willjoseph

I'm running into warnings when stashed.io is installed only on my machine, and I run a UE4 build distributed across multiple IncrediBuild agents. EXEC : warning : Stashed service could not be reached from this remote environment. Job hit:miss rate for consecutive full rebuild is around 1:3, probably due to this warning. All agents are Windows 10 with no special firewall settings that I know of.

bitsrules

@bitsrules

I just installed stashed.io and it work possible because of my compiler flags. Here is the command I am using
cl file.cpp /MP /GS /TP /analyze- /W3 /Zc:wchar_t /I"Z:\F\dev\include" /I"C:\dev\vcpkg\installed\x86-windows\include" /Gm- /Od /Ob0 /Fd"proxy.dir\Debug\proxy.pdb" /Zc:inline /fp:precise /D "WIN32" /D "_WINDOWS" /D "BOOST_CONFIG_SUPPRESS_OUTDATED_MESSAGE=1" /D "CMAKE_INTDIR=\"Debug\"" /D "_MBCS" /errorReport:prompt /WX- /Zc:forScope /RTC1 /Gd /Oy- /MD /std:c++17 /FC /Fa"Debug/" /EHsc /nologo /Fo"proxy.dir\Debug\" /Fp"proxy.dir\Debug\proxy.pch" /diagnostics:classic

I got it to work with a following the example in the docs i.e. cl /c /Yc a.cpp

Justin Randall

@Justin-Randall

@bitsrules Have you tried with /Z7?

bitsrules

@bitsrules

/Z7 did also not work. The solution was to move the source code from my shared drive to the local disk

Ben Peck

@bpeck

@jsmouret Have you had a chance to look into using virtual drives with stash?

Justin Randall

@Justin-Randall

It does work with mklink

olesteban

@olesteban

Hi, I installed yesterday this product for evaluation. Coming from linux ccache (even distcc) I'm surprised to see it depends on the absolute paths (two working copies of the same code don't share cache results). I've read it's a restriction for distributed use, but is it also local?

olesteban

@olesteban

I've seen (playscale/stashed.io#6) it's also local :-( OTOH, I'd say ccache does not depend on any environment variable), but (not sure, as I read about it long time ago) on the command line and a hash of the source files involved.

Justin Randall

@Justin-Randall

Will have another look and add a test case for relative paths.

Adrian Cowan

@othrayte

@Justin-Randall, in your update to #6 you mention the tradeoffs for relative path support and asked if "everyone [would] be OK with undefined debug behavior if the strange outcomes were well documented?". I imagine this would be acceptable to me, but also without relative path support almost all of the advantage of stashed would be lost for me. Could you go into what the kinds of strange outcomes might be? i.e. what the documentation might say.

Justin Randall

@Justin-Randall

Where possible, with the new feature, we will try to use relative paths. 3rd party libraries, system headers, etc... may be installed elsewhere, even if the content hash is identical. Currently Stashed chooses to treat that as a different hash (cache miss). If it did not, and your build retrieved some cache output built with different source locations, you may end up "missing" source files when debugging (as in, someone installed DirectX headers on their "F" drive, whereas you have them in a standard install location). The Visual Studio IDE will let you work around this manually, but it is probably not what most people would expect after building their changes and running under the debugger.

We really want Stashed to "just work" without much fuss, and where it would lead to different behavior, it should fall back to the compiler because "correct" is more important than "fast but incorrect".

Adrian Cowan

@othrayte

In the case that the source is the same and only difference was its absolute location used for finding the source when debugging then I wouldn't consider the result to be incorrect. It isn't uncommon with prebuilt thrid party libraries to need to locate the source when debugging because it was located elsewhere on their build server. In my use case the system compiler and std library are in common locations and the application code and 3rd party libraries are located in slightly different directories both between users cmputers and betweent each of the copies of the repo on my own computer. So with relative path and content matching I would not only be able to get cache hits from other computers but also get more cache hits on my own computer.

Thomas Sondergaard

@tsondergaard

Hi, I am running into situations where c.exe from stashed.io hangs. I would like to figure out more, but the debugger is not much use (with me at them helm) without debug symbols.

Are the debug symbols for stashed.io releases available for download or on a symbol server somewhere?

Thomas Sondergaard

@tsondergaard

Concerning the question "correct" vs "fast but incorrect", I like the idea of sticking with correct!

Justin Randall

@Justin-Randall

Still working through some deep design changes to support relative paths. Stashed has pretty extensive test coverage, but given the nature of the change, extra care is being taken to ensure it is both fast and correct. :)

bpaberg

@bpaberg

The new debug generation flag Zf (used together with Zi,ZI), would that help to make stashed work with pdb files (Zi) also?
https://docs.microsoft.com/en-us/cpp/build/reference/zf?view=vs-2017

Another question, does anyone know if Z7 (C7) debug info is supposed to work with debugging coroutines?

Justin Randall

@Justin-Randall

I am not confident that the new switch will help much since it is still routing through mspdbsrv.exe.

bpaberg

@bpaberg

Justin Randall

@Justin-Randall

There may be a scenario where it does work, since it is intended to deal with parallel builds, but the pdbserver is quirky. It wouldn't hurt to give it a shot.

bpaberg

@bpaberg

yea, i got the impression that the pdbserver got involved late enough to not interfere or something

bpaberg

@bpaberg

regarding /Z7 vs /Zi + /Zf, the thing is that we can not get debugging to work with /Z7 in our project, which means we need to use /Zi, which means no stashed benefit.

bpaberg

@bpaberg

we also use pch

hm, when using /Zi + pch, is it only for generating pch (/Yc + /Zi) stached will skip? using pch (/Yu + /Zi) workks with stashed?

bpaberg

@bpaberg

How do I move a license from one machine to another?

Justin Randall

@Justin-Randall

You can copy the mainlince key from your programdata folder. or use the license dashboard on the other computer if you prefer the ui.

Bartek Banachewicz

@bananu7

Hm, I've installed the cache and started a build, but the caches remain empty

Bartek Banachewicz

@bananu7

It works when I run cl.exe from console, but not from Visual Studio

Bartek Banachewicz

@bananu7

Nevermind, it's showing in "Job Unsupported". Oh well, uninstalling...

Justin Randall

@Justin-Randall

There should probably be some better feedback about that unsupported jobs. Mixing PDBs and PCH outputs will do that, which is a default for most VS projects.

Ky Waegel

@kwaegel

Are LINK : fatal error LNK1102: out of memory errors a known issue with Stashed? I only started getting those after setting up the cache.

bpaberg

@bpaberg

i can not reinstall stashed on two computers now, seems like the service can not be started

is that a known problem?

bpaberg

@bpaberg

seems that I can not clenly unistall stashed either

after uninstalling it seems that a wrapper is still active since it is spamming this message when building:
"warning : Stashed service is not available. This will not be accelerated. Is the service running?"

Liru Færs

@Lirusaito

Is it a known problem that sometimes builds just stop, until you manually disable frontends from dashboard?

In Logs, I get this Error: 2019/02/17 13:23:51.054 ERROR
Disable(): Object {reason: "msvs:v2017:211b1ebd:14.16.27023:x86_x64:remove d:\programs\microsoft visual studio\2017\community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\cl.exe: Access is denied."}

reason: "msvs:v2017:211b1ebd:14.16.27023:x86_x64:remove d:\programs\microsoft visual studio\2017\community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64\cl.exe: Access is denied."

BitmaniaK69

@BitmaniaK69

Hi all, I've a very silly question..Which is the best way to understand if stashed.io is working? I just installed but I don't know if is enabled and working apart watching the stats (I've no log tracked of the actions taken)

I want to demonstrate the advantages of this solution + incredibuild comparing times with or without stashed.io. What I'm doing is rebuild the project and taking times.

But I don't know how to enable/disable