Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Davis Goodin
    @dagood
    Ah yes, I merged dotnet/source-build#1705 to get ready to delete the release/3.1 branch but didn't end up doing it. Deleting now. Thanks for the reminder. :)
    Omair Majid
    @omajid
    Thanks for clearing this up. I saw that note, but didn't put 2 and 2 together
    Davis Goodin
    @dagood
    Np. Yeah, IMO the info in the note isn't quite enough to be sure about this anyway--pre-arcade could in theory only be for 5.0 (but before arcade-powered source-build), and release/3.1 for 3.1.
    Anton Chebotarev
    @NaivE2k4

    Hello! Sorry to bother you again. I'm still confused about runtime packs. The docs are very scarce. Could you please explain?

    So, to remind, i've got my source-built portable sdk (no internet environment), but we found out that it's not enough for self-contained app publish.
    Then i've pointed nuget source to artifacts/obj/x64/Release/blob-feed/packages and it found runtime pack there, if I got it right.

    By the way got another problem - I've tried to build runtime only with build.sh -- /p:rootrepo=core-setup. It made me a runtime and still a runtime pack in ../obj/.. but the test app couldnt publish because it demanded AspNetCore runtime pack also!

    So,
    1) Is it true that simple console app demands asp.net core runtime pack for scd publish?
    2) Where actually self-contained publishget the libraries to put with app? Do it even need a runtime (or just runtime pack)?
    3) What's runtime pack consist of? Is it good old coreclr+corefx+...? Does it builds from the same repos as actual runtime i build with build.sh -- /p:rootrepo=core-setup?

    The reason i'm so curious is because i need to provide sources for files i'm publishing, and if they're from runtime pack, i need to know how it gets build.
    The concept is hard to get for me :-[

    Davis Goodin
    @dagood

    Then i've pointed nuget source to artifacts/obj/x64/Release/blob-feed/packages and it found runtime pack there, if I got it right.

    Yeah. The issue here is that there's no way to distribute an RPM package that contains it in a "normal" place to pick it up: dotnet/source-build#1215. Your workaround here seems fine for now.

    1) Is it true that simple console app demands asp.net core runtime pack for scd publish?

    Yes, this is a limitation in NuGet. Even if you don't need ASP.NET at all, you still need to download all runtime packs to do a self-contained app. https://github.com/dotnet/installer/issues/1958#issuecomment-679153622

    2) Where actually self-contained publishget the libraries to put with app? Do it even need a runtime (or just runtime pack)?

    Just the runtime pack. This is how you're able to run e.g. publish -r win-x64 on a Linux host (if you have an internet connection)--it grabs the win-x64 runtime pack.

    3) What's runtime pack consist of? Is it good old coreclr+corefx+...? Does it builds from the same repos as actual runtime i build with build.sh -- /p:rootrepo=core-setup?

    You can open them up (just a zip, and it also gets extracted automatically to your nuget package cache when you use it), it's basically a runtime with some extra data and tooling to make it work for the publish scenarios. The best doc describing what these things are is at https://github.com/dotnet/designs/blob/main/accepted/2019/targeting-packs-and-runtime-packs.md. And yeah, everything in artifacts/obj/x64/Release/blob-feed/packages is built from source during the build.

    Anton Chebotarev
    @NaivE2k4
    @dagood thanks for the answers!
    Davis Goodin
    @dagood
    Np, glad to help. :)
    Demid Lupin
    @lupin-de-mid
    Is it artifacts/obj/x64/Release/blob-feed/packages something new after 3.1.105? because after build i found only
    dotnet-v3.1.105-SDK/artifacts/x64/Release$ ls -lh
    -rw-r--r-- 1 root root 889M Sep 24 03:38 Private.SourceBuilt.Artifacts.0.1.0-dev.tar.gz
    -rw-r--r-- 1 root root  33K Sep 24 00:48 SourceBuild.MSBuildSdkResolver.deps.json
    -rw-r--r-- 1 root root  11K Sep 24 00:48 SourceBuild.MSBuildSdkResolver.dll
    -rw-r--r-- 1 root root 2.1K Sep 24 00:48 SourceBuild.MSBuildSdkResolver.pdb
    -rw-r--r-- 1 root root 154M Sep 24 03:36 dotnet-sdk-3.1.105-astra.16-x64.tar.gz
    -rw-r--r-- 1 root root  52M Sep 24 03:36 dotnet-sdk-internal-3.1.105-astra.16-x64.tar.gz
    drwxr-xr-x 2 root root 4.0K Sep 24 00:51 netcorecli-fsc
    drwxr-xr-x 2 root root 4.0K Sep 24 02:24 runtime
    Demid Lupin
    @lupin-de-mid

    I try update to 108 and build new source tarball

    git status
    HEAD detached at v3.1.108-SDK
    nothing to commit, working tree clean
    
    ./build-source-tarball.sh ../tarbals_108/  --with-sdk /home/user/gitlab/libraries/net-bootstrap/ 
    Using custom bootstrap SDK from '/home/user/gitlab/libraries/net-bootstrap/', version 3.1.106
    Found bootstrap SDK 3.1.106, bootstrap Arcade 1.0.0-beta.19359.6
    +++ dirname /home/user/gitlab/libraries/source-build/build.sh
    ++ cd -P /home/user/gitlab/libraries/source-build
    ++ pwd
    + scriptroot=/home/user/gitlab/libraries/source-build
    + '[' false == true ']'
    ++ date +%m%d%H%M%S
    + /home/user/gitlab/libraries/source-build/eng/common/build.sh --restore --build -c Release --warnaserror false /bl:/home/user/gitlab/libraries/source-build/artifacts/log/Debug/Build_0924162014.binlog /flp:v=diag /p:ArchiveDownloadedPackages=true /p:TarballRoot=/home/user/gitlab/libraries/tarbals_108 /p:PackSourceBuildTarball=true /bl:/home/user/gitlab/libraries/source-build/artifacts/log/Debug/BuildTarball_0924162014.binlog
    Downloading 'https://dot.net/v1/dotnet-install.sh'
    dotnet_install: Warning: Unable to locate zlib. Probable prerequisite missing; install zlib.
    dotnet-install: Downloading link: https://dotnetcli.azureedge.net/dotnet/Sdk/3.1.106/dotnet-sdk-3.1.106-linux-x64.tar.gz

    Why is it trying download SDK?

    Demid Lupin
    @lupin-de-mid
    --with-sdk <dir>                   use the specified SDK to check out source code.  do not copy it to the tarball.  an external SDK will be required to build from the tarball.
    Davis Goodin
    @dagood
    Looks like you went to the wrong directory with the first question:
    artifacts/x64/Release vs. artifacts/obj/x64/Release
    Looking at the SDK download issue...
    Davis Goodin
    @dagood
    It may have the same root cause as dotnet/source-build#1711, mind filing a new issue?
    Demid Lupin
    @lupin-de-mid
    It may have the same root cause as dotnet/source-build#1711, mind filing a new issue?
    dotnet/source-build#1769
    Demid Lupin
    @lupin-de-mid
    Looks like you went to the wrong directory with the first question:
    artifacts/x64/Release vs. artifacts/obj/x64/Release
    Thank you :)
    Adrian Zimmer
    @ZimmerA
    Is build.sh supposed to work on a clean Ubuntu 20.04 installation or do i need to install some packages first? Because I tried just running build.sh and it kept telling me about missing packages. Is there a list of packages i need to have installed or do I need to re-start the build process after each error until it works?
    Davis Goodin
    @dagood
    No, yeah, you need to install prereqs first. I don't know if there's a great minimal list--we use this Dockerfile for the latest Ubuntu in our CI:
    https://github.com/dotnet/dotnet-buildtools-prereqs-docker/blob/master/src/ubuntu/18.04/amd64/Dockerfile
    The prereqs for the full build from source are the sum of all prereqs of all the repos we build, but CoreCLR and CoreFX have the most I think, so maybe these will be enough:
    https://github.com/dotnet/coreclr/blob/release/3.1/Documentation/building/linux-instructions.md
    https://github.com/dotnet/corefx/blob/release/3.1/Documentation/building/unix-instructions.md
    You could also look at the Fedora BuildRequireses, or other distros, but your package names will be different: https://src.fedoraproject.org/rpms/dotnet3.1/blob/master/f/dotnet3.1.spec
    @ZimmerA ^
    Adrian Zimmer
    @ZimmerA
    Awesome, thank you!
    Davis Goodin
    @dagood
    Np :)
    Adrian Zimmer
    @ZimmerA
    Does anyone know how to run the build with a newer clang version than 3.9?
    Im trying to make a conda package for dotnet core and the oldest clang version on conda is 8. CoreCLR requires 3.9 ("Please install clang-3.9 before running this script").
    Omair Majid
    @omajid
    The version of clang shouldn't matter. In Fedora, for example, we are building against clang 11. Is it finding any version of clang at all?
    Adrian Zimmer
    @ZimmerA
    I can use clang in the terminal so I think the build script should find it too.
    Adrian Zimmer
    @ZimmerA
    I might have misinterpreted something, i think the error is coming from it not finding llvm-ar
    Omair Majid
    @omajid
    Anyone know how to figure out the ordering of new repos in repos/known-good.proj? I am looking to add https://github.com/dotnet/cssparser which is needed by aspnetcore. Can I put it in any tier before Tier 3?
    Davis Goodin
    @dagood
    The ordering there shouldn't matter, tier 3before tier 3 sounds reasonable from memory. The important thing is that aspnetcore.proj has a dependency on it.
    known-good is a way to construct a partial tree when we aren't actively building all repos, otherwise it shouldn't be critical for the build ordering.
    Also wow, why isn't that repo using Arcade infra, and why is its readme so hostile. Urgh
    Davis Goodin
    @dagood
    To be clear, not a question for you, we're looking for how to reach out. :)
    Omair Majid
    @omajid
    Oh. I missed adding a dep from aspnetcore to cssparser...
    The dependency is expressed through a RepositoryReference?
    Davis Goodin
    @dagood
    Yep, RepositoryReference determines order, works like ProjectReference in this respect.
    So like <RepositoryReference Include="arcade" /> in aspnetcore.proj means it has to build arcade.proj (name by convention) before it builds aspnetcore.proj.
    Omair Majid
    @omajid
    Thanks, that makes sense.
    Omair Majid
    @omajid
    Thanks again for all the help! I opened dotnet/source-build#1814 to add cssparser
    Omair Majid
    @omajid
    Hey folks. I need some advice. https://github.com/microsoft/vstest/blob/master/src/Microsoft.TestPlatform.Extensions.BlameDataCollector/NetClientHangDumper.cs uses Microsoft.Diagnostics.NETCore.Client which makes Microsoft.Diagnostics.NETCore.Client suitable for building from source, right? This is a part of the https://github.com/dotnet/diagnostics/ repo which includes a ton of other things too. The src/Tools/dotnet-gcdump/ tool in the diagnostics repo fails to build with an error that looks like it wants an API not available from System.CommandLine (https://gist.github.com/omajid/a999ec286ffa494591d4b2bbb71c2f83). What is the best course of action here? Should I try and build a subset of diagnostics repo?
    Davis Goodin
    @dagood
    Hmm, yeah, if BlameDataCollector is used by the SDK (maybe a more general check is if the Microsoft.Diagnostics.NETCore.Client DLL ends up in the Microsoft-built SDK?) then we do need to build that from source (not as a ref, and no skip allowed).
    I think a subset is reasonable, depends on whether the Microsoft-built SDK includes dotnet-gcdump as a default tool I guess? Depending on how well dotnet/diagnostics supports arcade stuff, patching an <ExcludeFromSourceBuild>true</> prop might be good enough to specifically carve out that project.
    @crummel added the command-line-api repo which I think should create the System.CommandLine packages? So maybe you have a repo ordering issue, need to add a RepositoryReference to your diagnostics.proj to make the nupkg available?
    Davis Goodin
    @dagood
    Omair Majid
    @omajid
    Oh, I hadn't thought to check the Microsoft-built SDK. Looking at it, the dll does end up in the SDK:
    $ tar tf dotnet-sdk-5.0.100-rc.2.20479.15-linux-x64.tar.gz  | grep Diagnostics.NETCore.Client
    ./sdk/5.0.100-rc.2.20479.15/Extensions/Microsoft.Diagnostics.NETCore.Client.dll
    Davis Goodin
    @dagood
    Something to keep in mind though I suppose is that someone might use ILMerge (NuGet) or similar (single-file publish--but that would be strange in the SDK since it also contains a Runtime) so absence isn't a perfect indicator anyway. :P
    But that does confirm it at least.
    Omair Majid
    @omajid
    dotnet/diagnostics is fun. You call build.sh and it doesn't produce nuget packages :/
    Omair Majid
    @omajid
    BUILD: Product binaries are available at /home/omajid/devel/dotnet/diagnostics/artifacts/bin/Linux.x64.Debug
    $ find -iname '*.nupkg'
    ./.dotnet/templates/5.0.0-rc.1.20452.10/microsoft.dotnet.web.spa.projecttemplates.5.0.5.0.0-rc.1.20451.17.nupkg
    ./.dotnet/templates/5.0.0-rc.1.20452.10/microsoft.dotnet.web.projecttemplates.5.0.5.0.0-rc.1.20451.17.nupkg
    ./.dotnet/templates/5.0.0-rc.1.20452.10/microsoft.dotnet.common.itemtemplates.5.0.0-rc.1.20431.3.nupkg
    ./.dotnet/templates/5.0.0-rc.1.20452.10/microsoft.dotnet.test.projecttemplates.5.0.1.0.2-beta4.20420.1.nupkg
    ./.dotnet/templates/5.0.0-rc.1.20452.10/microsoft.dotnet.web.itemtemplates.5.0.0-rc.1.20451.17.nupkg
    ./.dotnet/templates/5.0.0-rc.1.20452.10/nunit3.dotnetnew.template.1.8.1.nupkg
    ./.dotnet/templates/5.0.0-rc.1.20452.10/microsoft.dotnet.common.projecttemplates.5.0.5.0.0-rc.1.20431.3.nupkg
    Omair Majid
    @omajid
    Anyone know if I can save up Private.SourceBuilt.Artifacts.0.1.0-5.0.100-bootstrap.9.tar.gz and provide it to a build instead of having it download it every time? With repeated builds of source-build, this takes up quite a bit of time.
    Davis Goodin
    @dagood
    In theory, overriding ExternalTarballsDir might work. dotnet/source-build#1659
    And yeah, I wonder if dotnet/diagnostics needs --pack passed to it? Arcade defaults to "do nothing" so sometimes a whole --restore --build --pack is needed. The repo-specific scripts tend to be oriented towards devs who want to run tests, not actually producing something usable.
    Omair Majid
    @omajid
    I tried using the arcade way, (calling ./build.sh --restore --build --pack + ExcludeFromSourceBuild but it's leading me to errors I dont even understand