Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Yoshifumi Kawai
    @neuecc
    Thank you for your explanation.
    Yes, it is the same as my thought, I've relieved.

    @AArnott
    Ah, I didn't set up Azure Pipelines (I was going to leave everything to you about CI)

    For mpc, the rest of the work is from this comment
    https://github.com/neuecc/MessagePack-CSharp/issues/617#issuecomment-551076028
    It is also necessary to update the documentation.

    Andrew Arnott
    @AArnott
    Thanks, @neuecc. I've updated the labels on #617 to signify that it's a 2.0 ship blocker and assigned to myself.
    What documentation needs to be updated? I've kept the README file up to date as I've made the various API breaking changes for 2.0, I think.
    Aleksander Heintz
    @Alxandr
    Any reason for the update to System.Collections.Immutable >= 1.6.0 in the latest rc release? It's breaking builds of azure functions.
    C:\Users\D45\.nuget\packages\microsoft.net.sdk.functions\1.0.29\build\netstandard1.0\Microsoft.NET.Sdk.Functions.Build.targets(41,5): error : System.IO.FileLoadException: Could not load file or assembly 'System.Collections.Immutable, Version=1.2.4.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. Could not find or load a specific file. (Exception from HRESULT: 0x80131621) [C:\Git\PolicyStaging\src\PolicyStaging\PolicyStaging.csproj]
    C:\Users\D45\.nuget\packages\microsoft.net.sdk.functions\1.0.29\build\netstandard1.0\Microsoft.NET.Sdk.Functions.Build.targets(41,5): error : File name: 'System.Collections.Immutable, Version=1.2.4.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' ---> System.IO.FileLoadException: Could not load file or assembly 'System.Collections.Immutable, Version=1.2.4.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. [C:\Git\PolicyStaging\src\PolicyStaging\PolicyStaging.csproj]
    C:\Users\D45\.nuget\packages\microsoft.net.sdk.functions\1.0.29\build\netstandard1.0\Microsoft.NET.Sdk.Functions.Build.targets(41,5): error :    at System.Runtime.Loader.AssemblyLoadContext.LoadFromPath(IntPtr ptrNativeAssemblyLoadContext, String ilPath, String niPath, ObjectHandleOnStack retAssembly) [C:\Git\PolicyStaging\src\PolicyStaging\PolicyStaging.csproj]
    C:\Users\D45\.nuget\packages\microsoft.net.sdk.functions\1.0.29\build\netstandard1.0\Microsoft.NET.Sdk.Functions.Build.targets(41,5): error :    at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyPath(String assemblyPath) [C:\Git\PolicyStaging\src\PolicyStaging\PolicyStaging.csproj]
    C:\Users\D45\.nuget\packages\microsoft.net.sdk.functions\1.0.29\build\netstandard1.0\Microsoft.NET.Sdk.Functions.Build.targets(41,5): error :    at System.Reflection.Assembly.LoadFrom(String assemblyFile) [C:\Git\PolicyStaging\src\PolicyStaging\PolicyStaging.csproj]
    C:\Users\D45\.nuget\packages\microsoft.net.sdk.functions\1.0.29\build\netstandard1.0\Microsoft.NET.Sdk.Functions.Build.targets(41,5): error :    at System.Reflection.Assembly.LoadFromResolveHandler(Object sender, ResolveEventArgs args) [C:\Git\PolicyStaging\src\PolicyStaging\PolicyStaging.csproj]
    C:\Users\D45\.nuget\packages\microsoft.net.sdk.functions\1.0.29\build\netstandard1.0\Microsoft.NET.Sdk.Functions.Build.targets(41,5): error :    at System.AppDomain.InvokeResolveEvent(ResolveEventHandler eventHandler, RuntimeAssembly assembly, String name) [C:\Git\PolicyStaging\src\PolicyStaging\PolicyStaging.csproj]
    C:\Users\D45\.nuget\packages\microsoft.net.sdk.functions\1.0.29\build\netstandard1.0\Microsoft.NET.Sdk.Functions.Build.targets(41,5): error :    at System.Reflection.RuntimeAssembly.GetExportedTypes(RuntimeAssembly assembly, ObjectHandleOnStack retTypes) [C:\Git\PolicyStaging\src\PolicyStaging\PolicyStaging.csproj]
    C:\Users\D45\.nuget\packages\microsoft.net.sdk.functions\1.0.29\build\netstandard1.0\Microsoft.NET.Sdk.Functions.Build.targets(41,5): error :    at System.Reflection.RuntimeAssembly.GetExportedTypes() [C:\Git\PolicyStaging\src\PolicyStaging\PolicyStaging.csproj]
    C:\Users\D45\.nuget\packages\microsoft.net.sdk.functions\1.0.29\build\netstandard1.0\Microsoft.NET.Sdk.Functions.Build.targets(41,5): error :    at MakeFunctionJson.FunctionJsonConverter.TryGenerateFunctionJsons() [C:\Git\PolicyStaging\src\PolicyStaging\PolicyStaging.csproj]
    C:\Users\D45\.nuget\packages\microsoft.net.sdk.functions\1.0.29\build\netstandard1.0\Microsoft.NET.Sdk.Functions.Build.targets(41,5): error :    at MakeFunctionJson.FunctionJsonConverter.TryRun() [C:\Git\PolicyStaging\src\PolicyStaging\PolicyStaging.csproj]
    C:\Users\D45\.nuget\packages\microsoft.net.sdk.functions\1.0.29\build\netstandard1.0\Microsoft.NET.Sdk.Functions.Build.targets(41,5): error :  [C:\Git\PolicyStaging\src\PolicyStaging\PolicyStaging.csproj]
    C:\Users\D45\.nuget\packages\microsoft.net.sdk.functions\1.0.29\build\netstandard1.0\Microsoft.NET.Sdk.Functions.Build.targets(41,5): error :  [C:\Git\PolicyStaging\src\PolicyStaging\PolicyStaging.csproj]
    C:\Users\D45\.nuget\packages\microsoft.net.sdk.functions\1.0.29\build\netstandard1.0\Microsoft.NET.Sdk
    Yoshifumi Kawai
    @neuecc
    @AArnott about mpc < documantation, and composite resolver for unity.
    Yoshifumi Kawai
    @neuecc
    and how to move v1 to v2(what's different)
    Andrew Arnott
    @AArnott
    @Alxandr No particular reason, except to try to stay a bit more current so folks have the latest libraries via transitive dependencies. Before the upgrade to 1.6.0 I see we were on 1.3.1 which is nearly 3 years old. What is the latest version that wouldn't break Azure Functions?
    Aside: I'm disappointed that Azure Functions breaks given newer assembly versions. Do you know why it's sensitive to this?
    @neuecc Can you write the MPC and unity documentation? I'm much less familiar with those. I can write the v1->v2 migration instructions.
    Aleksander Heintz
    @Alxandr
    @AArnott No clue. I'm also not able to reproduce it in a simple reproduction (Azure/azure-functions-vs-build-sdk#353). That being said, they do have some version issues at runtime due to them bringing their own runtime, but this is happening compile-time which is really confusing. It's failing to load System.Collections.Immutable while doing code-gen.
    Andrew Arnott
    @AArnott
    @Alxandr If you can provide a repro project in a github issue filed against messagepack in the next day or two, I think we can revert the S.C.I. dependency update and keep your issue active to track when we can upgrade it again.
    Aleksander Heintz
    @Alxandr
    @AArnott unfortunately I don't think I can. I'm afraid I'm going to have to debug into msbuild in a really complicated solution with ~70 projects :-/
    And it's work, so I can't just share the code either.
    Aleksander Heintz
    @Alxandr
    Ah. I reproduced it!
    Aleksander Heintz
    @Alxandr
    And issue as you wanted: neuecc/MessagePack-CSharp#654
    Yoshifumi Kawai
    @neuecc
    I'm now reviewing #646, and my rest work is write document of mpc and Unity. And I want to add mpc generate helper for UnityEditor, in v2 release. I'll post All PR within a day.
    Andrew Arnott
    @AArnott
    Awesome. Thanks, @neuecc !
    Aleksander Heintz
    @Alxandr
    Any chance of getting a new RC with the decreased immutable-collections? I'd love to get on the RC API to make sure it's still working for our application
    Andrew Arnott
    @AArnott
    Yes, I'll push one to nuget.org soon.
    Andrew Arnott
    @AArnott
    It looks like we'd better get #669 completed before the next release to avoid creating a nuget package id that will then be abandoned.
    Andrew Arnott
    @AArnott
    @neuecc It would be great if you could activate Azure Artifacts on your ils0086 account so we can create a CI feed for MessagePack, giving folks a way to always get the latest prerelease if they want it.
    Yoshifumi Kawai
    @neuecc
    @AArnott I've enabled Artifacts.
    Also, my v2 release blocker is gone.
    Andrew Arnott
    @AArnott
    Awesome
    Thanks. Do you mean you're willing to go for a 2.0 stable release, @neuecc?
    Andrew Arnott
    @AArnott
    @Alxandr 2.0.270-rc is now on nuget.org for your testing.
    Yoshifumi Kawai
    @neuecc
    @AArnott yes. (I've started to implement lz4 but it will be not in time to v2.)
    Andrew Arnott
    @AArnott
    ooh. sounds interesting. Thanks. I've branched off v2.0 from master. v2.0 builds a stable package and master is now tracking v2.1-alpha.
    I'm fine to give it a short while as 2.0-rc still to wait for feedback from @Alxandr or others in case there are any more changes we need to make.
    BTW as a matter of public API policy, I propose this, consistent with semver: API breaking changes get a major version bump. API additions, new features or noteworthy changes get a minor version bump. The only changes that can be made without a major or minor version bump are bug fixes.
    Yoshifumi Kawai
    @neuecc
    Okay. and agree to use semantic versioning.
    Yoshifumi Kawai
    @neuecc
    @AArnott sorry, is this final chance of breaking change? to implementing new lz4, I've noticed should change bool UseLZ4Comparresion to enum. #675
    Yoshifumi Kawai
    @neuecc
    thanks for the hard work in last few days.
    completely my concern is solved.
    wait the release:)
    Andrew Arnott
    @AArnott
    Yay. :)
    I'm happy with what we have now too. Since I don't need it stable this week, and we had some changes this past week, I suggest we give it another week of use to "stabilize" before removing the -rc and pushing the package.
    It's been great working with you, @neuecc
    Yoshifumi Kawai
    @neuecc

    Thanks, I’ve been happy too.

    my ex-colleague, clients are trying preview.
    I've been pointed one immediately. #698 :)

    Andrew Arnott
    @AArnott
    @neuecc What do you think of removing the MessagePackSerializer.Deserialize(Stream) methods in v2.0? We can support that scenario much better with #388 using a class that can track state between reads, thus enabling reading a message at a time from the stream but in a highly performant way. I think the way MessagePackSerializer.Deserialize(Stream) works today is unintuitive, and fixing it later to allow reading a message at a time wouldn't work because we need to store state between each read. I could even do it for 2.0 if necessary, but we'd need to agree on the removal of the method first.
    Yoshifumi Kawai
    @neuecc
    @AArnott Deserialize (Stream) is a standard method, so if it is deleted, it will be quite difficult to use as an API.
    for example, many people will expect the following API (sample from AspNetCoreMvcFormatter),
    MessagePackSerializer.DeserializeAsync (request.Body)
    Also, the processing of Buffer by Sequence is included,
    It is more efficient than user created buffer.
    Of course, I can understand the disadvantages (which causes unexpected behavior), but as a trade-off.
    What about APIs that provide length hints instead?
    For example, AspNetCoreMvcFormatter
    We can write DeserializeAsync (context.ModelType, request.Body, request.ContentLength)
    This is certainly useful for streams with a length prefix.
    Andrew Arnott
    @AArnott
    Yes, discoverability of the right API is a concern. and having a Serialize(Stream) method but no Deserialize(Stream) method feels odd.
    In your case of providing lengths, you're providing the length of the Body stream anyway, so that's no better than if we just read to the end of the stream. But yes, in general if folks know the length of a slice of the stream to read from, they should be able to pass that in if we depended on it.
    Nerdbank.Streams offers a Stream.Slice extension method that already restricts reading to a given length. But having it built in to messagepack might be useful for folks who don't have that.
    On the other hand, I wonder if a stream has msgpack in it, wouldn't it either end with that msgpack structure or have more msgpack structures? So if we offered an API that can repeatedly read from a stream such that the length in advance doesn't have to be known, that sounds like what folks expect based on the several issues that folks are asking for this on.
    Thinking more about this, how about a MessagePackStreamReader and MessagePackStreamWriter class? The reader of course allows for very efficient, async reading from the stream and can return ReadOnlySequence<byte> slices with entire msgpack structures in it. The writer could implement IBufferWriter<byte> so that it can be used to serialize with MessagePackSerializer directly. It might use Sequence<byte> internally for buffering but would allow us to be writing to the stream in the background even while serialization occurs.
    Andrew Arnott
    @AArnott
    I think if we remove the Stream-related APIs from MessagePackSerializer then folks would look at it just as a serializer and not as a streaming control system as well, and the two class names I propose would be pretty discoverable, IMO. As people type MessagePackS they would see the Serializer as well as the StreamReader/Writer classes in the completion list. But if we wanted it even more discoverable, we could add static CreateStreamReader and CreateStreamWriter methods to the MessagePackSerializer class.
    Or here's another option that you might like: just add a ref StreamReaderState parameter to the Deserialize(Stream method that folks would repeatedly pass in each time they read from the same stream, and that would allow us to incrementally read from the stream. This struct would have a bunch of internal fields that would retain the state we need (e.g. the buffers we read too far on). The only question would be what we do with DeserializeAsync since async methods can have ref parameters. We could use a class for StreamReaderState instead of a struct to allow the async method to work.
    Andrew Arnott
    @AArnott
    Here's a draft of the alternative solution I was describing.
    Yoshifumi Kawai
    @neuecc

    I commented on PR, but I think State is a good idea.
    It's a somewhat uncommon API, but it should be fine.

    Even if there is a State, I think that it is not bad to default to reading until the end if it is not specified.
    Here is my image of usage:
    90%-Ok to read end.
    9%-simply contiguous block.
    1%-concatenated other things.

    90% situation, read to end is fastest.
    9% situation, use state is good to use.
    1%, we don't care.

    Andrew Arnott
    @AArnott
    Sounds good, @neuecc. I'll proceed with the state API design. And we can keep overloads that omit the state parameter to keep it simply and familiar for folks in your 90% case.
    For the async method where we can't use ref parameters, I can use a StrongBox<state> parameter to achieve the same result.
    Aleksander Heintz
    @Alxandr
    @AArnott why *Strong*Box?