MessagePackDynamicObjectResolverException: can't find matched constructor.[MessagePackObject]
public class IntKeySample
{
[Key(3)]
public int A { get; }
[Key(10)]
public int B { get; }
[SerializationConstructor]
public IntKeySample(int a, int b)
{
A = a;
B = b;
}
}
Can you try suppressing deployment of your own System.Memory.dll and see if that fixes the issue I'll have to investigate how this is done... still getting up to speed on .NET subtleties. If you've got more details at the ready I'll take 'em, otherwise I'll look into it. Thanks.
object and leave them null though. Otherwise the only other option is write your own IMessagePackFormatter<T> for your type.
IBufferWriter, System.Pipelines, etc, but I can't seem to find an answer to a very simple question - what syntax should I use if I want to write a MessagePack payload via HTTP using HttpClient / HttpWebRequest, without incurring the cost of copying the buffer over to the underlying stream?
Stream, correct? You can then use the UsePipeWriter extension method to get a PipeWriter for use in writing to that Stream. Since PipeWriter implements IBufferWriter<byte>, you can pass it directly to MessagePackSerializer.Serialize.Although in effect this is what our built-in overload does, so is there a reason not to use that?
Note that neither of these actually streams serialization out. It makes effective use of many small arrays as buffers and then asynchronously streams them out. But to get actual serialize-more-than-you-have-memory-for-because-you're-transmitting-as-you-go behavior requires more work. Folks have done it with this library, but it's a little ugly, and almost never necessary.
[Union(2, typeof(Mission))]
[Union(1, typeof(Challenge))]
[Union(0, typeof(Quest))]
[Serializable, MessagePackObject(true), Table("Tasks")]
public abstract class Task : IGameReadModel
{
[PrimaryKey, CanBeNull] public string InstanceId { get; set; }
public TaskType? TaskType { get; set; }
public TaskLength? TaskLength { get; set; }
[CanBeNull] public string TargetId { get; set; }
[CanBeNull] public string StatId { get; set; }
[Ignore, CanBeNull] public string Type { get; set; }
public string Title { get; set; }
public string Description { get; set; }
public double Current { get; set; }
public double? Avg { get; set; }
public double Target { get; set; }
public string Condition { get; set; }
}
public class EmptyTask : Task
{
}
[Serializable, MessagePackObject(true)]
public class Quest : Task
{
public string MilestoneId { get; set; }
}
[Serializable, MessagePackObject(true)]
public class Challenge : Task
{
public ChallengeType? ChallengeType { get; set; }
}
[Serializable, MessagePackObject(true)]
public class Mission : Task
{
public MissionType? MissionType { get; set; }
} var challenge = item.CustomData[TaskType.Challenge.ToString()];
var serialized = MessagePack.MessagePackSerializer.Serialize(challenge);
var bin = serialized.Utf8();
Debug.Log(bin);
var deserialized = MessagePackSerializer.Deserialize<Task>(serialized);MessagePack.MessagePackSerializationException : Failed to deserialize Chance.Domain.Game.Domain.Infrastructure.Task value.
----> MessagePack.MessagePackSerializationException : Unexpected msgpack code 218 (str 16) encountered.but the Debug shows this:
��{"InstanceId":null,"TaskType":null,"TaskLength":null,"ChallengeType":null,"TargetId":"#####","StatId":"####","Type":"######","Title":"#####","Description":"#######","Current":235,"Avg":0.34,"Target":45,"Condition":">="}so I can see the data is there, but can't figure out why it's throwing an error
@Kevdome3000 Your problem is that you're serializing a Challenge as a Challenge but then deserializing the base class of Task. You must serialize and deserialize with the same static type. So for example if you call Serialize<Challenge> (which you do in your code snippet, although the generic type argument is inferred by C#) then you must call Deserialize<Challenge>. Challenge is not a Union type and thus it is serialized without any union header.
But if you want/need to deserialize a Task, then you should serialize a Task, like so:
MessagePackSerializer.Serialize<Task>(challenge);Note that the object you're passing in is still an instance of type Challenge but the generic type argument you specify (explicitly!) is the base type -- the union type. This way, the Task formatter will be used and it will include the union header, and then go on to serialize the Challenge object itself (including the unique Challenge.ChallengeType field). Then when you deserialize a Task, the union header it requires will be there, it will recognize that the actual object type is Challenge and everything will be happy.
Hello, I am getting the following error (I am using Unity):
MessagePackSerializationException: Unexpected msgpack code 230 (negative fixint) encountered.
The error is caused when deserializing the following class:
[MessagePackObject]
public class SimpleMessage
{
[Key(0)]
public string message;
[Key(1)]
public int imageIndex = -1;
[Key(2)]
public bool hasGifID = false;
[Key(3)]
public MessageDate messageDate;
[Key(4)]
public uint sender;
}Here is MessageDate:
[MessagePackObject]
public class MessageDate
{
[Key(0)]
public int year;
[Key(1)]
public int month;
[Key(2)]
public int day;
[Key(3)]
public int hour;
[Key(4)]
public int minute;
[Key(5)]
public int second;
}@AArnott Thank you for responding.
Now I am getting code 226
MessagePackSerializationException: Unexpected msgpack code 226 (negative fixint) encountered.
MessagePack.MessagePackReader.ThrowInvalidCode (System.Byte code) (at Assets/Scripts/MessagePack/MessagePackReader.cs:942)
MessagePack.MessagePackReader.TryReadMapHeader (System.Int32& count) (at Assets/Scripts/MessagePack/MessagePackReader.cs:444)
MessagePack.MessagePackReader.ReadMapHeader () (at Assets/Scripts/MessagePack/MessagePackReader.cs:387)
MessagePack.Formatters.DictionaryFormatterBase`5[TKey,TValue,TIntermediate,TEnumerator,TDictionary].Deserialize (MessagePack.MessagePackReader& reader, MessagePack.MessagePackSerializerOptions options) (at Assets/Scripts/MessagePack/Formatters/DictionaryFormatter.cs:84)
MessagePack.Formatters.OperationDataFormatter1.Deserialize (MessagePack.MessagePackReader& reader, MessagePack.MessagePackSerializerOptions options) (at <2dda8b71668f401ab737424e74148583>:0)
MessagePack.MessagePackSerializer.Deserialize[T] (MessagePack.MessagePackReader& reader, MessagePack.MessagePackSerializerOptions options) (at Assets/Scripts/MessagePack/MessagePackSerializer.cs:246)
Rethrow as MessagePackSerializationException: Failed to deserialize OperationData value.
MessagePack.MessagePackSerializer.Deserialize[T] (MessagePack.MessagePackReader& reader, MessagePack.MessagePackSerializerOptions options) (at Assets/Scripts/MessagePack/MessagePackSerializer.cs:251)
MessagePack.MessagePackSerializer.Deserialize[T] (System.ReadOnlyMemory`1[T] buffer, MessagePack.MessagePackSerializerOptions options, System.Threading.CancellationToken cancellationToken) (at Assets/Scripts/MessagePack/MessagePackSerializer.cs:270)
Serializer.Deserialize[T] (System.ArraySegment`1[T] buffer) (at Assets/Scripts/Server-Client/ClientSetup.cs:154)This is what I have on ClientSetup.cs:154:
var resolver = MessagePack.Resolvers.CompositeResolver.Create(
MessagePack.Resolvers.StandardResolver.Instance,
MessagePack.Resolvers.ContractlessStandardResolver.Instance,
MessagePack.Resolvers.BuiltinResolver.Instance,
MessagePack.Resolvers.PrimitiveObjectResolver.Instance
);
Options = MessagePackSerializerOptions.Standard.WithResolver(resolver);
ArraySegment<byte> buffer //This is an ArraySegment that I get from the server after it is serialized
MessagePackSerializer.Deserialize<SimpleMessage>(buffer, Options);Here is serialization of the server:
MessagePackSerializer.Serialize<SimpleMessage>(simpleMessage, Options)
Hi all! If I understand correctly, the collection formatters only serialize the elements of the collection, not the fields / properties of the collection.
This is a simplified example:
[MessagePackFormatter(typeof(GenericCollectionFormatter<Email, ComplexEmailsList>))]
public class ComplexEmailsList : List<Email>
{
public ComplexEmailsList()
{
}
public string Organization;
}Will the Organization value be serialized and deserialized? And is it possible to do it?
For your special collection type, you'll probably want to write your own formatter. I imagine you'll have a doubly-nested array: one array for your overall
ComplexEmailsList type with two elements: [Organization, []] where the second element is the nested array for the actual collection elements.
Hi,
First of all, thank you for your great work developing this library, and happy new year ( a bit late but the first post in the new year).
I want to ask for opinions and recommendations about my use case. I have a custom type that consists of only primitives (int, double, string). I need to serialize a collection of such items to the file (~3 million, the serialized binary size is ~200 MB). The serialization happens not often, so the performance can be ignored. The most important part is the deserialization and I want to optimize it as much as possible (memory-wise and time-wise).
[MessagePackObject]
public class MyType
{
[Key(0)] public uint Field1;
[Key(1)] public double Field2;
[Key(2)] public string Field3;
//....
}
private static void CreateMessagePackFile()
{
var items = new List<MyType>(); // filled collection
using (var fs = new FileStream("items.bin", FileMode.Create, FileAccess.Write))
{
MessagePackSerializer.Serialize(fs, items);
}
}
private static void LoadMessagePackFile()
{
using var stream = File.OpenRead("items.bin");
var items = MessagePackSerializer.Deserialize<List<MyType>>(stream);
}Is this the most efficient way to do it? I need all items to be in memory because the collection will be constantly used and iterated.
Another question is about the lazy deserialization of fields. Some string fields are very rarely used. Is it possible somehow to deserialize the field only when it's accessed? Something like the ZeroFormatter had. I saw your comments about writing a custom formatter for this use case. Could you tell me is this really possible and maybe share some code snippet on how to do it?
Thanks.
3 replies
TypelessContractlessStandardResolver? the documentation seems to indicate that mpc works only with annotated types? in my scenario this is not really feasible, I got literally hundreds of types most of which has 1-2 dozen properties.
And if not, how do I use MessagePack.Generator (mpc.exe) with CustomResolver?
dotnet mpc -ei "System.Ulid" -i . -o "Assets\Scripts\Frameworks\MessagePack"
Project Compilation Start:CodeGenTemp
Project Compilation Complete:00:00:26.9081418
Method Collect Start
Fail in console app running on MessagepackCompiler.RunAsync
MessagePackCompiler.CodeAnalysis.MessagePackGeneratorResolveFailedException: Serialization Object must mark MessagePackObjectAttribute. type: global::Unity.Mathematics.float4x4I have written a CustomFormatter
public sealed class Float4x4Formatter : global::MessagePack.Formatters.IMessagePackFormatter<global::Unity.Mathematics.float4x4>And tried to use it via [MessagePackFormatter(typeof(ArrayFormatter<float4x4>))]
[Key(6)]
[MessagePackFormatter(typeof(ArrayFormatter<float4x4>))]
public float4x4[] LocalToWorld;But I am not sure how to make sure the CustomFormatter is discovered.
dotnet mpc -ei "System.Ulid,Unity.Physics.Aabb,Unity.Mathematics.float4x4" and register it manually. UlidMessagePackFormatter is a good example on how to do it.
fail: Microsoft.AspNetCore.Server.Kestrel[13]
Connection id "0HMH52VP9QPUC", Request id "0HMH52VP9QPUC:00000023": An unhandled exception was thrown by the application.
MessagePack.MessagePackSerializationException: Failed to deserialize Ultra.CommonFramework.Caching.Faster.FastCacheEntries`1[[System.Collections.Generic.List`1[[Ultra.CachingModule.Domain.CachedPageData, Ultra.CachingModule, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]], System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]] value.
---> System.NotSupportedException: Specified method is not supported.
at Microsoft.Extensions.Primitives.StringValues.System.Collections.Generic.ICollection<System.String>.Add(String item)
at MessagePack.Formatters.GenericCollectionFormatter`2.Add(TCollection collection, Int32 index, TElement value, MessagePackSerializerOptions options)
at MessagePack.Formatters.CollectionFormatterBase`4.Deserialize(MessagePackReader& reader, MessagePackSerializerOptions options)
at MessagePack.Formatters.KeyValuePairFormatter`2.Deserialize(MessagePackReader& reader, MessagePackSerializerOptions options)
at MessagePack.Formatters.ListFormatter`1.Deserialize(MessagePackReader& reader, MessagePackSerializerOptions options)
at MessagePack.Formatters.Ultra_CachingModule_Domain_CachedPageDataFormatter2.Deserialize(MessagePackReader& reader, MessagePackSerializerOptions options)
at MessagePack.Formatters.ListFormatter`1.Deserialize(MessagePackReader& reader, MessagePackSerializerOptions options)
at MessagePack.Formatters.Ultra_CommonFramework_Caching_Faster_FastCacheEntries`1\[\[System_Collections_Generic_List`1\[\[Ultra_CachingModule_Domain_CachedPageData\, Ultra_CachingModule\]\]\, System_Private_CoreLib\]\]Formatter1.Deserialize(MessagePackReader& reader, MessagePackSerializerOptions options)
at MessagePack.MessagePackSerializer.Deserialize[T](MessagePackReader& reader, MessagePackSerializerOptions options)
--- End of inner exception stack trace ---
at MessagePack.MessagePackSerializer.Deserialize[T](MessagePackReader& reader, MessagePackSerializerOptions options)
at MessagePack.MessagePackSerializer.Deserialize[T](ReadOnlyMemory`1 buffer, MessagePackSerializerOptions options, CancellationToken cancellationToken)[MessagePackObject]
public class CachedPageData
{
[Key(0)]
public CompressionType CompressionType { get; set; }
[Key(1)]
public byte[] ResponseData { get; set; }
[Key(2)]
public bool IsMinified { get; set; }
[Key(3)]
public List<KeyValuePair<string, StringValues>> Headers { get; set; }
}
\x84\xA3\x41\x53\x47\x00\xA3\x41\x53\x49\x93\x90\x90\x90\xA3\x41\x53\x53\x93\x00\x00\x00\xA3\x41\x52\x53\x93\x00\x00\x00
public class WSVJStateMsg {
public int ASG { get; set; } // ActiveSceneGroup
public int[,] ASI { get; set; } // ActiveSceneItems
public float[] ASS { get; set; } // ActiveScrollerSpeed
public float[] ARS { get; set; } // ActiveRotatorSpeed
}
I have a question about the SerializationConstructor.
Even if there is no constructor that accepts all parameters, it works without [SerializationConstructor].
This is an example that works.
[MessagePackObject(true)]
public record Test
{
public string A { get; init; } = string.Empty;
public string B { get; init; } = string.Empty;
public string C { get; init; } = string.Empty;
//[SerializationConstructor] no need
public Test(string a)
{
A = a;
B = "No B";
C = "No C";
}
}If param is of a different type, construct must be specified. In this case, using [SerializationConstructor] does not solve the problem.
This is an example that doesn't work.
[MessagePackObject(true)]
public record Test
{
public string A { get; init; } = string.Empty;
public string B { get; init; } = string.Empty;
public string C { get; init; } = string.Empty;
[SerializationConstructor]
public Test(int a)
{
if (a == 1) A = "A is 1";
}
}Why are you adding [SerializationConstructor]?