Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 05:07
    ItSkary commented #1273
  • 05:06
    ItSkary commented #1273
  • May 14 21:00
    PeterThornton4Ing opened #2185
  • May 14 21:00
    PeterThornton4Ing labeled #2185
  • May 14 18:43
    milos12345 labeled #2184
  • May 14 18:43
    milos12345 opened #2184
  • May 13 09:07
    seikosantana commented #1546
  • May 13 07:49
    dgodwin1175 edited #2183
  • May 13 07:32
    dgodwin1175 edited #2183
  • May 13 07:31
    dgodwin1175 opened #2183
  • May 13 07:31
    dgodwin1175 labeled #2183
  • May 12 08:48
    ArsenijK labeled #2181
  • May 12 08:48
    ArsenijK opened #2181
  • May 12 08:34
    ArsenijK commented #2169
  • May 10 22:39
    devmaestro1 commented #2180
  • May 10 22:37
    devmaestro1 commented #2180
  • May 09 16:15
    haggerty-william commented #1162
  • May 09 09:42
    devmaestro1 commented #2180
  • May 09 02:34
    devmaestro1 commented #2180
  • May 09 02:32
    devmaestro1 commented #2180
weirdyang
@weirdyang
No.
The JsonConvert is from NewtonSoft, could that be an issue?
Leonardo Nascimento
@lbnascimento
I'm not sure, could you show me how you're using objects of this class? How you're inserting, how they're being serialized/deserialized etc. Where is this error happening?
weirdyang
@weirdyang
   internal async Task<IEnumerable<LogMessage>> ProcessLogEvents(IEnumerable<LogEvent> logEvents, int clientId)
        {
            List<LogMessage> messages = new List<LogMessage>();
            foreach (var evt in logEvents)
            {
                var logMessage = new LogMessage()
                {
                    Level = evt.Level,
                    Message = evt.RenderMessage(),
                    TimeStamp = evt.Timestamp,
                    UserId = clientId
                };
                if (evt.Exception != null)
                {
                    logMessage.Exception = evt.Exception.ToString();
                }

                if (evt.Properties.Any())
                {

                    string complete = await Task.Run(() => Newtonsoft.Json.JsonConvert.SerializeObject(evt.Properties));
                    logMessage.Properties = LiteDB.JsonSerializer.Deserialize(complete).AsDocument;

                }
                messages.Add(logMessage);
            }
            return messages;
        }
Converting the log events into the model for storage
//retrieval

                return _repository.Query<T>()
                .OrderBy(x => x.Id, order: Query.Descending)
                .Where(predicate)
                .Skip(skip)
                .Limit(take)
                .ToList();
            return Ok(results);
the error happens when I return the ActionResult
the last step
Leonardo Nascimento
@lbnascimento
I see you're deserializing LogEvent.Properties with NewtonSoft.Json and serializing it right after with LiteDB.JsonSerializer, there's probably some form of incompatibility (I believe Newtonsoft serializes dates as strings, while LiteDB serializes them to a custom document)
weirdyang
@weirdyang
but there's no datetime in the properteis.
properties*
Leonardo Nascimento
@lbnascimento
What type is it?
weirdyang
@weirdyang
strings
{{"custom":{"TypeTag":null,"Properties":[{"Name":"message","Value":{"Value":"Yesenia-orange"}}]},"SourceContext":{"Value":"Sample.Program"},"Application":{"Value":"Sample"}}}

{{"SourceContext":{"Value":"Sample.Program"},"Application":{"Value":"Sample"}}}
Leonardo Nascimento
@lbnascimento
I believe Newtonsoft is getting lost trying to serialize a BsonValue
weirdyang
@weirdyang
ahah oh no
Leonardo Nascimento
@lbnascimento
Could you try something like: logMessage.Properties = BsonMapper.Global.Serialize(evt.Properties).AsDocument?
weirdyang
@weirdyang
Ok
Leonardo Nascimento
@lbnascimento
Also, if it makes your class easier to use, you can make LogMessage.Properties to be a Dictionary<string,string> and the BsonMapper will serialize it to a document when inserting
weirdyang
@weirdyang

I tried that but it ends up becoming

{{"custom": "{"TypeTag":null,"Properties":[{"Name":"message","Value":{"Value":"Yesenia-orange"}}]},""}

i.e the value of custom is a string, and I cant access the properties.
using a bsondocument, i can do Properties.Custom.TypeTag
if I use dictionary<string, string> i need to do a json(properties.custom) then access the properties
weirdyang
@weirdyang
Newtonsoft.Json.JsonSerializationException: Error getting value from 'AsBoolean' on 'LiteDB.BsonValue'. ---> System.InvalidCastException: Unable to cast object of type 'System.String' to type 'System.Boolean'. at lambda_method(Closure , Object )
Leonardo Nascimento
@lbnascimento
Are you still calling the NewtonSoft serializer?
weirdyang
@weirdyang
                if (evt.Properties.Any())
                {

                    logMessage.Properties = BsonMapper.Global.Serialize(evt.Properties).AsDocument;

                }
                messages.Add(logMessage);
I'm using a new database too
Leonardo Nascimento
@lbnascimento
In which line is this exception being thrown?
weirdyang
@weirdyang

When the controller action returns the result in

Ok(result)

//retrieval

                return _repository.Query<T>()
                .OrderBy(x => x.Id, order: Query.Descending)
                .Where(predicate)
                .Skip(skip)
                .Limit(take)
                .ToList();
           var results = getLogMessages();
            return Ok(results);
Leonardo Nascimento
@lbnascimento
Well, the Newtonsoft serializer is being called somewhere (maybe inside this Ok method), and it is failing to serialize BsonValues (it's trying to cast string to DateTime, string to Boolean etc.)
weirdyang
@weirdyang
yea, the mvc builder uses it. the weird thing is none of the propeties have boolean or datetime
Leonardo Nascimento
@lbnascimento
The class BsonValue has properties for all valid bson types: AsBoolean, AsInt32, AsDateTime...
These properties obviously only work if the internal value of that BsonValue matches the property, it will throw otherwise
Newtonsoft is probably trying to serialize every property in the BsonValue (BsonDocument inherits from BsonValue) and that's why it's throwing
weirdyang
@weirdyang
Thansk for the hint @lbnascimento :)
    public class BsonValueConverter : Newtonsoft.Json.JsonConverter<BsonValue>
    {
        public override BsonValue ReadJson(Newtonsoft.Json.JsonReader reader, Type objectType, [AllowNull] BsonValue existingValue, bool hasExistingValue, Newtonsoft.Json.JsonSerializer serializer)
        {
            throw new NotImplementedException();
        }

        public override void WriteJson(Newtonsoft.Json.JsonWriter writer, [AllowNull] BsonValue value, Newtonsoft.Json.JsonSerializer serializer)
        {
            writer.WriteValue(value.ToString());
        }
    }
Tim Moore
@tjmoore

Hi, I'm working on a big project refactor and part of that involves abstracting the data model POCO classes from the data access layer. This allows for multiple database options, possibly MongoDB at least, and removing dependencies on the database assemblies in the rest of the project when using the data model classes.

I've been changing ObjectId properties to string and using Guids for ID instead, although these may not be as efficient as an ObjectId. I have a problem though with this approach on deserialising existing databases as ObjectId doesn't cast to string.

Is there a recommended way around this? I suppose I could register a mapping of ObjectId to string, but would that prevent me serialising an ObjectId if I really wanted to?

Alternatively a generic solution that still uses ObjectId, good for LiteDB and MongoDB (as they have virtually the same thing I believe), but I need that abstraction with no dependency on LiteDB in the model and dependent non-DB code needs access to an ID as a string.

If any of that makes sense.

Leonardo Nascimento
@lbnascimento
@tjmoore While I believe it may be possible to create a context-aware mapping for ObjectId (using the UserVersion pragma or something like that), I think it would be janky.
I believe your best bet is to simply convert the ObjectId properties into strings both in your classes and in your datafiles.
Leonardo Nascimento
@lbnascimento
@tjmoore I came up with this piece of code:
var collectionNames = db.GetCollectionNames().ToList();
db.CheckpointSize = 0;

foreach (var collectionName in collectionNames)
{
    db.BeginTrans();

    var tmpColName = collectionName + "_tmp";
    var col = db.GetCollection(collectionName);
    var tmpCol = db.GetCollection(tmpColName);

    var indexes = db.Execute($"select $ from $indexes where collection = '{collectionName}'").ToList();

    foreach(var index in indexes)
    {
        tmpCol.EnsureIndex(index["name"].AsString, index["expression"].AsString, index["unique"].AsBoolean);
    }

    foreach(var doc in col.FindAll())
    {
        var newDoc = new BsonDocument();
        foreach(var kvp in doc)
        {
            if(kvp.Value.IsObjectId)
            {
                newDoc[kvp.Key] = kvp.Value.AsObjectId.ToString();
            }
            else
            {
                newDoc[kvp.Key] = kvp.Value;
            }
        }

        tmpCol.Insert(newDoc);
    }

    db.Commit();

    db.DropCollection(collectionName);
    db.RenameCollection(tmpColName, collectionName);

    db.Checkpoint();
}

db.CheckpointSize = 1000;
db.Rebuild();
It processes all collections in a datafile, transforming ObjectIds into strings. I ran some tests and it seems to run fine - of course, you should do your own testing, to ensure that it works properly and that it fits your needs.
If you have a distributed aplication (like a mobile app), you can use db.UserVersion to determine whether the file has to be processed or not
Tim Moore
@tjmoore
@lbnascimento Cool, thanks for that. I'll give it a go. It does seem like the simplest way to do it.
Nate Komodo
@NateKomodo
How does one use UpdateMany? the example is to use an anonymous type, but i get a type conversion error when i do this
db.GetCollection<Setting>("settings").UpdateMany(s => new { setting.Name, setting.Value }, s => s.Name == setting.Name);
The example is Extend expression must return an anonymous class to be merge with entities. Eg: col.UpdateMany(x => new { Name = x.Name.ToUpper() }, x => x.Age > 10)
weirdyang
@weirdyang
col.UpdateMany(x => new Customer { Name = x.Name.ToUpper(), Salary: 100 }, x => x.Name == "John")
image.png
Leonardo Nascimento
@lbnascimento
@NateKomodo I don't know which version you're running, but the example is probably wrong. In the latest version, it is correct: col.UpdateMany(x => new Customer { Name = x.Name.ToUpper(), Salary: 100 }, x => x.Name == "John")
Mark Gerrits
@skinny
Hi all! I have been using v5 of LiteDB with pleasure but I'm running into an issue with enums. By default enum values are serialized as strings but that won't work with Linq so I added BsonMapper.Global.EnumAsInteger = true; to my application. That solved the problem with my "first" enum but introduced one for another enum I use. This enum is defined on my class as public DocumentType[] SupportedDocumentTypes { get; set; }