Hi, I've been hacking on ml-net in the last couple days in order to get 2 requirements working:
- a custom IDataView constructed from c# IEnumerable<T> where T: struct.
- Add support for c# enums types, which turns into ml.net key types if base enum's integer is unsigned
I'm trying to acheive this without forking ml-net itself which is not an easy task, as I have to constantly avoid ml-net's default c# <-> DataViewCollumn auto binder which is not compatible with my requirements.
My question is the following: Is it possible to customize the c# to IDataView binder? I'd like to set a global Transform.CustomMapping<InputT, OutputT> but without InputT and convert things manually.
Can you explain a bit more on why you'd want/need to do this?
I'm storing all the training data in a fixed compact memory space, which is accessed using an index with getters/setters, it would be wasteful to convert each entry to a GC instance
i'd assume inside ML.NET's code would be the easiest to accomplish this. but it may be possible without forking
it would be nice to be able to override the ML's binder globally, as I would like to implement some specific convention based mapping without attributes. but if the first problem is solved (structs) then I wouldn't mind wrapping my structs into structs that conforms ML's attributes
struct first, to get an understanding of the different pieces, before trying to implement it for a general IEnumerable<T> where T : struct. You seem to have a specific example in mind (I'm storing all the training data in a fixed compact memory space, which is accessed using an index with getters/setters), so I'd try doing it for that struct specifically
System.ArgumentOutOfRangeException : Schema mismatch for score column 'Score': expected vector of two or more items of type Single, got Vector<Single, 1>
Hi all! I have a question which perhaps someone could point me in the right direction.
I am trying to write a custom IDataView to do Transform on. The issue I am encounting is when
attempting to extract the GetColumn<float[]>("Score") column.
ArgumentOutOfRangeException: requested column not active. Arg_ParamName_NameMy question is how do I retrieve to Score column? It appears to be in the OutputSchema I have however do not specific implementation for retrieving it in my DataView.
IDataView.GetRowCursor method. When you are creating a cursor for the underlying IDataView that you are reading the "Score" column from, make sure to pass the "Score" column in to GetRowCursor
2 replies
var textColumn = transformedDataView.Schema["Text"];
var tokensColumn = transformedDataView.Schema["TokenizedText"];
using (var cursor = transformedDataView.GetRowCursor(
new[] { textColumn, tokensColumn }))
1 reply