libmir/public

pirocks

@pirocks

The clCreateProgramWithIL error went away after manually selecting the most recent version of DerelictCL. I then got an error in the same place as the fprintf error, except for toStringz. Adding an import fixed that, but now I'm getting the following:

../../.dub/packages/dcompute-0.1.0/dcompute/source/dcompute/driver/error.d(139,32): Error: cannot implicitly convert expression `__lambda1` of type `void delegate(Status _status) @system` to `immutable(void delegate(Status) nothrow @nogc)`
../../.dub/packages/dcompute-0.1.0/dcompute/source/dcompute/driver/error.d(139,32): Error: cannot implicitly convert expression `__lambda1` of type `void delegate(Status _status) @system` to `immutable(void delegate(Status) nothrow @nogc)`

Nicholas Wilson

@thewilsonator

Hmm, that does seem odd, but that's my fault for not testing it properly.

pirocks

@pirocks

If relevant my ldc version is:

LDC - the LLVM D compiler (1.7.0git-958e58c):
  based on DMD v2.077.1 and LLVM 3.8.1
  built with DMD64 D Compiler v2.077.1
  Default target: x86_64-pc-linux-gnu
  Host CPU: broadwell

Nicholas Wilson

@thewilsonator

Unless you have need to use the D_betterC version try without it. That should "work" (note the @BUG@ just above). You will need to set onDriverError yourself because of it, see e.g.

https://github.com/libmir/dcompute/blob/master/source/dcompute/tests/main.d#L40

Thats not a compiler problem, thats me not testing properly. I really need to set up CI, but given the hardware required I haven't got around to it yet.

Thanks for pointing out the issues.

pirocks

@pirocks

Without -betterC I get:

Invalid bitcast
  %3 = bitcast float addrspace(1)* %res_arg to float*
Invalid bitcast
  %5 = bitcast float addrspace(1)* %x_arg to float*
Invalid bitcast
  %9 = bitcast float addrspace(1)* %y_arg to float*
LLVM ERROR: Broken function found, compilation aborted!

Nicholas Wilson

@thewilsonator

Hmm, can you try with an LLVM that is 3.9 or greater (e.g. from the LDC release page)?

Also what registered targets does the LDC you are using have (just below the output of ldc2 --version you posted)?

Sebastian Wilzbach

@wilzbach

I have two good news:
1) libmir/mir-algorithm#122 - examples on the mir docs will be runnable soon (see http://files.wilzbach.me/dlang/mir-algorithm/mir_ndslice_algorithm.html)
2) https://tour.dlang.org/tour/en/dub/mir - the tour will be finally moving to integrate mir (thought writing a good one-page summary might turn out to be challenging)

Ilya Yaroshenko

@9il

Some news:
PR "Tarjan graph algorithm" libmir/mir-algorithm#121
Issue "ndslice based API for dopt" henrygouk/dopt#6
Issue "Dcompute based backend for dopt" henrygouk/dopt#6

Ilya Yaroshenko

@9il

EDIT: Issue "ndslice based API for dopt" henrygouk/dopt#7

Ilya Yaroshenko

@9il

Mir Random PR https://github.com/libmir/mir-random/pull/73/files

kerdemdemir

@kerdemdemir

Hi Everybody I am trying to use multivariateNormalVar in mir.random.ndvariable for creating two separated data clusters (it is done in pyhton like: https://beckernick.github.io/logistic-regression-from-scratch/)

It doesn't work if I don't select resulting vector same size as sigma matrix

double[100] x1;   // Only works if the size is only 2 
Random* gen = threadLocalPtr!Random;
auto mu = [0.0, 0.0].sliced;
auto sigma = [1.0, 0.75, 0.75, 1].sliced(2,2);
auto rv = multivariateNormalVar(mu, sigma);
rv(gen, x1[]);

How can I use multivariateNormalVar to create data sets bigger than size 2

Shigeki Karita

@ShigekiKarita

how about this? (3dim random normal x 10)

/+dub.sdl:
dependency "lubeck" version="~>0.0.4"
dependency "numir" version="~>0.1.0"
libs "blas"
+/

import mir.ndslice : map, sliced, slicedField, ndarray;
import mir.random : threadLocalPtr, Random;
import mir.random.variable : NormalVariable;
import mir.random.algorithm : field;
import lubeck : mtimes;

import numir : alongDim;
import std.stdio;

void main() {
    Random* gen = threadLocalPtr!Random;
    auto mu = [0.0, 0.0, 0.0].sliced;
    auto sigma = [1.0, 0.75, 0.0,
                  0.75, 1.0, 0.75,
                  0.0, 0.75, 1].sliced(3,3);
    auto xs = field(gen, NormalVariable!double(0, 1)).slicedField(10, 3);
    auto x1 = xs.mtimes(sigma).alongDim!1.map!(x => x + mu).ndarray;
    x1.writeln; // 10 x 3 dim
}

https://run.dlang.io/gist/bd6dd9a2f6606151c707a6bdf6d0be36?compiler=ldc&args=-release
The multi normal random value is just an affine transformation of the standard normal random values. https://en.wikipedia.org/wiki/Multivariate_normal_distribution#Affine_transformation

for creating two separated data clusters

Oh you wanna create two clusters! you can take my example in https://github.com/ShigekiKarita/d-tree/blob/master/example/plot_boundary/app.d

it looks like this

kerdemdemir

@kerdemdemir

Thanks I will check it out for now I come up with this solution

double[num_dimensions] x;

double[num_observations] x1;
double[num_observations] x2;
Random* gen = threadLocalPtr!Random;

auto mu = [0.0, 0.0].sliced;
auto sigma = [1.0, 0.75, 0.75, 1].sliced(num_dimensions,num_dimensions);
auto rv = multivariateNormalVar(mu, sigma);


void GenerateAndAssign(R)( R range, int index )
{
    rv(gen, x[]);
    range[index..(index+2)] = x;    
}
iota(0, num_observations, 2).each!(  a=> GenerateAndAssign(x1[], a) );

mu = [1.0, 4.0].sliced;
rv = multivariateNormalVar(mu, sigma);
iota(0, num_observations, 2).each!(  a=> GenerateAndAssign(x2[], a) );

kerdemdemir

@kerdemdemir

Hi I am trying really hard to use desicionTree just as https://github.com/ShigekiKarita/d-tree/blob/master/example/plot_boundary/app.d

Unfortunately I couldn't made this work :

enum numberOfFeatures = 7; 
enum numberOfOutputs = 2;
auto dataMatrix = dataRaw.sliced( dataRaw.length/numberOfFeatures , numberOfFeatures ).slice;  
auto labelVector = labelRaw.sliced( ).slice;


auto gtree = ClassificationTree!gini(numberOfOutputs);
gtree.fit(dataMatrix, labelVector);

I spent too much time on it but couldn't succeed to compile

But meanwhile the code in the example compiles:

auto nsamples = 200;
auto ndim = 2;
auto xs = normal(nsamples, ndim).slice;
// TODO: add to numir.random
auto gen = Random(unpredictableSeed);
auto rv = BernoulliVariable!double(0.5);
auto ys = iota(nsamples).map!(i => cast(long) rv(gen)).slice;
foreach (i; 0 .. nsamples) {
    if (ys[i] == 1.0) { xs[i][] += 2.0; }
}

auto gtree = ClassificationTree!gini(2, 10);
gtree.fit(xs, ys);

I really can't see what I am doing wrong I printed all slices their shapes are allright

The final error I am getting is

../../.dub/packages/d-tree-0.0.1/d-tree/source/dtree/decision.d(101,17): Error: template mir.ndslice.slice.Slice!(cast(SliceKind)2, [1LU], double).Slice.opIndexUnary cannot deduce function from argument types !("++")(double), candidates are:
../../.dub/packages/mir-algorithm-0.9.3/mir-algorithm/source/mir/ndslice/slice.d(2710,18): mir.ndslice.slice.Slice!(cast(SliceKind)2, [1LU], double).Slice.opIndexUnary(string op)(size_t[packs[0]] _indexes...)

kerdemdemir

@kerdemdemir

Very sorry guys because the spam. I found my label vector was typed "double" which does not have ++ operator. I found the solution.

Shigeki Karita

@ShigekiKarita

I made a helper package mir-pybuffer for communication with numpy arrays in buffer protocol (sorry for posting again)

Bastiaan Veelo

@veelo

Hey there, I'm new here. Does mir support nd arrays over arbitrary intervals, i.e., with indices not starting at 0?

Shigeki Karita

@ShigekiKarita

how about iota([5], 1) that is [1, 2, 3, 4, 5]

Bastiaan Veelo

@veelo

Thanks, but that is just an array filled with values 1 .. 5, with indices running from 0 .. 4. I am looking for something to translate from Extended Pascal

type PositiveInteger = 1 .. MaxInt;
     Matrix (n, m: PositiveInteger) = array [1 .. n, 1 .. m] of Integer;
var mat: Matrix(4, 7);

where mat[1,1] would refer to the top left element. Or mat[1][1], that would be fine too.

Bastiaan Veelo

@veelo

mat[0,0] would be out of bounds.

Bastiaan Veelo

@veelo

I guess this is the closest I can come:

auto mat = slice!int(4, 7);
auto idx = IotaIterator!int(-1);

and use mat[idx[1], idx[1]] to address the top-left element.

Ilya Yaroshenko

@9il

Hi Bastiaan,
mir does not support arbitrary intervals for indexing.
Slightly more portable variant of your solution:

auto mat = slice!int(4, 7);
auto idx = IotaIterator!sizediff_t(-1);

Bastiaan Veelo

@veelo

Hi Ilya, thanks for your confirmation. Very impressive work, by the way!

Ilya Yaroshenko

@9il

Mir Optim preview - https://github.com/libmir/mir-optim . Also, Lubeck v0.1.0 has been released (new system libraries dependency configuration).

Lance Bachmeier

@bachmeil

Does mir-random work for parallel random number generation?

Nicholas Wilson

@thewilsonator

You should be able to use the PCG generators with different streams in parallel. Use https://github.com/libmir/mir-random/blob/master/source/mir/random/engine/pcg.d#L175 with the second argument different for each item of parallel work.

Lance Bachmeier

@bachmeil

Thanks!

Nicholas Wilson

@thewilsonator

No problem.

Yannick Koechlin

@yannick

is there a sample project somewhere or some docs how i would use mir-runtime ?

Ilya Yaroshenko

@9il

Ilya Yaroshenko, [21.11.18 10:14]
See the meson file and subprojects folder in the root of mir-algorithm. They import mir-core for example
Also add dips flags into your project

add_project_arguments([
    '-dip25',
    '-dip1000',
    '-dip1008',
    ], language: 'd')

jonathan MERCIER

@bioinfornatics

Hi, i would like to know if dcompute will evolve in order to use the library https://github.com/KhronosGroup/SPIRV-LLVM-Translator ? thanks

Nicholas Wilson

@thewilsonator

Yes ldc-developers/ldc#2935

Where communities thrive

People

Repo info

Activity