eaplatanios/tensorflow_scala

Nazar

@nazarblch

@cosmir17 , you are right, there is no dim, use myMatrix.rank instead

in package org.platanios.tensorflow.api.tensors.ops there is argmax function you need argmax(myMatrix, null)

Nazar

@nazarblch

or argmax(myMatrix, Tensor[Int](0 until myMatrix.rank) ) , I hope at least one will work

Sean Kwak

@cosmir17

Thanks, Nazar. I wrote the codes you wrote and there is no compilation error on my IDE. I agree that one of them will work (I need to write some test code). Nonetheless, unlike numpy's argmax(oneParameter), this 'argmax' always returns a Tensor object. I need to unwrap Tensor. is there such api? like Option handling in Scala?

forgot to type @, @nazarblch

Nazar

@nazarblch

numpy makes a bad thing in this case) for one element tensor there is .scalar method, if there is more than one element -- unwrap it with .entriesIterator method

Sean Kwak

@cosmir17

Thank you, @nazarblch. This is truly wonderful :) Now, the rest of case handling will be ok as Scala has rich case handling APIs (pattern matching & Option).

JOSE FARFAN

@jose1003

Hi, i am using the MNIST example from https://github.com/eaplatanios/tensorflow_scala/blob/master/modules/examples/src/main/scala/org/platanios/tensorflow/examples/MNIST.scala

and I am getting this error

but, anyway, I can compile and run the process, but I cannot remove the hint from the compiler

Sean Kwak

@cosmir17

Quick question, How can I translate the following numpy code to Scala_tensorflow?
current_state = np.asmatrix(np.hstack((a[i:i+number], b, c))) <=this part
self.sess.run(q, feed_dict={x: current_state}) //this is already done

Sean Kwak

@cosmir17

I came up with the following workaround. Can anyone verify if it makes sense?
val a: List[Float] = ???
val aTensor = Tensor[Float](a).reshape(Shape(-1, a.size))
val newState: Tensor[Float] = concatenate(Seq(aTensor, b, c), null)

Sean Kwak

@cosmir17

Hi, I want to produce a tensor(2d-array) object from an existing tensor variable with a specific index value modified.
Any advice for me? (copying with modification is more desirable than assignment)
val myTensor: Tensor[Float] = ???
val floatValue: Float = 1.3f

assign to 'myTensor(index1)(index2) <= floatValue' OR
copy and modify a specific coordinate value, 'myTensor.copy(myTensor(index1)(index2) = floatValue)'

Matthew Moloney

@moloneymb

Hi folks, perhaps I missed it, but how would I go about saving a checkpoint during a training run?

Sean Kwak

@cosmir17

@moloneymb I found ...CheckpointSaver(summariesDir, StepHookTrigger(1000))),.......
http://platanios.org/tensorflow_scala/

There is 'tf.saver(....)' method. I couldn't find an actual example. If this doesn't help, please shout for other's help :)

Matthew Moloney

@moloneymb

@cosmir17 My goodness I'm blind... I don't know how I missed it. Thanks

Sean Kwak

@cosmir17

@moloneymb My pleasure 👍Speaking of which, do you know how to extract training data from session? rather than saving to a file or extracting individual node weight, e.g. sess.run(W1)?

Matthew Moloney

@moloneymb

@cosmir17 I do not, I'm just starting to play with this

Anthony Platanios

@eaplatanios

@moloneymb @cosmir17 You can see an example of how to use the tf.Saver directly in the implementation of the CheckpointSaver hook.

@cosmir17 What do you mean by "extract training data from session"?

Sean Kwak

@cosmir17

@eaplatanios
I found that there is a way to retrieve updated weight values.
https://www.quora.com/How-do-I-print-weights-of-a-fully-connected-neural-network-in-tensorflow
This was for Python tensorflow but I tested and I saw that it worked with the Scala version too.
//train your network here
sess.run(optimizer, feed_dict={x:data, y:labels}
print("layer 1 weights:", sess.run(W1))
print("layer 2 weights:", sess.run(W2))

I was trying to persist training information (using akka persistence) and it seems only 'session' contains relevant information.
https://github.com/cosmir17/Scala-akka-tensorflow-sharetrade-helper/blob/1313169c97c2d6d716911e03ba66d9cea7aed4ab/src/main/scala/QDecisionPolicyActor.scala#L68

I don't want to serialise a file created locally by Tensorflow. Once I get these training information in runtime. I can do something like, https://doc.akka.io/docs/akka/2.5.4/java/distributed-data.html (akka cluster)

To note that my personal project is for fun but it will be so interesting to experiment Tensorflow and Akka together in distributed environment.

Sean Kwak

@cosmir17

@moloneymb me too :)

drivenbyentropy

@drivenbyentropy

Hi, was wondering if the TFRecordWriter.write function is thread-safe. Does anyone have experience with this? Thanks!

Haiping

@Oceania2018

@moloneymb Can Tensorflow_scala train linear regression model?

Matthew Moloney

@moloneymb

@Oceania2018 it must be able to

evanthomas

@evanthomas

I haven't been using scala/tf for a while and now I'm just porting my code from 0.2 to 0.4. I'm getting this exception:

Exception in thread "main" org.platanios.tensorflow.jni.FailedPreconditionException: Input 1 ('Transpose') for 'Conv2D' was not previously added to ShapeRefiner.
    at org.platanios.tensorflow.jni.Op$.finish(Native Method)
    at org.platanios.tensorflow.api.ops.Op$Builder.$anonfun$build$1(Op.scala:2646)
    at org.platanios.tensorflow.api.utilities.package$.using(package.scala:31)
    at org.platanios.tensorflow.api.ops.Op$Builder.build(Op.scala:2589)
    at org.platanios.tensorflow.api.ops.NN.conv2D(NN.scala:1240)
    at org.platanios.tensorflow.api.ops.NN.conv2D$(NN.scala:1219)
    at org.platanios.tensorflow.api.package$tf$.conv2D(package.scala:294)
    at co.mumbler.models.layers.ConvLayer.apply(Layers.scala:47)
    at co.mumbler.main.Style$.$anonfun$net_preloaded$1(Style.scala:176)
    at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:237)
    at scala.collection.immutable.List.foreach(List.scala:392)
    at scala.collection.TraversableLike.map(TraversableLike.scala:237)
    at scala.collection.TraversableLike.map$(TraversableLike.scala:230)
    at scala.collection.immutable.List.map(List.scala:298)
    at co.mumbler.main.Style$.net_preloaded(Style.scala:174)
    at co.mumbler.main.Style$.$anonfun$apply$1(Style.scala:35)
    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
    at org.platanios.tensorflow.api.ops.Op$.createWith(Op.scala:2043)
    at org.platanios.tensorflow.api.ops.package$API.createWith(package.scala:120)
    at org.platanios.tensorflow.api.ops.package$API.createWith$(package.scala:114)
    at org.platanios.tensorflow.api.package$tf$.createWith(package.scala:294)
    at co.mumbler.main.Style$.apply(Style.scala:32)
    at co.mumbler.main.Main$.main(Main.scala:193)
    at co.mumbler.main.Main.main(Main.scala)

Anthony Platanios

@eaplatanios

@drivenbyentropy The TFRecordWriter uses a buffered output stream and I don't believe that's thread-safe. However, it'd be pretty simple to make a thread-safe implementation by modifying the original implementation here.

@Oceania2018 Yes, it can. You can simply modify the model from the MNIST example in the readme file to just use a single linear layer and an L2 loss function and that would be equivalent to linear regression.

@evanthomas Could you please share a code example?

evanthomas

@evanthomas

I’m sometimes developing on linux and sometimes on OSX. I currently manage this by manually changing the classifier in build.sbt:

//      "org.platanios"         %% "tensorflow"      % "0.4.1" classifier "linux-gpu-x86_64",
      "org.platanios"         %% "tensorflow"      % "0.4.1" classifier “darwin-cpu-x86_64”,

but this is not ideal. Can I make a “universal” build.sbt? On a related note is possible to assemble a univeral jar that will run on both systems?

evanthomas

@evanthomas

@eaplatanios I’ve opened an issue with a simple code that reproduces the exception I’m getting.

evanthomas

@evanthomas

I need to make my code generic so I need to demonstrate to the API that the types I’m supporting are supported by the API. So I find myself doing things like:

def apply[T: TF: IsDecimal: IsReal](m: MLStructure): Layer[T]

So first a basic scala question (sorry). I can find the syntax def f[A: B]() documented. In particuar, the syntax Class1: Class2. Does it mean the Class1 must implement Class2? How does it differ to A <: B? It’s hard to google for it.

Secondly, is this how the API should be used? The name TF strikes me as being private. If not, how should make my code generic?

evanthomas

@evanthomas

I'd like to try FLOAT16 which I think should work nicely for my application but I notice the Optimizer interface seems to only support Float and Double.

def minimize[T: TF : IsFloatOrDouble, I: LongDefault : TF : IsIntOrLong]

Is FLOAT16 coming soon or is there a work around?

Sean Kwak

@cosmir17

I have a simple question that anyone can answer. How can I save Session to file(s), one or two lines of codes, in the standard way?

The only examples I found are the following. It appears that they alone actually don't save training data to files, and working on their own. They have to be declared in some API (e.g. tf.learn.InMemoryEstimator(...))

SummarySaver(summariesDir, tf.learn.StepHookTrigger(100))
CheckpointSaver(summariesDir, tf.learn.StepHookTrigger(1000))

Once I figure out how to save 'session', I am going to do the following, paths are going to be substituted from the file paths created during training process. Is it the right way? or is there a simpler and more standard way?

   val savePath = Paths.get(".../model-store-python")
   val metaPath = Paths.get(".../model-store-python/my-model.meta")
   val metaGraphDef = MetaGraphDef.parseFrom(new BufferedInputStream(new FileInputStream(metaPath.toFile)))
   val saver = tf.Saver.fromMetaGraphDef(metaGraphDef)
   saver.restore(sess, tf.Saver.latestCheckpoint(savePath, "my-model.meta").get)

Codie (蒂诺)

@lasso-codie

hey, I've got two questions. Before diving deeper into the code, is there a limit to the memory of a tensor (e.g. java arrays can have only 2GB) and is there also a possibility to do some memory hacking by using strides?

Could not see anything for strides in the API docs

Lakshya KD

@LakshKD

Hey, I was trying running some examples using tensorflow for scala and it is working fine with Scala SBT project but I want to do it in Scala Gradle project and the issue which I was facing is as follows: I was able to add the dependency in gradle file but when importing tensorflow for scala in scala code it is showing error. I was trying importing like this: import org.platanios.tensorflow.api.ops.io.data.Dataset
import org.platanios.tensorflow.api.types.DataType
import org.platanios.tensorflow.api.{tf, _} Can someone please help showing some small demo that how it can be done in Gradle. Thanks

@eaplatanios

Mandar Chandorkar

@mandar2812

Hi @/all Just wanted to ask all Tensorflow scala users if they have used the tf.data.datasetFromOutputSlices() method for creating a TF Dataset. I just reported issue #165 which details problems with using this method in a practical example. Has any one faced something similar?

Haiping

@Oceania2018

Jupyter SciSharp.webm

Interesting demo. run tensorflow in Jupyter notebook and based on .NET.

jxtps

@jxtps

Would it make sense for TF4S to use JavaCPP Presets for TensorFlow? See eaplatanios/tensorflow_scala#162

Ashwin Ravishakar

@ashwinravishankar

Hello @/all. I used <TensorObject>.toProto to convert to TensorProto object. Am unable to currently find an equivalent way to convert back a TensorProto object to a Tensor. Any help please?

Aggelos Biboudis

@biboudis

Why is ControlFlow package private for api? What is the proposed way to use a whileLoop directly?

damien

@damienstanton

is this repo on the sonatype snapshot resolver? I cannot seem to get sbt to locate either the prebuilt binaries nor the other one

[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[warn]     ::          UNRESOLVED DEPENDENCIES         ::
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[warn]     :: org.platanios#tensorflow;0.4.0: not found

disregard - I fixed it. There are a couple of typos in the documentation, so I'll open a PR to get them fixed :thumbsup:

damien

@damienstanton

essentially the sbt coordinates are wrong (no implicit scalaVersion underscore), and they point to an old version (0.4.0 instead of 0.4.1).

Иван Сердюк

@oceanfish81_twitter

https://fwdays.com/event/data-science-fwdays-2019 - September 7;
https://aiconf.com.ua/en/home/ - September 15;
https://conference.data-science.com.ua/ - October 19;
https://devoxx.com.ua/ - November 1-2;
https://aibooster.com.ua/cv2019/ - November 9;
https://www.ukraine.buildstuff.events/ - November 18-19

who is able to give a talk, at one of these conferences?

Where communities thrive

People

Repo info

Activity