Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Aug 13 20:15
    sbrunk commented #161
  • Aug 11 07:54
    mandar2812 commented #161
  • Aug 11 00:41
    eaplatanios commented #161
  • Aug 02 18:33
    nazarblch commented #162
  • Jul 29 15:28
    jxtps commented #171
  • Jul 27 04:40
    jxtps opened #172
  • Jul 27 04:38
    jxtps opened #171
  • Jul 27 04:27
    jxtps commented #162
  • Jul 26 23:50
    MiyazawaKohei opened #170
  • Jul 26 23:32
    MiyazawaKohei opened #169
  • Jul 26 18:16
    mandar2812 commented #161
  • Jul 25 03:49
    eaplatanios commented #161
  • Jul 17 16:17
    ashwinravishankar edited #168
  • Jul 16 20:12
    sbrunk commented #161
  • Jul 16 20:11
    sbrunk commented #161
  • Jul 16 10:58
    DirkToewe commented #161
  • Jul 15 07:57
    mandar2812 commented #161
  • Jul 14 18:37
    eaplatanios commented #161
  • Jul 13 12:52
    DirkToewe commented #161
  • Jul 13 05:38
    MiyazawaKohei commented #166
Nazar
@nazarblch
@cosmir17 , you are right, there is no dim, use myMatrix.rank instead
in package org.platanios.tensorflow.api.tensors.ops there is argmax function you need argmax(myMatrix, null)
Nazar
@nazarblch
or argmax(myMatrix, Tensor[Int](0 until myMatrix.rank) ) , I hope at least one will work
Sean Kwak
@cosmir17
Thanks, Nazar. I wrote the codes you wrote and there is no compilation error on my IDE. I agree that one of them will work (I need to write some test code). Nonetheless, unlike numpy's argmax(oneParameter), this 'argmax' always returns a Tensor object. I need to unwrap Tensor. is there such api? like Option handling in Scala?
forgot to type @, @nazarblch
Nazar
@nazarblch
numpy makes a bad thing in this case) for one element tensor there is .scalar method, if there is more than one element -- unwrap it with .entriesIterator method
Sean Kwak
@cosmir17
Thank you, @nazarblch. This is truly wonderful :) Now, the rest of case handling will be ok as Scala has rich case handling APIs (pattern matching & Option).
and I am getting this error
image.png
but, anyway, I can compile and run the process, but I cannot remove the hint from the compiler
Sean Kwak
@cosmir17
Quick question, How can I translate the following numpy code to Scala_tensorflow?
current_state = np.asmatrix(np.hstack((a[i:i+number], b, c))) <=this part
self.sess.run(q, feed_dict={x: current_state}) //this is already done
Sean Kwak
@cosmir17
I came up with the following workaround. Can anyone verify if it makes sense?
val a: List[Float] = ???
val aTensor = Tensor[Float](a).reshape(Shape(-1, a.size))
val newState: Tensor[Float] = concatenate(Seq(aTensor, b, c), null)
Sean Kwak
@cosmir17
Hi, I want to produce a tensor(2d-array) object from an existing tensor variable with a specific index value modified.
Any advice for me? (copying with modification is more desirable than assignment)
val myTensor: Tensor[Float] = ???
val floatValue: Float = 1.3f
  1. assign to 'myTensor(index1)(index2) <= floatValue' OR
  2. copy and modify a specific coordinate value, 'myTensor.copy(myTensor(index1)(index2) = floatValue)'
Matthew Moloney
@moloneymb
Hi folks, perhaps I missed it, but how would I go about saving a checkpoint during a training run?
Sean Kwak
@cosmir17

@moloneymb I found ...CheckpointSaver(summariesDir, StepHookTrigger(1000))),.......
http://platanios.org/tensorflow_scala/

There is 'tf.saver(....)' method. I couldn't find an actual example. If this doesn't help, please shout for other's help :)

Matthew Moloney
@moloneymb
@cosmir17 My goodness I'm blind... I don't know how I missed it. Thanks
Sean Kwak
@cosmir17
@moloneymb My pleasure 👍Speaking of which, do you know how to extract training data from session? rather than saving to a file or extracting individual node weight, e.g. sess.run(W1)?
Matthew Moloney
@moloneymb
@cosmir17 I do not, I'm just starting to play with this
Anthony Platanios
@eaplatanios
@moloneymb @cosmir17 You can see an example of how to use the tf.Saver directly in the implementation of the CheckpointSaver hook.
@cosmir17 What do you mean by "extract training data from session"?
Sean Kwak
@cosmir17

@eaplatanios
I found that there is a way to retrieve updated weight values.
https://www.quora.com/How-do-I-print-weights-of-a-fully-connected-neural-network-in-tensorflow
This was for Python tensorflow but I tested and I saw that it worked with the Scala version too.
//train your network here
sess.run(optimizer, feed_dict={x:data, y:labels}
print("layer 1 weights:", sess.run(W1))
print("layer 2 weights:", sess.run(W2))

I was trying to persist training information (using akka persistence) and it seems only 'session' contains relevant information.
https://github.com/cosmir17/Scala-akka-tensorflow-sharetrade-helper/blob/1313169c97c2d6d716911e03ba66d9cea7aed4ab/src/main/scala/QDecisionPolicyActor.scala#L68

I don't want to serialise a file created locally by Tensorflow. Once I get these training information in runtime. I can do something like, https://doc.akka.io/docs/akka/2.5.4/java/distributed-data.html (akka cluster)

To note that my personal project is for fun but it will be so interesting to experiment Tensorflow and Akka together in distributed environment.

Sean Kwak
@cosmir17
@moloneymb me too :)
drivenbyentropy
@drivenbyentropy
Hi, was wondering if the TFRecordWriter.write function is thread-safe. Does anyone have experience with this? Thanks!
Haiping
@Oceania2018
@moloneymb Can Tensorflow_scala train linear regression model?
Matthew Moloney
@moloneymb
@Oceania2018 it must be able to
evanthomas
@evanthomas
I haven't been using scala/tf for a while and now I'm just porting my code from 0.2 to 0.4. I'm getting this exception:
Exception in thread "main" org.platanios.tensorflow.jni.FailedPreconditionException: Input 1 ('Transpose') for 'Conv2D' was not previously added to ShapeRefiner.
    at org.platanios.tensorflow.jni.Op$.finish(Native Method)
    at org.platanios.tensorflow.api.ops.Op$Builder.$anonfun$build$1(Op.scala:2646)
    at org.platanios.tensorflow.api.utilities.package$.using(package.scala:31)
    at org.platanios.tensorflow.api.ops.Op$Builder.build(Op.scala:2589)
    at org.platanios.tensorflow.api.ops.NN.conv2D(NN.scala:1240)
    at org.platanios.tensorflow.api.ops.NN.conv2D$(NN.scala:1219)
    at org.platanios.tensorflow.api.package$tf$.conv2D(package.scala:294)
    at co.mumbler.models.layers.ConvLayer.apply(Layers.scala:47)
    at co.mumbler.main.Style$.$anonfun$net_preloaded$1(Style.scala:176)
    at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:237)
    at scala.collection.immutable.List.foreach(List.scala:392)
    at scala.collection.TraversableLike.map(TraversableLike.scala:237)
    at scala.collection.TraversableLike.map$(TraversableLike.scala:230)
    at scala.collection.immutable.List.map(List.scala:298)
    at co.mumbler.main.Style$.net_preloaded(Style.scala:174)
    at co.mumbler.main.Style$.$anonfun$apply$1(Style.scala:35)
    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
    at org.platanios.tensorflow.api.ops.Op$.createWith(Op.scala:2043)
    at org.platanios.tensorflow.api.ops.package$API.createWith(package.scala:120)
    at org.platanios.tensorflow.api.ops.package$API.createWith$(package.scala:114)
    at org.platanios.tensorflow.api.package$tf$.createWith(package.scala:294)
    at co.mumbler.main.Style$.apply(Style.scala:32)
    at co.mumbler.main.Main$.main(Main.scala:193)
    at co.mumbler.main.Main.main(Main.scala)
Anthony Platanios
@eaplatanios
@drivenbyentropy The TFRecordWriter uses a buffered output stream and I don't believe that's thread-safe. However, it'd be pretty simple to make a thread-safe implementation by modifying the original implementation here.
@Oceania2018 Yes, it can. You can simply modify the model from the MNIST example in the readme file to just use a single linear layer and an L2 loss function and that would be equivalent to linear regression.
@evanthomas Could you please share a code example?
evanthomas
@evanthomas

I’m sometimes developing on linux and sometimes on OSX. I currently manage this by manually changing the classifier in build.sbt:

//      "org.platanios"         %% "tensorflow"      % "0.4.1" classifier "linux-gpu-x86_64",
      "org.platanios"         %% "tensorflow"      % "0.4.1" classifier “darwin-cpu-x86_64”,

but this is not ideal. Can I make a “universal” build.sbt? On a related note is possible to assemble a univeral jar that will run on both systems?

evanthomas
@evanthomas
@eaplatanios I’ve opened an issue with a simple code that reproduces the exception I’m getting.
evanthomas
@evanthomas

I need to make my code generic so I need to demonstrate to the API that the types I’m supporting are supported by the API. So I find myself doing things like:

def apply[T: TF: IsDecimal: IsReal](m: MLStructure): Layer[T]

So first a basic scala question (sorry). I can find the syntax def f[A: B]() documented. In particuar, the syntax Class1: Class2. Does it mean the Class1 must implement Class2? How does it differ to A <: B? It’s hard to google for it.

Secondly, is this how the API should be used? The name TF strikes me as being private. If not, how should make my code generic?

evanthomas
@evanthomas

I'd like to try FLOAT16 which I think should work nicely for my application but I notice the Optimizer interface seems to only support Float and Double.

def minimize[T: TF : IsFloatOrDouble, I: LongDefault : TF : IsIntOrLong]

Is FLOAT16 coming soon or is there a work around?

Sean Kwak
@cosmir17

I have a simple question that anyone can answer. How can I save Session to file(s), one or two lines of codes, in the standard way?

The only examples I found are the following. It appears that they alone actually don't save training data to files, and working on their own. They have to be declared in some API (e.g. tf.learn.InMemoryEstimator(...))

SummarySaver(summariesDir, tf.learn.StepHookTrigger(100))
CheckpointSaver(summariesDir, tf.learn.StepHookTrigger(1000))

Once I figure out how to save 'session', I am going to do the following, paths are going to be substituted from the file paths created during training process. Is it the right way? or is there a simpler and more standard way?

   val savePath = Paths.get(".../model-store-python")
   val metaPath = Paths.get(".../model-store-python/my-model.meta")
   val metaGraphDef = MetaGraphDef.parseFrom(new BufferedInputStream(new FileInputStream(metaPath.toFile)))
   val saver = tf.Saver.fromMetaGraphDef(metaGraphDef)
   saver.restore(sess, tf.Saver.latestCheckpoint(savePath, "my-model.meta").get)
Codie (蒂诺)
@lasso-codie
hey, I've got two questions. Before diving deeper into the code, is there a limit to the memory of a tensor (e.g. java arrays can have only 2GB) and is there also a possibility to do some memory hacking by using strides?
Could not see anything for strides in the API docs
Lakshya KD
@LakshKD
Hey, I was trying running some examples using tensorflow for scala and it is working fine with Scala SBT project but I want to do it in Scala Gradle project and the issue which I was facing is as follows: I was able to add the dependency in gradle file but when importing tensorflow for scala in scala code it is showing error. I was trying importing like this: import org.platanios.tensorflow.api.ops.io.data.Dataset
import org.platanios.tensorflow.api.types.DataType
import org.platanios.tensorflow.api.{tf, _} Can someone please help showing some small demo that how it can be done in Gradle. Thanks
@eaplatanios
Mandar Chandorkar
@mandar2812
Hi @/all Just wanted to ask all Tensorflow scala users if they have used the tf.data.datasetFromOutputSlices() method for creating a TF Dataset. I just reported issue #165 which details problems with using this method in a practical example. Has any one faced something similar?
Haiping
@Oceania2018
Interesting demo. run tensorflow in Jupyter notebook and based on .NET.
jxtps
@jxtps
Would it make sense for TF4S to use JavaCPP Presets for TensorFlow? See eaplatanios/tensorflow_scala#162
Ashwin Ravishakar
@ashwinravishankar
Hello @/all. I used <TensorObject>.toProto to convert to TensorProto object. Am unable to currently find an equivalent way to convert back a TensorProto object to a Tensor. Any help please?
Aggelos Biboudis
@biboudis
Why is ControlFlow package private for api? What is the proposed way to use a whileLoop directly?
damien
@damienstanton
is this repo on the sonatype snapshot resolver? I cannot seem to get sbt to locate either the prebuilt binaries nor the other one
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[warn]     ::          UNRESOLVED DEPENDENCIES         ::
[warn]     ::::::::::::::::::::::::::::::::::::::::::::::
[warn]     :: org.platanios#tensorflow;0.4.0: not found
disregard - I fixed it. There are a couple of typos in the documentation, so I'll open a PR to get them fixed :thumbsup:
damien
@damienstanton
essentially the sbt coordinates are wrong (no implicit scalaVersion underscore), and they point to an old version (0.4.0 instead of 0.4.1).
who is able to give a talk, at one of these conferences?