Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Apr 15 18:19

    eaplatanios on master

    [DOC] Updated the release notes. (compare)

  • Apr 15 02:27

    eaplatanios on master

    Setting version to 0.6.4 Setting version to 0.6.5-SNAPSH… (compare)

  • Apr 15 02:01

    eaplatanios on v0.6.4

    (compare)

  • Apr 15 02:01

    eaplatanios on v0.6.3

    (compare)

  • Apr 15 02:01

    eaplatanios on master

    Setting version to 0.6.3 Setting version to 0.6.4-SNAPSH… Setting version to 0.6.3 and 3 more (compare)

  • Apr 15 00:19

    eaplatanios on master

    [OPS] Saver fixes. (compare)

  • Apr 14 20:25

    eaplatanios on master

    Setting version to 0.6.2 Setting version to 0.6.3-SNAPSH… (compare)

  • Apr 14 19:58

    eaplatanios on v0.6.2

    (compare)

  • Apr 14 19:58

    eaplatanios on master

    Setting version to 0.6.2 Setting version to 0.6.3-SNAPSH… (compare)

  • Apr 14 19:17

    eaplatanios on master

    [OPS] Improvements to 'Tensor.f… (compare)

  • Apr 14 15:29

    eaplatanios on master

    [OPS] Added a check for 'stack'… [DOC] Updated the release notes. (compare)

  • Apr 07 02:44

    eaplatanios on master

    Setting version to 0.6.1 Setting version to 0.6.2-SNAPSH… (compare)

  • Apr 07 02:18

    eaplatanios on v0.6.1

    (compare)

  • Apr 07 02:18

    eaplatanios on master

    Setting version to 0.6.1 Setting version to 0.6.2-SNAPSH… (compare)

  • Apr 07 01:30

    eaplatanios on master

    [CORE] Fix to the Session memor… (compare)

  • Apr 07 01:11

    eaplatanios on master

    [DOC] Updated the release notes. (compare)

  • Apr 07 01:10

    eaplatanios on master

    [CORE] No more session locking. (compare)

  • Apr 05 18:59

    eaplatanios on master

    Setting version to 0.6.0 Setting version to 0.6.1-SNAPSH… (compare)

  • Apr 05 15:52

    eaplatanios on v0.6.0

    (compare)

  • Apr 05 15:52

    eaplatanios on master

    Setting version to 0.6.0 Setting version to 0.6.1-SNAPSH… (compare)

Anthony Platanios
@eaplatanios
@ai_cluster_twitter that's correct. Sorry for not responding earlier! Keep in mind that I haven't updated the library in a while and haven't verified all works fine with TF 2.x. I hope to get a chance to look into this soon.
Izmar Verhage
@izzyreal
Does anyone know how to call Image.decodePng against an Array[Byte]? I'd like to load an image from memory rather than from file.
Izmar Verhage
@izzyreal
With TF Python it's a breeze, just file.read() and pass the result to convert_to_tensorand that to decode_png.
But any combination I tried in Scala doesn't give expected results. The best I got was
Expected image (JPEG, PNG, or GIF), got unknown format starting with '\302\211PNG\r\n\032\n\000\000\000\rIHD'
     [[{{node DecodePng}}]]
and
got unknown format starting with '?PNG\r\n\032\n\000\000\000\rIHDR'
Izmar Verhage
@izzyreal
This is the code that gives the last result
    ImageIO.write(bufferedImage, "png", outputStream)
    val data = Output.constant(new String(outputStream.toByteArray))
    val image = Image.decodePng(data, 3)
James McCabe
@oranda
Wonderful project @eaplatanios. One question. Running MNIST example ends in accuracy of only 0.942/0.931. Is that expected? v0.4.2
Corey O'Connor
@coreyoconnor
Having a great time using this library for numerical processing: https://www.youtube.com/watch?v=lDPT55pmrzc
One question tho: How do I pronounce the author's name Platanios? I trip over it ever time :\
James McCabe
@oranda
@xtordoir pronounces it, though I don't know if he is correct: https://www.youtube.com/watch?v=83iprWIhjMM
Corey O'Connor
@coreyoconnor
@oranda thanks!
James McCabe
@oranda
May I ask the community a general question? How much are the giants (Google, Facebook, etc) using Scala for ML at the current time? Maybe behind the scenes? It's hard to believe that with such large codebases they are still 100% on Python. If they are, do you think the next couple of years could see a breakthrough? Any anecdotal reports of Scala with ML?
Anthony Platanios
@eaplatanios
@coreyoconnor glad you've been enjoying the library! Regarding my name haha many people have trouble pronouncing it, but @xtordoir does a good job. :)
@oranda it depends a lot on the company, but most of them do not use Scala much for ML. Oftentimes models are developed using python and then deployed using TF, Torch, etc, in C++, Java, ... There are also many companies that use Python which is also surprising to me. However, there are a few places that also use Scala for ML, including Microsoft (where I work). I'm also aware of Netflix doing some ML stuff using Scala (@jeremyrsmith may be able to say more) and Twitter. AFAIK it's not used much in Google.
Mandar Chandorkar
@mandar2812

@oranda I implemented my entire PhD research in Scala, loved using Scala for ML! I work in industry now and one thing I learned is that Scala has a very bad reputation. My CTO told me that he feels that one needs a PhD to understand Scala code. The Scala community and its emphasis on a lot of the advanced syntactic features of the language have not helped one bit.

Sometimes its better to hide those capabilities and emphasize how easy it is to get started with a language if we want to attract new talent. But the community has a bias for Monads/DSLs/Type Theory etc.

All of those advanced capabilities and extended freedoms mean that I can probably write maintainable yet simple (whatever that means) Scala code, but if I take a peek into someone else's library, I might be out of my depth very very quickly and even after years of experience I might still have to invest hours/days to understand those advanced design patterns.
So I don't see many companies adopting Scala for ML, and I am not surprised.
Sometimes excess freedoms are not a good thing.
James McCabe
@oranda
@eaplatanios and @mandar2812 thanks for your replies. I have some code that uses tensorflow_scala at https://github.com/oranda/ScalaTensorflowPOC/blob/master/src/main/scala/com/oranda/tensorflow/MNISTSimple.scala. I have tried to keep the code as simple as possible, avoiding things like implicits and advanced Scala features, and a Python dev should be able to understand it. I would love to get your opinions on this style, and how this code can be improved.
Corey O'Connor
@coreyoconnor

@coreyoconnor glad you've been enjoying the library! Regarding my name haha many people have trouble pronouncing it, but @xtordoir does a good job. :)

thanks!

I've put out the second video in a series on tensorflow for scala: https://www.youtube.com/watch?v=NwZ5WviQJVQ
Yiğit Ünlü
@argestes
Hey ppl!
I'm trying to use the library with Scala 2.13
So far I've failed
Corey O'Connor
@coreyoconnor
What version are you using?
IIRC you will need to use a SNAPSHOT version
Anthony Platanios
@eaplatanios
Yes I would suggest 0.5.2-SNAPSHOT. I haven’t published a release yet because I added support for Windows and I’m currently testing that.
danoliv
@danoliv
Hello everybody, I work as an engineer in a data science team, we use 100% scala in production at the moment. We use mostly spark ml, xgboost. We plan to deploy some deeplearning models in production, is tensorflow_scala production ready ? I've used it successfully with tensorflow 1.0 for some personal project but we are planing to use tensorflow 2.2. Thanks
Corey O'Connor
@coreyoconnor

I've put out the second video in a series on tensorflow for scala: https://www.youtube.com/watch?v=NwZ5WviQJVQ

Third video in the series: https://www.youtube.com/watch?v=WpqH9aqR3Mo

Tho I don't actually talk about tensorflow but ZIO library design shrug

Anthony Platanios
@eaplatanios
@danoliv 0.5.2-SNAPSHOT is built for TF 2.2 but still relies on building graphs and does not yet support model training in eager mode. Regarding stability, we’re in the process of evaluating whether we can use it in production. No issues so far, but I’ll let you know if something pops up. In either case, I’m glad to hear you’re considering it :)
@coreyoconnor thanks for the nice videos you’re making! I’ll take a look at this one as I’m not very familiar with ZIO.
danoliv
@danoliv
@eaplatanios thanks for your answer, I am looking forward to test it. Is it available on some public repo ?
Raghav Bharadwaj
@raghavbharadwaj
Hello!! Are 0.5.x-SNAPSHOT releases available with prebuilt binaries? I can't seem to locate them. Thanks for the help!
Anthony Platanios
@eaplatanios
Hi! There is now a full release (0.5.4) that is built for TF 2.3 and also has support for Windows (along with pre-compiled TF binaries).
jeremyrsmith
@jeremyrsmith
@eaplatanios that is worthy of a @/all :grinning:
Anthony Platanios
@eaplatanios
heh fair point. I haven't really been active here to know all the tricks :) but now that I do @/all it turns out there is an issue with the 0.5.4 release (only darwin binaries are included), as pointed out by @mandar2812 and I just pushed a 0.5.5 release that hopefully resolves this. Sorry about that but including support for windows has been a bit of a painful process because I don't have a good way to cross-compile.
Corey O'Connor
@coreyoconnor
cool! thanks!
Third part of my series on tensorflow in scala: https://www.youtube.com/watch?v=Wza6wy_hdSg
Which finally contains some tensorflow... at the end anyways
Corey O'Connor
@coreyoconnor
One aspect I'm still not sure on: Resource management. As I understand: Initialize a session with a graph and that graph will be released on session close. Use tf.createWith(graph) for constructed nodes to be owned by the given graph. In which case only a session.close() is required to release all resources. Sound about right?
Sören Brunk
@sbrunk
@eaplatanios it seems like 0.5.5 is only published for Scala 2.12 right now. Would it be possible to publish 2.13 artefacts as well?
Anthony Platanios
@eaplatanios
@sbrunk sorry my bad I did this in a rush. I will try to publish them either tomorrow or Monday.
@coreyoconnor in general you don’t ever have to explicitly call .close() on anything as TF Scala will try to free native objects when references to them are garbage collected. Of course this is too ideal and in practice you might run into memory issues if your sessions hold large objects and you set the JVM memory limit high. However, I’d say you can ignore that until you notice memory becoming an issue for your use case.
Also, session.close() will not release tensors that are returned from calls to session.run.
Corey O'Connor
@coreyoconnor
@eaplatanios thanks! good to know :)
Sören Brunk
@sbrunk
@eaplatanios thanks :)
Sami Badawi
@sami-badawi
I am new to tensorflow_scala and not sure where to start. Is there some example project?
Alexey Novakov
@novakov-alexey
Hi there. How can I calculate binary accuracy like here? https://www.tensorflow.org/api_docs/python/tf/keras/metrics/BinaryAccuracy
Alexey Novakov
@novakov-alexey
I have come up with such implementation, not sure if it is really correct:
  val batchSize = 100
  val accMetric = tf.metrics.MapMetric(
    (v: (Output[Float], (Output[Float], Output[Float]))) => {
      val positives = v._1 > 0.5f
      val binary = tf
        .select(
          positives,
          tf.fill(Shape(batchSize, positives.shape(1)))(1f),
          tf.fill(Shape(batchSize, positives.shape(1)))(0f)
        )
        .toFloat
      (binary, v._2._2.toFloat)
    },
    tf.metrics.Accuracy("Accuracy")
  )
Alexey Novakov
@novakov-alexey
https://github.com/novakov-alexey/tensorflow-scala-example I have created one example here, would love someone's feedback on that
Valeriy Avanesov
@akopich
Hi all. As far as I understand, tf.globalVariablesInitializer() initializes all the global variables. But is there a way to initialize a single one?