Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 00:49
    SethTisue commented #755
  • 00:45
    SethTisue edited #754
  • 00:44
    SethTisue edited #754
  • Oct 26 20:17
    krisweiss opened #2211
  • Oct 26 15:33

    julienrf on main

    Fix links to IntelliJ and SBT … Remove the IntelliJ and SBT ent… Merge pull request #1286 from s… (compare)

  • Oct 26 15:33
    julienrf closed #1286
  • Oct 26 15:33
    julienrf closed #1285
  • Oct 26 15:33
    julienrf commented #1286
  • Oct 26 15:32
    s5bug synchronize #1286
  • Oct 26 13:16
    SethTisue labeled #2210
  • Oct 26 13:16
    SethTisue commented #2210
  • Oct 26 11:49
    krisweiss edited #2210
  • Oct 26 11:48
    krisweiss opened #2210
  • Oct 26 10:59
    julienrf commented #1286
  • Oct 26 08:57
    s5bug commented #1286
  • Oct 26 08:10
    julienrf commented #1286
  • Oct 26 07:49
    s5bug commented #1286
  • Oct 26 07:48
    s5bug opened #1286
  • Oct 26 07:45
    julienrf commented #1285
  • Oct 26 07:40
    julienrf commented #2205
Rohan Sircar
@rohan-sircar
so for java libs it should be single %
(as that is a java lib)
Rob Norris
@tpolecat
@jimka2001 there is no way to iterate loaded classes as far as I know. The reflections library above will scan your classpath if it can, but this isn't possible in general.
"classpath" is only a meaningful idea for certain kinds of classloaders.
Jim Newton
@jimka2001
@tpolecat yes it seems reflections doesn't really work. The following prints [] without realizing that Class1 is a subclass of Abstract1 :-(
object sanityTest {
  def main(argv: Array[String]):Unit = {
    import genus._

    import org.reflections.Reflections
    val reflections = new Reflections("cl-robdd-scala")
    abstract class Abstract1
    trait Trait1
    class Class1 extends Abstract1 with Trait1
    println(reflections.getSubTypesOf(classOf[Abstract1]))
  }
}
Jim Newton
@jimka2001
although I wonder whether I'm doing something wrong, as at least getSubTypesOf should know that Abstract1 is a subtype of itself.
I wonder that the argument to the constructor new Reflections(...) is supposed to be. It seems to work (create an object) even if I give it the name of a non-existent project.

Interesting, when I use "" as the argument of the constructor, I get something more promising.

object sanityTest {
  def main(argv: Array[String]):Unit = {
    import genus._

    import org.reflections.Reflections
    val reflections = new Reflections("")
    abstract class Abstract1
    trait Trait1
    class Class1 extends Abstract1 with Trait1
    abstract class Class2 extends java.lang.Number

    println(reflections)
    println(reflections.getSubTypesOf(classOf[Abstract1]))
    println(reflections.getSubTypesOf(classOf[java.lang.Number]))
  }
}

This prints:

org.reflections.Reflections@229d10bd
[class rte.sanityTest$Class1$1]
[class rte.sanityTest$Class2$1]
kiti_Nomad
@Kiti-Nomad
scala3 is released, new features include ceylon's set theory type.
Dale Wijnand
@dwijnand
What's that? Can't find that working on Google.
Luis Miguel Mejía Suárez
@BalmungSan
@ImGrayMouser_twitter because Set is defined as generic for all types, and they exploit the fact that at runtime any value would have a equals and hashcode methods to implement it. Now, that implies that the Set will behave according to the underlying elements.
And in the case of Arrays that means that equality is based in having the same object (same memory address) rather than the same values.
Which makes a lot of sense, because that is the only sensible equals implementation for something that is mutable.
Now, the stdlib could have restricted Set to only work for types that have a sensible equals using a typeclass. But, that would have just made it weird to use and more limited, at the end the current equals of Array is, in fact, sensible and useful; just not in your case.
Rob Norris
@tpolecat
If you tried to use a TreeSet then it wouldn't typecheck (as you wish).
Geoffrey Knauth
@gknauth
I mentioned the recent release of Scala 3, and a coworker said Databricks has only recently started supporting Scala 2.12. Is that correct? I’m trying to figure out the timeline to being able to use Scala 3 with Databricks.
Luis Miguel Mejía Suárez
@BalmungSan
Disclaimer I have no relationship with them, nor with the compiler team. And I am just a random person in the Internet with some resentments about Spark. So do not take me too serious.
Seth Tisue
@SethTisue
@gknauth Spark has been on 2.12 for a long time. they're close to shipping 2.13 support: https://issues.apache.org/jira/browse/SPARK-25075 . and there are comments on the same ticket talking about the 3 situation
Luis Miguel Mejía Suárez
@BalmungSan
But I guess Spark won't support Scala 3 until 2 years. I hope I am wrong.
Geoffrey Knauth
@gknauth
Thanks @SethTisue
Seth Tisue
@SethTisue
2 years sounds like a reasonable guess
Geoffrey Knauth
@gknauth
Thanks also @BalmungSan , good to clear the pond of rocks before jumping in.
Khagay Nagdimov
@khagaydv01
Hello. I'm encountering an interesting situation. I have a method that is wrapped in a Try. That method has a sub-method that throws an exception but, the return value of that sub-method is a String. It seems that the exception gets thrown and the thread crashes even though the parent method is wrapped in a Try. Is that expected behavior? I would have thought the exception gets wrapped in a Failure.
Jim Newton
@jimka2001
Is there a better idiom for following? I.e., without interrupting the exception being thrown, take a different finally action based on whether the expression finished or aborted? I'm using a mutable variable.
      println("[ starting " + testName)
      var finished = false
      try{
        testFun
        finished = true
      }
      finally{
        if (finished)
          println("] finished " + testName)
        else
          println("] aborted" + testName)
      }
Seth Tisue
@SethTisue
@khagaydv01 what kind of exception is it? Try distinguishes fatal from nonfatal exceptions and only catches the nonfatal ones
Khagay Nagdimov
@khagaydv01
@SethTisue a custom exception that derived from RuntimeException
Jim Newton
@jimka2001
in this particular case it is a java.lang.OutOfMemoryError: Java heap space
keep in mind I don't want the exception tampered with. I still want it to fail, I just want to indicate that the test case finished successfully or that it aborted
Luis Miguel Mejía Suárez
@BalmungSan
@khagaydv01 can you create a MCVE in Scastie?
That should work.
@jimka2001 AFAIK the moment you have a fatal exception like a memory error you can't do anything at all.
But I may be wrong.
Khagay Nagdimov
@khagaydv01
weird thing is when i throw that same exception in the method that is wrapped a Try, it gets wrapped in a Failure
seems like sub-methods that throw exception that aren't wrapped in a Try crash the thread
even if the parent method is wrapped in a Try
Walter Chang
@weihsiu

is it possible to make an extension of a f-bounded type?

object FBounded:
  trait Self[A <: Self[A]]:
    def bar(): Unit = ()

  extension [A <: Self[A]](self: A)
    def foo(): Unit = ()

  trait Me extends Self[Me]

  val self: Self[Me] = new Me {}
  self.bar()
  self.foo() // fails to compile

this last line failed to compile:

[error] 12 |  self.foo()
[error]    |  ^^^^^^^^
[error]    |  value foo is not a member of FBounded.Self[FBounded.Me].
[error]    |  An extension method was tried, but could not be fully constructed:
[error]    |
[error]    |      FBounded.foo[A](self)
[error] one error found
Luis Miguel Mejía Suárez
@BalmungSan
@khagaydv01 is that sub method run in another thread or is Spark related?
@weihsiu it works if you leave its type as Me: https://scastie.scala-lang.org/BalmungSan/IGf7rvujQZmupUSma2i3EA/1
Which makes sense Self[Me] is not a A <: Self[A]
Walter Chang
@weihsiu
@BalmungSan ah, thanks!!
Khagay Nagdimov
@khagaydv01
@BalmungSan yes, it's running in a spark application but, at that moment, there is no spark method called
Luis Miguel Mejía Suárez
@BalmungSan
@khagaydv01 I mean, maybe your code looks like this?
def foo(i: Int): String = {
  if (i == 0) throw new CustomException()
  else i.toString
}

def bar(data: Dataset[Int]): Try[Dataset[String]] =
  Try(data.map(foo))
Seth Tisue
@SethTisue
the lampepfl/dotty room has been shut down, so this is now a Scala 2 and 3 room
Luis Miguel Mejía Suárez
@BalmungSan
:open_mouth:
Bjorn Regnell
@bjornregnell
:thumbsup:
Khagay Nagdimov
@khagaydv01
@BalmungSan yes! just like that
Luis Miguel Mejía Suárez
@BalmungSan
@khagaydv01 you need to understand two things:
  1. Try(data.map(foo)) is running in the master node, whereas the execution of foo is running in multiple executor nodes; as such Try won't be able to catch those exceptions.
  2. data.map(foo) didn't do anything at all, it just added a node to the DAG telling the runtime that when running the pipeline to reach data it then needs to run a map(foo) - So, that call didn't throw any exception, the moment the exception is thrown is when you run an action on the resulting Dataset.
You may do data.map(x =>Try(foo(x))) which would catch each individual exception and you would have a Dataset of Try of String.
But I actually do not know if Spark would support that and if it would make it easier to deal with failures.
My gut feeling is that there won't be an Encoder for Try. But, hopefully, I would be wrong.
DavidGoodenough
@DavidGoodenough
Would it be sensible to prefix all remarks with either "Scala-2" or "Scala-3" so that the context is obvious now that the dotty room has merged in here?
Luis Miguel Mejía Suárez
@BalmungSan
Not really a prefix but yeah, it would help if people start to make it clear if their question is Scala 2 specific, Scala 3 specific (as well as with many other categories like sbt, Spark, FP, design, etc)
In any case, I guess most of the time the context would be clear.
Seth Tisue
@SethTisue
Martin's ScalaCon keynote starts in 5 minutes 🍿