Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Feb 27 02:01
    scala-steward opened #256
  • Feb 27 02:01
    scala-steward opened #255
  • Feb 20 15:55
    codecov[bot] commented #249
  • Feb 20 15:54
    codecov[bot] commented #251
  • Feb 20 15:54
    codecov[bot] commented #251
  • Feb 20 15:50
    codecov[bot] commented #251
  • Feb 20 15:50
    scala-steward synchronize #251
  • Feb 20 15:49
    scala-steward synchronize #249
  • Feb 20 14:45

    sergeykolbasov on master

    Update cats-effect to 2.3.3 (#2… (compare)

  • Feb 20 14:45
    sergeykolbasov closed #252
  • Feb 20 14:45

    sergeykolbasov on master

    Update sbt-mdoc to 2.2.18 (#253) (compare)

  • Feb 20 14:45
    sergeykolbasov closed #253
  • Feb 19 03:56
    scala-steward closed #236
  • Feb 19 03:56
    scala-steward commented #236
  • Feb 19 03:56
    scala-steward opened #254
  • Feb 17 21:04
    codecov[bot] commented #253
  • Feb 17 21:04
    codecov[bot] commented #253
  • Feb 17 20:59
    scala-steward closed #243
  • Feb 17 20:59
    scala-steward commented #243
  • Feb 17 20:59
    scala-steward opened #253
Sergey Kolbasov
@sergeykolbasov

@kubukoz

Nope, there is no logstash integration. At Zalando we ship logs with a separate agent as a part of Kubernetes deployment, so it never was a priority for us to have such functionality in this lib.

However, it's not the first time I hear that question, so I'm open for contributions :)

Sergey Kolbasov
@sergeykolbasov
Hm, I don't think it's possible to define java-compatible static field
Jakub Kozłowski
@kubukoz
Yeah, I tried to do it in Java and refer to the Scala sources but hit a bug in bloop
Can't remember what sbt did...
Well, it compiled but I'm not sure it worked.
Sergey Kolbasov
@sergeykolbasov

@kubukoz

I think I solved that issue with StaticLoggerBinder.

You need a plain Java class:

package org.slf4j.impl;

import oh.boi.ExternalLogger;

public class StaticLoggerBinder extends ExternalLogger {
    public static String REQUESTED_API_VERSION = "1.7";
    private static StaticLoggerBinder instance = new StaticLoggerBinder();

    public static StaticLoggerBinder getSingleton() {
        return instance;
    }
}

together with a Scala one:

package oh.boi

import cats.effect.{ContextShift, Clock, Effect, IO, Timer}
import io.odin._
import io.odin.slf4j.OdinLoggerBinder

import scala.concurrent.ExecutionContext

class ExternalLogger extends OdinLoggerBinder[IO] {

  val ec: ExecutionContext = scala.concurrent.ExecutionContext.global
  implicit val timer: Timer[IO] = IO.timer(ec)
  implicit val clock: Clock[IO] = timer.clock
  implicit val cs: ContextShift[IO] = IO.contextShift(ec)
  implicit val F: Effect[IO] = IO.ioEffect

  val loggers: PartialFunction[String, Logger[IO]] = {
    case "some.external.package.SpecificClass" =>
      consoleLogger[IO](minLevel = Level.Warn) //disable noisy external logs
    case _ => //if wildcard case isn't provided, default logger is no-op
      consoleLogger[IO]()
  }
}
I'll update documentation later
Jakub Kozłowski
@kubukoz
that's what I tried to do ;)
Sergey Kolbasov
@sergeykolbasov
well, works for me
Rohan Sircar
@rohan-sircar
Hi, if you don't mind me asking, how is scala.js support coming along?
Sergey Kolbasov
@sergeykolbasov

Hello

Sorry, didn't test it yet. In theory there is a PR opened, but have to check it out

Rohan Sircar
@rohan-sircar
Will be awesome when it comes out. But take your time.
Nice package name btw - package oh.boi . I put wow.doge in one of my projects.
Sergey Kolbasov
@sergeykolbasov
such wov, very amaze :)
Rohan Sircar
@rohan-sircar
very logging, much functional, many thanks
I had a few questions, I went through the documentation and couldn't find the answers there but maybe I missed it.
  1. In the Slf4j bridge, is there a built in way to do a catch all of all the classes under a package, or would I need to use something like a regex?
    val loggers: PartialFunction[String, Logger[IO]] = {
     case "org.asynchttpclient.netty.channel.DefaultChannelPool" =>
       consoleLogger[IO](minLevel = Level.Warn)
     case _ => //if wildcard case isn't provided, default logger is no-op
       consoleLogger[IO]()
    }
    Suppose I wanted to catch all classes under org.asynchttpclient under one case clause.
Rohan Sircar
@rohan-sircar

2 - Is the logger instance supposed to be passed around, or should each class create it's own logger instance?
3 - I've been unable to integrate the Resource monad in my code that uses Monix Task, so I'm using the unsafe method for getting the logger instance -

val logger = consoleLogger().withAsyncUnsafe()

But I'm not sure what unsafe means here, does it refer to flushing the loggers buffer? Is it possible to flush the buffer in a shutdown hook thus making it safe?

Sergey Kolbasov
@sergeykolbasov

Hey @rohan-sircar, thanks for the questions

  1. It is a partial function, therefore you could use all the nice things that Scala gives you, including if guards, that are quite handy and much more powerful then just a wildcards:
val loggers: PartialFunction[String, Logger[IO]] = {
 case asyncHttpClient if asyncHttpClient.startsWith("org.asynchttpclient.netty") =>
   consoleLogger[IO](minLevel = Level.Warn)
 case _ => //if wildcard case isn't provided, default logger is no-op
   consoleLogger[IO]()
}
  1. I'd suggest to have a single instance for everything, as it's just easier to manage. But you might have different needs for different classes, and in case you can't just route the message using stuff from io.odin.config._ to different loggers, just provide another instance

  2. Resource is hard to integrate for sure. What I could suggest is to have on the very top of you application (i.e. TaskApp.main) a Resource monad that you .use(_ => Task.never). Often you start multiple resources there, such as HTTP clients, some I/O channels like database connection etc etc. So instead of encapsulating all the I/O initialization inside of implementations, making them side-effectful by that, you do that in the main. In Odin .unsafe stuff doesn't use the shutdown hook sadly, but you could make it on your own though

Rohan Sircar
@rohan-sircar
Thanks for the quick response actually :)
  1. Oh, right, I should have thought of that
  2. I had a feeling that the instance is supposed to be passed around. I'm coming from OO-land and there loggers are instantiated per class so I asked. Wouldn't this make it a little boilerplate-y, since almost every class would need to have the logger in it's constructor?
  3. I did try to use Resource that way, but without the .use(_ => Task.never) part, it just free'd all the resources once the control reached the end of the Task chain, and since I'm new to effect programming I was wondering what's up :)
    I do have an application shutdown method like -
    def makePrimaryStage(
      backend: AppTypes.HttpBackend,
      actorSystem: classic.ActorSystem
    ) = {
    new PrimaryStage {
      scene = new DefaultUI().scene
      onCloseRequest = () => {
        val f2 = actorSystem.terminate()
        val f1 = backend.close().runToFuture
        ......
     }
    }
    }
    It's a JavaFX application. I was thinking I could close the logger here, but what method am I supposed to use? I couldn't find anything named stop, terminate or flush. Maybe I should go through the API docs.
    A little off the note - I was thinking of using a thin cake module to provide the http backend, actor system and logger to reduce the boilerplate, but that's discouraged in fp I suppose?
Sergey Kolbasov
@sergeykolbasov

Can't you just extend i.e. TaskApp or IOApp in the main object of you application and run the resource allocation + .use in the main method?

it'll take care of the shutdown hook on its own for all the running resources.

Ie, actor system could be allocated as a resource as well, something like:

val acquire: Task[ActorSystem] = Task(createActorSystem)
def release(actorSystem: ActorSystem): Task[Unit] = Task(actorSystem.terminate())

Resource.make(acquire)(release) // Resource[Task, ActorSystem]
Essentially, your whole application initialization lives inside of a single for-comprehension, and resources take care of a safe release of anything you have running, would it be logger, open I/O, or actor system
dumb example from the app I have open atm:
def init[F[_]: ConcurrentEffect: Timer: ContextShift]: Resource[F, Unit] =
    for {
      config <- Config.load[F]
      logger <- InternalLogger.init[F](config.app)
      oauthService <- OAuthService.init(config.oauth)
      storageService = StorageService.noop[F]
      contentService = ContentService.noop[F]
      routes = Routes.all(storageService, contentService)
      httpServer <- HttpServer.init(config.http, routes, logger)
      _ <- httpServer.run
    } yield {
      ()
    }
Sergey Kolbasov
@sergeykolbasov
but just in case Resource has an allocated method that would return you Task[(B, Task[Unit])], where the first element of a tuple is an allocated resource and second is a release token

Wouldn't this make it a little boilerplate-y, since almost every class would need to have the logger in it's constructor?

Yeah, it does. You might use implicit argument in constructor for it if you wish. On the positive side, you have a full control over the logger from outside of the class, would it be test, benchmarks or whatever. It's called composition, and composition is nice :)

Rohan Sircar
@rohan-sircar

Composition and greater control are what made me want to switch to using IO monads and ditch OO based DI and FXML injection, but the problem in this case is that I am trying to interface with a framework that doesn't support effect monads(JavaFX), and since I'm new to IO monads it's going to take me a while to figure things out. My goal was to try and run green threads on the JavaFX application thread to avoid nested Platform.runLater() calls.

I am unable to use IOApp/TaskApp because I need to use JFXApp, adding both causes a conflict in the main methods. This is what my main class looks like right now -

object Main extends JFXApp {

  val logger = consoleLogger().withAsyncUnsafe()

  lazy val schedulers: Schedulers = new Schedulers()

  implicit lazy val defaultScheduler: Scheduler = schedulers.fx

  lazy val backendTask = AsyncHttpClientMonixBackend()
  lazy val actorSystemTask = Task {
    classic.ActorSystem(
      name = "FXActorSystem"
    )
  }

  lazy val application = for {
    _ <- logger.info("Starting application..")
    backend <- backendTask
    actorSystem <- actorSystemTask
    appStage <- Task { makePrimaryStage(backend, actorSystem) } 
    _ <- Task {
      // this stage refers to implicit jfx stage
      // probably makes this impure, but I can't think of a better way right now
      stage = appStage
    }
  ... other stuff
  } yield ()
  application.timed.runToFuture
    .onComplete(res =>
      res match {
        case Failure(exception) => {
          println("Application start failed. Reason -")
          exception.printStackTrace()
        }
        case Success((duration, _)) =>
          println(
            s"Application started successfully in ${duration.toSeconds} seconds"
          )
      }
    )

If I wanted to switch to resource, I would have to see if .use(_ => Task.never) works with the FX application thread or not.

Sergey Kolbasov
@sergeykolbasov

Random educated guess, since I've never worked with ScalaFX

Try to pack the instance of JFXApp into resource itself instead of extending it in your main. It seems that you need only stage variable in the end, and run your application as a TaskApp instead.

Essentially, my question is if you really must extend JFXApp in your main object, and if you don't then you have all the power to do whatever you want

Rohan Sircar
@rohan-sircar

Try to pack the instance of JFXApp into resource itself instead of extending it in your main.

Thanks, this was the key. I had thought about getting rid of JFXApp before, but I couldn't figure out how. Apparently, since it's a trait it can be instantiated like a regular class.

I experimented a lot with it yesterday and I finally managed to account for everything.

object Main extends TaskApp {

  override def run(args: List[String]): Task[ExitCode] = {
    lazy val appResource = for {
      logger <- consoleLogger().withAsync()
      backend <- AsyncHttpClientMonixBackend.resource()
      actorSystem <-
        Resource.make(logger.info("Creating Actor System") >> Task {
          classic.ActorSystem(
            name = "FXActorSystem"
          )
        })(sys =>
          logger.info("Shutting down actor system") >> Task.fromFuture(
            sys.terminate()
          ) >> logger.info("Actor System terminated")
        )
      fxApp <- Resource.make(logger.info("Creating FX Application") >> Task {
        val app: JFXApp = new JFXApp {
          //fx scheduler

          val application =
            for {
              appStage <- logger.info("Inside JFX application stage") >> Task(
                makePrimaryStage(backend, actorSystem)
              )
              _ <- Task { stage = appStage }
              // other stuff
              }
            } yield ()
          application.timed.runAsync( ... )

          override def stopApp() = {
            Platform.exit()
            // System.exit(0) not required
          }
        }
        app
      })(app => logger.info("Stopping FX Application") >> Task(app.stopApp()))
    } yield (fxApp)
    appResource
      .use(fxApp => Task(fxApp.main(args.toArray)))
      .as(ExitCode.Success)
  }

Just as you said, it's a TaskApp main class, with a Resource chain instantiating all the resources and passing them to my fx app, which has a task chain with all my previous app code. Everything else works the same as before.
An interesting thing to note is how the resource chain must be blocked with FXApp#main. This blocks it until the close button of the GUI is pressed. Everything is shut down properly after that. Using Task.never breaks the close button.

Sergey Kolbasov
@sergeykolbasov
glad that it worked for you :)
Rohan Sircar
@rohan-sircar
Hi Sergey, hope you're doing well. I had a few more questions -
1) While using VisualVM I noticed that low values of timeWindow for async loggers seem to create a large amount of garbage. Even in an idle application, a 1GB heap fills up quite fast before being GC'ed. Higher values naturally lead to delayed logs. Is this expected behavior?
2) Can I have a filelogger in the SLF4J bridge? The filelogger function returns a Resource whereas the loggers field of the bridge expects a Logger[IO]
Sergey Kolbasov
@sergeykolbasov

Hi @rohan-sircar

Regarding the first one. How many logs do you generate? It might be possible that queue grows faster than it's cleared, so the heap would predictably grow.

You can have a file logger in external bridge I think, although it's tricky.
Run fileLogger.allocated to get a pair of acquire-release tokens F[(Logger[F], F[Unit])], then just .unsafeRunSync i.e. (in case you're using IO) to get a pair of (Logger[F], IO[Unit]).
Voilà, here is your logger. I'd also add a shutdown hook to the JVM to call the second part: IO[Unit] on system exit, which is a cancellation token that safely releases resource.

Thanks for asking, I think it's a good topic to explore easier allocation of Resource-based loggers for SLF4J
Rohan Sircar
@rohan-sircar
Thanks for the detailed explanation, it works well now. Although if I set the logger to the same file as the main application logger they start overwriting each other, but I suppose that is to be expected.
Regarding the former, looks like it's a bad interaction with the profiler. As soon as I start profiling, memory allocation shoots up.
Rohan Sircar
@rohan-sircar
image.png
The jagged lines are when I started profiling. If I set the timeWindow to a high value, this does not happen. That said, I don't think the problem is with Odin itself.
Jakub Kozłowski
@kubukoz
Did Logger.apply[F] disappear in 0.10.0?
ah, I accidentally got 0.1.0 and 0.10.0 in the classpath :joy: sorry. Metals was directing me to the wrong sources too
Jakub Kozłowski
@kubukoz
This message was deleted