Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jul 22 13:36

    mergify[bot] on master

    Update mdoc, sbt-mdoc to 2.2.22 Merge pull request #1047 from s… (compare)

  • Jul 22 13:36
    mergify[bot] closed #1047
  • Jul 22 13:07
    mergify[bot] labeled #1047
  • Jul 22 13:06
    scala-steward opened #1047
  • Jul 22 05:13
    adamw commented #675
  • Jul 21 20:28
    vakhtang commented #675
  • Jul 21 15:36
    mergify[bot] labeled #1046
  • Jul 21 15:35
    scala-steward opened #1046
  • Jul 21 02:04

    mergify[bot] on master

    Update http4s-blaze-client, htt… Merge pull request #1045 from s… (compare)

  • Jul 21 02:04
    mergify[bot] closed #1045
  • Jul 21 01:38
    mergify[bot] labeled #1045
  • Jul 21 01:37
    scala-steward opened #1045
  • Jul 20 14:32

    mergify[bot] on master

    Update slf4j-api to 1.7.32 Merge pull request #1044 from s… (compare)

  • Jul 20 14:32
    mergify[bot] closed #1044
  • Jul 20 14:06
    mergify[bot] labeled #1044
  • Jul 20 14:06
    scala-steward opened #1044
  • Jul 19 11:38
    mergify[bot] labeled #1043
  • Jul 19 11:38
    scala-steward opened #1043
  • Jul 19 08:52

    adamw on master

    Added separate instance for Mon… Merge pull request #1042 from m… (compare)

  • Jul 19 08:52
    adamw closed #1042
Joshua Portway
@jportway
Hi - I'm trying to Stub a Websocket, but as soon as I try to open the websocket using the stub backend the websocket terminates immediately - my websocket code never even seems to receieve the initial message. The actual code itself seems to work fine when using a real live backend, it only fails with the stubbed backend. Here is my stubbing code :
        val backend = HttpClientZioBackend.stubLayer

        val webSocketStub = WebSocketStub
          .initialReceive(
            List(WebSocketFrame.text("Hello from the server!"))
          )
          .thenRespond{ _ => List(WebSocketFrame.text("nothing more to say here")) }

        val stubEffect:URIO[SttpClientStubbing with Logging, Unit]  = for {
          _ <- log.info("stubbing")
          _ <- whenAnyRequest.thenRespond(webSocketStub)
        } yield ()
Joshua Portway
@jportway

Here is my send code:

send(basicRequest.get(uri"wss://website.com/blah").response(responseType)).tap(res => log.info(s"websocket request fibre terminated with $res")).fork

When using a normal backend, that send task will keep going until the websocket gets closed. But when I use the stubbed backend it seems to close immediately and I get this logged :

websocket request fibre terminated with Response(sttp.ws.testing.WebSocketStub@7ce08c7f,200,OK,List(),List(),RequestMetadata(GET,http://example.com,List()))
Joshua Portway
@jportway
I still have no joy with the problem I described above - as far as I can tell I'm following the documentation for stubbing pretty much exactly and no amount of blind fiddling seems to produce a different result. When the stubbed backend returns a WebSocketStub the request seems to end immediately with the WebsocketStub itself as the body of the response. Should I file a bug for this, or am I completely misunderstanding something?
7 replies
Jonathan Neufeld
@jonathan-neufeld-asurion
I'm running into a dependency conflict and trying to sort it out, what version of Sttp model is supposed to pair with Sttp core 2.2.9?
3 replies
Jonathan Neufeld
@jonathan-neufeld-asurion
Ok I think I'm officially in JAR hell w.r.t. Sttp model / core
Circe client wants v1.1.4 but Tapir wants 1.3.4 (of model)
5 replies
William Haw
@williamhaw
hi, I have some legacy clients that are using a Future-based backend, however I would like to use a ZIO backend instead.
so far I have this:
class FutureToZioBackend[S](delegate: SttpBackend[Task, S]) extends SttpBackend[Future, S] {

  override def send[T, R >: S with capabilities.Effect[Future]](request: Request[T, R]): Future[Response[T]] =
    Runtime.default.unsafeRunToFuture(delegate.send(request))

  override def close(): Future[Unit] = Runtime.default.unsafeRunToFuture(delegate.close())

  override def responseMonad: MonadError[Future] = ???
}
  1. Is there a better way than to use unsafeRunToFuture in the send() method?
  2. how do I convert delegate.responseMonad which is of type MonadError[Task] into MonadError[Future]?
5 replies
Fernando Mora
@fernandomora

Taking a look to https://sttp.softwaremill.com/en/latest/responses/body.html#streaming it says
"The “unsafe” variants return the stream directly to the user, and then it’s up to the user of the code to consume and close the stream, releasing any resources held by the HTTP connection."

How can I "close" the returned stream?

I have checked the code for the http4s backend and It seems ResponseAsStreamUnsafe is discarding the closing Stream function https://github.com/softwaremill/sttp/blob/8dcc4878b9f400fb45c10d80e5878f5a3d605830/core/src/main/scala/sttp/client3/internal/BodyFromResponseAs.scala#L54
so the signalBodyComplete effect seems never run https://github.com/softwaremill/sttp/blob/8dcc4878b9f400fb45c10d80e5878f5a3d605830/http4s-backend/src/main/scala/sttp/client3/http4s/Http4sBackend.scala#L60
and the Deferred responseBodyCompleteVar would be never got https://github.com/softwaremill/sttp/blob/8dcc4878b9f400fb45c10d80e5878f5a3d605830/http4s-backend/src/main/scala/sttp/client3/http4s/Http4sBackend.scala#L74

Could this be problematic? How can I assure the connection is being released?

2 replies
harrylaou
@harrylaou:matrix.org
[m]

Hi ,
I am migrating a service that was using akka-http to sttp-zio

By using AsyncHttpClientZioBackend.layer() I am getting a java.io.IOException: Too many open files
My current understanding is that this will create many "backends" and these reserve tcp connections.
Whats is the recommendation? should I just increase ulimit in the deployment or try to use a single backend and inject it to the layer , something like

private val sttpClient: AsyncHttpClient = asyncHttpClient()
  val sttpClientLayer: Layer[Throwable, SttpClient] = AsyncHttpClientZioBackend.layerUsingClient(sttpClient)

(I assume that this just creates a singleton backend in the service)

???

2 replies
mvillafuertem
@mvillafuertem
Hi there @adamw, I am trying to test a simple code using sttp and zio with repeat and I get this In Suite "SttpRequestWithRepeatPolicySpec$", test "Test 1" has taken more than 1 m to execute. If this is not expected, consider using TestAspect.timeout to timeout runaway tests for faster diagnostics.
9 replies

object SttpRequestWithRepeatPolicySpec extends DefaultRunnableSpec {

  // @see https://requestbin.com/r/ene80m1n53nb
  implicit val customConfig: Configuration                        = Configuration.default
  val backend: Task[SttpBackend[Task, Nothing, WebSocketHandler]] = AsyncHttpClientZioBackend()

  case class Response(success: Boolean)
  private val uri        = "https://ene80m1n53nb.x.pipedream.net/"
  private val requestGET = basicRequest.get(uri"$uri").response(asJson[Response])

  override def spec: Spec[TestEnvironment, TestFailure[Throwable], TestSuccess] =
    suite(getClass.getSimpleName)(
      testM(s"Test 1")(
        assertM(
          backend.flatMap { implicit backend =>
            for {
              fiber <- requestGET
                         .send()
                         .map(_.body)
                         .absolve
                         .repeat(
                           (Schedule.spaced(2.second) >>>
                             Schedule.recurWhile[Long](_ < 5))
                             .tapOutput[Console](n => putStr(n.toString + " ").exitCode) *>
                             Schedule
                               .collectAll[Response]
                               .tapInput[Console, Response](response => putStrLn(response.toString).exitCode)
                         )
                         .catchAll(a =>
                           zio.console.putStr(a.getMessage) >>>
                             RIO.effect(Chunk(Response(false)))
                         )
                         .fork
              _     <- TestClock.adjust(20.seconds)
              _     <- fiber.join.ensuring(backend.close().ignore)
            } yield ()
          }
        )(equalTo(()))
      )
    )
}
Here the full code https://raw.githubusercontent.com/mvillafuertem/scala/4533aac8dcace8ebc7ec28ef480c00df575fdae8/modules/script/SttpRequestWithRepeatPolicy.sc
Glen Marchesani
@fizzy33
Anyone know the status on cats effect 3 ? I can see it in master and unless the naming convention has changed I don't see any ce3 artifacts out there... thanks in advance
2 replies
λoλcat
@catostrophe
@adamw http4s team is planning to release 0.23.x as a "stable" long-term series compatible with CE3, leaving 1.0.0-Mxx for further work until 1.0.0 is released (which will take quite a long time I suppose).
I think it makes sense to downgrade http4s to 0.23.x when it's published in both sttp and tapir. what do you think?
2 replies
harrylaou
@harrylaou:matrix.org
[m]

Hi ,

I am using sttp-zio and when running some tests with stubbing I am getting

Fiber failed.
An unchecked error was produced.
java.lang.NullPointerException

I can recover with catchAllCause, I have the stack trace but cannot see why this is failing. Any ideas? Has anyone seen this before when stubbing?

harrylaou
@harrylaou:matrix.org
[m]

:point_up: Edit: Hi ,

I am using sttp-zio and when running some tests with stubbing I am getting

Fiber failed.
An unchecked error was produced.
java.lang.NullPointerException

I can recover with catchAllCause, I have the stack trace but cannot see why this is failing.
Using it with unsafeRunToFuture

Any ideas? Has anyone seen this before when stubbing?

10 replies
Glen Marchesani
@fizzy33
Is there a way to build a backend that can stream and do websockets ?
Glen Marchesani
@fizzy33
oh my issue is in scala js. Is there any way to do websockets in scala js seems a better way to ask that.
Glen Marchesani
@fizzy33
looks like the answer is no, I guess in scala js just use websocket api directly
17 replies
Glen Marchesani
@fizzy33
okay I am at a loss on this one
          request
            .response(asStream[F, fs2.Stream[F,ByteBuffer]])
            .readTimeout(Duration.Inf)
            .post(uri)
            .send()
1 reply
error message is
wrong number of type parameters for method asStream: [F[_], T, S](s: sttp.capabilities.Streams[S])(f: s.BinaryStream => F[T]): sttp.client3.ResponseAs[Either[String,T],sttp.capabilities.Effect[F] with S]
            .response(asStream[F, fs2.Stream[F,ByteBuffer]])
I suspect my issue is I ideally want to use the same sttp backend for streaming bytes (for server sent events) and for websockets. Not sure if I can do that
the sttp backend that is present is not a specific container it is
  val instance: F[sttp.client3.SttpBackend[F, WebSockets]]
oof that error message is wrong the error messsage is
/Users/glen/code/accur8/remote-api/client/shared/src/main/scala/a8/remoteapi/client/DefaultSttpBackend.scala:31:16
Cannot prove that sttp.capabilities.WebSockets with sttp.capabilities.Effect[[_]F[_]] <:< Nothing.
          .send(backend)
Glen Marchesani
@fizzy33
okay I worked around that by having two backends one for WebSockets and one for Fs2Streams
2 replies
abstract class DefaultSttpBackend[F[_] : Async] {

  val instance: F[sttp.client3.SttpBackend[F, WebSockets]] = ...
  val instance2: F[sttp.client3.SttpBackend[F, Fs2Streams[F]]] = ...

  ...

}
is there a way to get both capabilities with a single backend instance ?
Glen Marchesani
@fizzy33
sorry for all the noise... I took on a large refactoring of a scala js and jvm app using monix to using cats and fs2. I see most of my issues stem now from fs2 not being in scala js (where monix is in both).
I have restarted and doing just jvm first.
Glen Marchesani
@fizzy33
I have my core question which is this, do I need three separate backends to do streams, websockets and regular or can I do all those with the same one ?
  val backendN: SttpBackend[F,Nothing] = ???
  val backendStream: SttpBackend[F,Fs2Streams[F]] = ???
  val backendWs: SttpBackend[F,WebSockets] = ???
In this use case I am AsyncHttpClientBackend so I can minimally re-use asyncHttpClient: AsyncHttpClient
across all three
Glen Marchesani
@fizzy33
oh snap @adam sorry I just saw your reply now to this.
i.e. -- just use with: SttpBackend[F, WebSockets with Fs2Streams[F]]
Glen Marchesani
@fizzy33
that is all working now thanks @adamw
Glen Marchesani
@fizzy33

one last one. I worked around this with a strategically placed .asInstanceOf[] but have created an isolated reproduction to see if/what I am missing


import cats.effect.IO
import sttp.client3.impl.cats.FetchCatsBackend
import sttp.client3._

object CannotGetThereFromHere {

  val backend = FetchCatsBackend[IO]()

  val responseIO = basicRequest.send(backend)

}

That gives a compile error of.

Cannot prove that sttp.client3.Empty[Unit] =:= sttp.client3.Identity[Unit].
  val responseIO = basicRequest.send(backend)
3 replies
Shouldn't that "just work (tm)", am I missing an import or something
William Haw
@williamhaw

ok figured it out w.r.t delegating from Future backend to ZIO backend:

class FutureToZioBackend[S](val delegate: SttpBackend[Task, S])(implicit ec: ExecutionContext)
    extends SttpBackend[Future, S] {

  override def send[T, R >: S with Effect[Future]](request: Request[T, R]): Future[Response[T]] =
    Runtime.default.unsafeRunToFuture(
      delegate.send(
        MapEffect.apply[Future, Task, Identity, T, S](
          request: Request[T, S with Effect[Future]],
          futureToTask,
          taskToFuture,
          this.responseMonad,
          delegate.responseMonad
        )
      )
    )

  private val futureToTask = new FunctionK[Future, Task] {
    override def apply[A](fa: Future[A]): Task[A] = ZIO.fromFuture(implicit ec => fa)
  }

  private val taskToFuture = new FunctionK[Task, Future] {
    override def apply[A](ta: Task[A]): Future[A] = Runtime.default.unsafeRunToFuture(ta)
  }

  override def close(): Future[Unit] = Runtime.default.unsafeRunToFuture(delegate.close())

  override val responseMonad: MonadError[Future] = new FutureMonad()
}

thanks @adamw

Daniel Vigovszky
@vigoo
hi! is there any special reason why slf4j-backend is not released for Scala 3? - if not I'd open an issue / PR
2 replies
vonchav
@voonchav_gitlab
@adamw I think Circe 0.14.1 can be used by Scala 2.x too. Any thoughts?
1 reply
Gerard Downes
@GDownes
Hi all, quite new to this.
I've been using the sttp client with different backends. I'd like to be able to modify/intercept the request and response of a request regardless of backend.
I have tried wrapping the SttpBackend send method, but the F type has made it difficult accessing the response. Would anyone have any ideas for implementing this type of interceptor, I'd like to be able to modify outgoing and receive incoming headers?
4 replies
Glen Marchesani
@fizzy33

So I am trying to get a simple websocket app working.


import a8.remoteapi.client.JvmHttpClient
import cats.effect.{Async, IO, IOApp, Sync}
import org.typelevel.log4cats.slf4j.Slf4jLogger
import sttp.client3.asynchttpclient.fs2.AsyncHttpClientFs2Backend
import sttp.client3._
import sttp.ws.WebSocket

import java.util.concurrent.TimeUnit
import scala.concurrent.duration.FiniteDuration

object SimpleWebSocket extends IOApp.Simple {

//  val wsUri = uri"ws://localhost:8080/mailbox/ws"
  val wsUri = uri"wss://echo.websocket.org"

  override def run: IO[Unit] = {
    AsyncHttpClientFs2Backend.resource[IO]().use { backend =>

      val webSocketIO =
        basicRequest
          .get(wsUri)
          .response(asWebSocketAlways[IO,WebSocket[IO]](ws => Async[IO].pure(ws)))
          .send(backend)
          .map(_.body)

      for {
        websocket <- webSocketIO
        isOpen <- websocket.isOpen()
        _ <- IO.println(isOpen)
        m0 <- websocket.receiveText()
        _ <- IO.print(m0)
      } yield ()
    }
  }

}

When I run I get that the web socket is closed, though in the following logs the web socket was successfully open

15:24:42.643 [AsyncHttpClient-3-1] DEBUG org.asynchttpclient.netty.handler.WebSocketHandler - 

Request DefaultFullHttpRequest(decodeResult: success, version: HTTP/1.1, content: EmptyByteBufBE)
GET / HTTP/1.1
accept-encoding: gzip, deflate
upgrade: websocket
connection: upgrade
sec-websocket-key: yLQXgVQ5t42dT7YrB0hNDw==
sec-websocket-version: 13
origin: https://echo.websocket.org
host: echo.websocket.org
accept: */*
user-agent: AHC/2.1

Response DefaultHttpResponse(decodeResult: success, version: HTTP/1.1)
HTTP/1.1 101 Web Socket Protocol Handshake
Connection: Upgrade
Date: Thu, 03 Jun 2021 19:16:27 GMT
Sec-WebSocket-Accept: XzSACqZ5kqTZpo4+wo2g9pdUSjk=
Server: Kaazing Gateway
Upgrade: websocket

false
Exception in thread "main" sttp.ws.WebSocketClosed
    at sttp.client3.asynchttpclient.WebSocketImpl.$anonfun$send$1(WebSocketImpl.scala:53)
    at cats.effect.IOFiber.runLoop(IOFiber.scala:383)
    at cats.effect.IOFiber.asyncContinueSuccessfulR(IOFiber.scala:1132)
    at cats.effect.IOFiber.run(IOFiber.scala:126)
    at cats.effect.unsafe.WorkerThread.run(WorkerThread.scala:359)
15:24:44.851 [AsyncHttpClient-3-1] DEBUG io.netty.buffer.PoolThreadCache - Freed 13 thread-local buffer(s) from thread: AsyncHttpClient-3-1
Glen Marchesani
@fizzy33
This message was deleted
I can see the websocket gets closed right after it gets opened but can't figure out why
in case it was the response getting managed I tweaked the code with the same result
  override def run: IO[Unit] = {
    AsyncHttpClientFs2Backend.resource[IO]().use { backend =>

      basicRequest
        .get(wsUri)
        .response(asWebSocketAlways[IO,WebSocket[IO]](ws => Async[IO].pure(ws)))
        .send(backend)
        .flatMap { response =>
          val websocket = response.body
          for {
            _ <- websocket.sendText("foo")
            isOpen <- websocket.isOpen()
            _ <- IO.println(isOpen)
    //        _ <- websocket.sendText(initialMessageJson)
            m0 <- websocket.receiveText()
            _ <- IO.print(m0)
          } yield ()
        }
    }
  }
3 replies
Brent
@brentsony

Is there a special API for getting binary data?

val req = quickRequest.readTimeout(timeoutMs.millis).get(uri"""$url""")
val backend = HttpURLConnectionBackend()
val x = Try { req.send(backend) }.fold(e => Failure(e), resp =>

resp.body.getBytes // is approx. twice the size of resp.contentLength!

Glen Marchesani
@fizzy33
I am not in front of a computer something like this though
    request
      .response(asByteArray)
Brent
@brentsony
For example, where url returns an image (such as a .png).
Glen Marchesani
@fizzy33
so
val req = quickRequest.readTimeout(timeoutMs.millis).get(uri"""$url""").response(asByteArray)