Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jul 28 17:26
    adamw commented #675
  • Jul 28 14:37
    vakhtang commented #675
  • Jul 28 10:22

    adamw on v3.3.12

    (compare)

  • Jul 28 10:22

    adamw on master

    Update dependencies Release 3.3.12 (compare)

  • Jul 28 09:48
    adamw closed #1036
  • Jul 28 09:48
    adamw closed #1046
  • Jul 28 09:46
    adamw commented #1052
  • Jul 28 09:46

    adamw on master

    update client if request-timeou… Merge pull request #1052 from m… (compare)

  • Jul 28 09:46
    adamw closed #1052
  • Jul 28 09:20

    mergify[bot] on master

    Update cats-effect, cats-effect… Merge pull request #1048 from s… (compare)

  • Jul 28 09:20
    mergify[bot] closed #1048
  • Jul 28 08:54

    adamw on master

    Update json4s-core, json4s-nati… Merge pull request #1049 from s… (compare)

  • Jul 28 08:54
    adamw closed #1049
  • Jul 27 19:34

    mergify[bot] on master

    Update zio, zio-streams to 1.0.… Merge pull request #1053 from s… (compare)

  • Jul 27 19:34
    mergify[bot] closed #1053
  • Jul 27 19:08
    mergify[bot] labeled #1053
  • Jul 27 19:08
    scala-steward opened #1053
  • Jul 27 14:03
    markusheilig opened #1052
  • Jul 27 09:29

    mergify[bot] on master

    Update akka-http to 10.2.5 Merge pull request #1051 from s… (compare)

  • Jul 27 09:29
    mergify[bot] closed #1051
Glen Marchesani
@fizzy33
oh snap @adam sorry I just saw your reply now to this.
i.e. -- just use with: SttpBackend[F, WebSockets with Fs2Streams[F]]
Glen Marchesani
@fizzy33
that is all working now thanks @adamw
Glen Marchesani
@fizzy33

one last one. I worked around this with a strategically placed .asInstanceOf[] but have created an isolated reproduction to see if/what I am missing


import cats.effect.IO
import sttp.client3.impl.cats.FetchCatsBackend
import sttp.client3._

object CannotGetThereFromHere {

  val backend = FetchCatsBackend[IO]()

  val responseIO = basicRequest.send(backend)

}

That gives a compile error of.

Cannot prove that sttp.client3.Empty[Unit] =:= sttp.client3.Identity[Unit].
  val responseIO = basicRequest.send(backend)
3 replies
Shouldn't that "just work (tm)", am I missing an import or something
William Haw
@williamhaw

ok figured it out w.r.t delegating from Future backend to ZIO backend:

class FutureToZioBackend[S](val delegate: SttpBackend[Task, S])(implicit ec: ExecutionContext)
    extends SttpBackend[Future, S] {

  override def send[T, R >: S with Effect[Future]](request: Request[T, R]): Future[Response[T]] =
    Runtime.default.unsafeRunToFuture(
      delegate.send(
        MapEffect.apply[Future, Task, Identity, T, S](
          request: Request[T, S with Effect[Future]],
          futureToTask,
          taskToFuture,
          this.responseMonad,
          delegate.responseMonad
        )
      )
    )

  private val futureToTask = new FunctionK[Future, Task] {
    override def apply[A](fa: Future[A]): Task[A] = ZIO.fromFuture(implicit ec => fa)
  }

  private val taskToFuture = new FunctionK[Task, Future] {
    override def apply[A](ta: Task[A]): Future[A] = Runtime.default.unsafeRunToFuture(ta)
  }

  override def close(): Future[Unit] = Runtime.default.unsafeRunToFuture(delegate.close())

  override val responseMonad: MonadError[Future] = new FutureMonad()
}

thanks @adamw

Daniel Vigovszky
@vigoo
hi! is there any special reason why slf4j-backend is not released for Scala 3? - if not I'd open an issue / PR
2 replies
vonchav
@voonchav_gitlab
@adamw I think Circe 0.14.1 can be used by Scala 2.x too. Any thoughts?
1 reply
Gerard Downes
@GDownes
Hi all, quite new to this.
I've been using the sttp client with different backends. I'd like to be able to modify/intercept the request and response of a request regardless of backend.
I have tried wrapping the SttpBackend send method, but the F type has made it difficult accessing the response. Would anyone have any ideas for implementing this type of interceptor, I'd like to be able to modify outgoing and receive incoming headers?
4 replies
Glen Marchesani
@fizzy33

So I am trying to get a simple websocket app working.


import a8.remoteapi.client.JvmHttpClient
import cats.effect.{Async, IO, IOApp, Sync}
import org.typelevel.log4cats.slf4j.Slf4jLogger
import sttp.client3.asynchttpclient.fs2.AsyncHttpClientFs2Backend
import sttp.client3._
import sttp.ws.WebSocket

import java.util.concurrent.TimeUnit
import scala.concurrent.duration.FiniteDuration

object SimpleWebSocket extends IOApp.Simple {

//  val wsUri = uri"ws://localhost:8080/mailbox/ws"
  val wsUri = uri"wss://echo.websocket.org"

  override def run: IO[Unit] = {
    AsyncHttpClientFs2Backend.resource[IO]().use { backend =>

      val webSocketIO =
        basicRequest
          .get(wsUri)
          .response(asWebSocketAlways[IO,WebSocket[IO]](ws => Async[IO].pure(ws)))
          .send(backend)
          .map(_.body)

      for {
        websocket <- webSocketIO
        isOpen <- websocket.isOpen()
        _ <- IO.println(isOpen)
        m0 <- websocket.receiveText()
        _ <- IO.print(m0)
      } yield ()
    }
  }

}

When I run I get that the web socket is closed, though in the following logs the web socket was successfully open

15:24:42.643 [AsyncHttpClient-3-1] DEBUG org.asynchttpclient.netty.handler.WebSocketHandler - 

Request DefaultFullHttpRequest(decodeResult: success, version: HTTP/1.1, content: EmptyByteBufBE)
GET / HTTP/1.1
accept-encoding: gzip, deflate
upgrade: websocket
connection: upgrade
sec-websocket-key: yLQXgVQ5t42dT7YrB0hNDw==
sec-websocket-version: 13
origin: https://echo.websocket.org
host: echo.websocket.org
accept: */*
user-agent: AHC/2.1

Response DefaultHttpResponse(decodeResult: success, version: HTTP/1.1)
HTTP/1.1 101 Web Socket Protocol Handshake
Connection: Upgrade
Date: Thu, 03 Jun 2021 19:16:27 GMT
Sec-WebSocket-Accept: XzSACqZ5kqTZpo4+wo2g9pdUSjk=
Server: Kaazing Gateway
Upgrade: websocket

false
Exception in thread "main" sttp.ws.WebSocketClosed
    at sttp.client3.asynchttpclient.WebSocketImpl.$anonfun$send$1(WebSocketImpl.scala:53)
    at cats.effect.IOFiber.runLoop(IOFiber.scala:383)
    at cats.effect.IOFiber.asyncContinueSuccessfulR(IOFiber.scala:1132)
    at cats.effect.IOFiber.run(IOFiber.scala:126)
    at cats.effect.unsafe.WorkerThread.run(WorkerThread.scala:359)
15:24:44.851 [AsyncHttpClient-3-1] DEBUG io.netty.buffer.PoolThreadCache - Freed 13 thread-local buffer(s) from thread: AsyncHttpClient-3-1
Glen Marchesani
@fizzy33
This message was deleted
I can see the websocket gets closed right after it gets opened but can't figure out why
in case it was the response getting managed I tweaked the code with the same result
  override def run: IO[Unit] = {
    AsyncHttpClientFs2Backend.resource[IO]().use { backend =>

      basicRequest
        .get(wsUri)
        .response(asWebSocketAlways[IO,WebSocket[IO]](ws => Async[IO].pure(ws)))
        .send(backend)
        .flatMap { response =>
          val websocket = response.body
          for {
            _ <- websocket.sendText("foo")
            isOpen <- websocket.isOpen()
            _ <- IO.println(isOpen)
    //        _ <- websocket.sendText(initialMessageJson)
            m0 <- websocket.receiveText()
            _ <- IO.print(m0)
          } yield ()
        }
    }
  }
3 replies
Brent
@brentsony

Is there a special API for getting binary data?

val req = quickRequest.readTimeout(timeoutMs.millis).get(uri"""$url""")
val backend = HttpURLConnectionBackend()
val x = Try { req.send(backend) }.fold(e => Failure(e), resp =>

resp.body.getBytes // is approx. twice the size of resp.contentLength!

Glen Marchesani
@fizzy33
I am not in front of a computer something like this though
    request
      .response(asByteArray)
Brent
@brentsony
For example, where url returns an image (such as a .png).
Glen Marchesani
@fizzy33
so
val req = quickRequest.readTimeout(timeoutMs.millis).get(uri"""$url""").response(asByteArray)
making sure you have
import sttp.client3._
I think should do it for you
Brent
@brentsony
Yep - that did it! Thank you.
Andrii Ilin
@AndreyIlin
Hi, guys.
Where can I find some kind of compatibility matrix for sttp-model, getting NoClassDefFound when try to use tapir 0.15.x and sttp client3 in the same project?
1 reply
eugeniyk
@eugeniyk

Hi, guys.
Where can I find some kind of compatibility matrix for sttp-model, getting NoClassDefFound when try to use tapir 0.15.x and sttp client3 in the same project?

Faced same in our projects
Concrete classes:

  1. MediaType.copy
    https://www.javadoc.io/doc/com.softwaremill.sttp.model/core_2.12/1.4.0/sttp/model/MediaType.html. vs
    https://www.javadoc.io/doc/com.softwaremill.sttp.model/core_2.12/1.1.4/sttp/model/MediaType.html

  2. Uri (host: String vs host: Option[String])

@adamw I think it's time to release core v2 with new package names to achieve compatibility between v2 and v3
at the same time you may want to consider using MIMA?

4 replies
etienne
@crakjie

is it cheating to do

.response(asStream(MonixStreams)(i => Task(i)))

instead of

.response(asStreamUnsafe(MonixStreams))

?

1 reply
Glen Marchesani
@fizzy33
I am trying to limit the total number of http connections at any given moment. Is there an idiomatic way to do this ? I tried using the AsyncHttpCLient setting for that BUT the AsyncHttpClient appears to fail when you meet that number instead of waiting for a connection...
11:23:50.465 | DEBUG | 18 | a8.sync.http | http request failed on the 2 retry -- https://rm-api-dev-1-ext.goodville-dev.assure.dxc.com/riskmasterapi/events/34864/claimgc/12420/claimants - (http.scala:42)
sttp.client3.SttpClientException$ReadException: Exception when sending request: GET https://rm-api-dev-1-ext.goodville-dev.assure.dxc.com/riskmasterapi/events/34864/claimgc/12420/claimants
    at sttp.client3.SttpClientException$.defaultExceptionToSttpClientException(SttpClientException.scala:44)
    at sttp.client3.asynchttpclient.AsyncHttpClientBackend.$anonfun$adjustExceptions$1(AsyncHttpClientBackend.scala:227)
    at sttp.client3.SttpClientException$$anonfun$adjustExceptions$1.applyOrElse(SttpClientException.scala:57)
    at sttp.client3.SttpClientException$$anonfun$adjustExceptions$1.applyOrElse(SttpClientException.scala:56)
    at cats.ApplicativeError.$anonfun$recoverWith$1(ApplicativeError.scala:106)
    at cats.effect.IOFiber.handleErrorWithK(IOFiber.scala:1283)
    at cats.effect.IOFiber.failed(IOFiber.scala:1057)
    at cats.effect.IOFiber.uncancelableFailureK(IOFiber.scala:1308)
    at cats.effect.IOFiber.failed(IOFiber.scala:1059)
    at cats.effect.IOFiber.onCancelFailureK(IOFiber.scala:1296)
    at cats.effect.IOFiber.failed(IOFiber.scala:1058)
    at cats.effect.IOFiber.unmaskFailureK(IOFiber.scala:1318)
    at cats.effect.IOFiber.failed(IOFiber.scala:1060)
    at cats.effect.IOFiber.onCancelFailureK(IOFiber.scala:1296)
    at cats.effect.IOFiber.failed(IOFiber.scala:1058)
    at cats.effect.IOFiber.runLoop(IOFiber.scala:752)
    at cats.effect.IOFiber.asyncContinueSuccessfulR(IOFiber.scala:1132)
    at cats.effect.IOFiber.run(IOFiber.scala:126)
    at cats.effect.unsafe.WorkerThread.run(WorkerThread.scala:359)
Caused by: org.asynchttpclient.exception.TooManyConnectionsException: Too many connections: 20
anyone have any recommendations on how to do this in a composable way...
I could hack this to make it work inside our retry framework but I am figuring someone has had to do this already
hmm even simpler I can just wrap my use in a semaphore
3 replies
Glen Marchesani
@fizzy33
nm and thanks for listening
harrylaou
@harrylaou:matrix.org
[m]

Hi ,
I am trying to stub a response to be used in a asStreamUnsafe response.
When using a asByteArray response the stub is working.

When using the a asStreamUnsafe response
I am getting

An unchecked error was produced.
java.lang.ClassCastException: [B incompatible with scala.util.Either

I have prepared a scastie similar to what I am doing. https://scastie.scala-lang.org/QOf83EyOSZCQsfnGoZrUPg
The only difference is that instead of a typed error I am using a throwable in scastie.

and the error in scastie is

An unchecked error was produced.
java.lang.ClassCastException: java.lang.String cannot be cast to scala.util.Either

I am not sure if it is related to softwaremill/sttp#389, and is a similar thing that has to do with the stub
or if I am doing something wrong ?

Any help would be appreciated.

harrylaou
@harrylaou:matrix.org
[m]
Stubbing works for asStreamAlwaysUnsafe and when stubbing with .thenRespond(Response.ok(ZStream.fromIterable("Stub response".getBytes)))
( I have updated the scastie link)
What would be the correct way to stub for asStreamUnsafe ?
7 replies
Jeremiah Malina
@jjmalina

I'm trying to upgrade a spark app from Scala 2.11 to Scala 2.12 (I know it's old..) but currently running into this issue on sttp 2.2.9:

java.lang.NoSuchMethodError: scala.Product.$init$(Lscala/Product;)V
    at sttp.client.SttpBackendOptions.<init>(SttpBackendOptions.scala:12)
    at sttp.client.SttpBackendOptions$.<init>(SttpBackendOptions.scala:100)
    at sttp.client.SttpBackendOptions$.<clinit>(SttpBackendOptions.scala)
    at sttp.client.HttpURLConnectionBackend$.apply$default$1(HttpURLConnectionBackend.scala:308)
    at com.newscred.services.NewsCredAnalyticsAPIClient.<init>(NewsCredAnalyticsAPI.scala:46)

The offending line of code is implicit var backend = HttpURLConnectionBackend()
Anyone have an idea what the issue might be? Thanks in advance :)

harrylaou
@harrylaou:matrix.org
[m]
@jjmalina: usually , when NoSuchMethodError happens, it is a mixup with library versions.
you can use sbt dependencyTree to see the versions of the library in question, and then use exclude in build.sbt exclude transitive dependencies
Jeremiah Malina
@jjmalina
@harrylaou:matrix.org thanks! how do I know which transitive dependencies to exclude? is it correct to assume that the java.lang.NoSuchMethodError: scala.Product.$init$(Lscala/Product;) error implies that the wrong version of scala is being used somehow?
it looks like everything is ok with scala 2.12
Jeremiah Malina
@jjmalina
Turns out that the issue was that I was running my app in EMR on Spark which was running Scala 2.11, upgraded EMR and it solved the error I got with sttp, though now I'm having a similar error with json4s :/
Saad Khawaja
@SaadKhawaja
The sttp doc page here https://sttp.softwaremill.com/en/latest/backends/http4s.html ask to use "com.softwaremill.sttp.client3" %% "http4s-backend" % "3.3.6" but this is not in maven central. Where could this be found. Thanks.
8 replies
Philippe Derome
@phderome
I’ve been toying with WebSocketZio and WebSocketStreamFs2 examples and I see the Fs2 example uses API asWebSocketStream which is quite nice and useful and the Zio one uses asWebSocketAlways which while useful is a bit less interesting. Would it make sense to extend both examples to achieve some parity on API usage (i.e. both using asWebSocketStream and asWebSocketAlways)? I didn’t look at the Akka example mind you.
4 replies
If there’s some interest, I could do a PR to add asWebSocketStream to th ZIO solution.
Philippe Derome
@phderome
Ashwin Bhaskar
@ashwinbhaskar
How can I mock requests and response when using AsyncHttpClientZioBackend.stubLayer? When using AsyncHttpClientZioBackend.stub I could do AsyncHttpClientZioBackend.stub.whenRequestMatches(...).thenResponse(...).
2 replies
Łukasz Drygała
@ldrygala
Hi guys, I would like to test a client that streams data from a websocket. Unfortunately, it fails with class sttp.ws.testing.WebSocketStub cannot be cast to class akka.stream.scaladsl.Flow (sttp.ws.testing.WebSocketStub and akka.stream.scaladsl.Flow are in unnamed module of loader 'app')
4 replies
Ashwin Bhaskar
@ashwinbhaskar
Is there an equivalent of akka.http.scaladsl.model.Uri's parseAndResolve for sttp.model.Uri?
1 reply
rabzu
@rabzu
How can I get BinaryStream from
(sttp.capabilities.Streams[sttp.capabilities.zio.ZioStreams] & Singleton)# BinaryStream?
I have:
val fileStream: (Streams[ZioStreams] & Singleton)#BinaryStream <- sttpClient
                            .send(downloadRequest)

uploadRequest = basicRequest.streamBody(ZioStreams)(fileStream)
Found:    (fileStream : 
  (sttp.capabilities.Streams[sttp.capabilities.zio.ZioStreams] & Singleton)#
    BinaryStream
)
Required: sttp.capabilities.zio.ZioStreams.BinaryStream²

where:    BinaryStream  is a type in class Streams with bounds 
          BinaryStream² is a type in trait ZioStreams which is an alias of zio.stream.Stream[Throwable, Byte]
rabzu
@rabzu
So error is in basicRequest.streamBody(ZioStreams)(fileStream)
9 replies
λoλcat
@catostrophe
The latest tapir 0.18.0 is CE2-only compatible and its http4s module depends on http4s 0.22.0-RC1, while sttp 3.3.9 CE2 http4s module depends on http4s 0.21.x, so they are incompatible. It may be force-fixed in sbt but the risk of runtime bincompat issues is high
6 replies
truongio
@truongio

Hi, I'm getting some weird EOF error from sttp that I'm unsure about.

o.h.b.p.Command$EOF$: EOF
from:

logger_name: "sttp.client3.logging.slf4j.Slf4jLoggingBackend"
logger_name: "sttp.tapir.server.http4s.Http4sServerInterpreter"

I'm using sttp client version 3.2.3. Does anyone know what might be causing this and how to resolve it?

3 replies