Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 15:03
    adamw commented #1115
  • Sep 22 19:16
    adamw commented #1003
  • Sep 22 19:11
    nickrobison edited #1115
  • Sep 22 19:11
    nickrobison edited #1115
  • Sep 22 19:11
    nickrobison edited #1115
  • Sep 22 19:10
    nickrobison opened #1115
  • Sep 22 12:46
    mvillafuertem commented #1003
  • Sep 22 10:05
    bartekzylinski synchronize #1113
  • Sep 22 10:05

    bartekzylinski on empty-gzip-fix

    Changed condition for body deco… (compare)

  • Sep 22 09:58
    bartekzylinski synchronize #1113
  • Sep 22 09:58

    bartekzylinski on empty-gzip-fix

    Changed method of checking if b… Removed depreacted akka-http API (compare)

  • Sep 22 06:55
    bartekzylinski synchronize #1113
  • Sep 22 06:55

    bartekzylinski on empty-gzip-fix

    Fixed handling of empty gziped … (compare)

  • Sep 21 16:17

    mergify[bot] on master

    Update http4s-blaze-client, htt… Merge pull request #1114 from s… (compare)

  • Sep 21 16:17
    mergify[bot] closed #1114
  • Sep 21 15:47
    mergify[bot] labeled #1114
  • Sep 21 15:47
    scala-steward opened #1114
  • Sep 21 13:35
    bartekzylinski review_requested #1113
  • Sep 21 13:35
    bartekzylinski assigned #1113
  • Sep 21 13:35
    bartekzylinski opened #1113
Felix Dietze
@fdietze
Hi, I'm trying to use FetchCatsBackend[IO] with ScalaJS, but I'm getting this error message, what am I missing?
import sttp.client3.impl.cats.FetchCatsBackend
import cats.effect._
import sttp.client3._

  implicit val ec: scala.concurrent.ExecutionContext =
    scala.concurrent.ExecutionContext.global
  implicit val contextShift: ContextShift[IO]        = IO.contextShift(ec)

  // https://sttp.softwaremill.com/en/latest/backends/javascript/fetch.html?highlight=fetch#node-js
  val g         = scalajs.js.Dynamic.global.globalThis
  val nodeFetch = g.require("node-fetch")
  g.fetch = nodeFetch
  g.Headers = nodeFetch.Headers
  g.Request = nodeFetch.Request

  val sttpBackend = FetchCatsBackend[IO]()
[error]      could not find implicit value for evidence parameter of type cats.effect.kernel.Async[cats.effect.IO] (Could not find an instance of Monad for cats.effect.IO)
[error]      Error occurred in an application involving default arguments.
[error]      L59:   val sttpBackend = FetchCatsBackend[IO]()
[error]                                                   ^
Felix Dietze
@fdietze
solved it, I'm working with Cats 2, where the latest sttp release defaults to Cats 3
Felix Dietze
@fdietze
Is node-fetch the only backend for scalajs? I can't get it to work... :(
Seems to be a problem with abortcontroller
1 reply
ex0ns
@ex0ns

Hello
I'm trying to write a generic interface for all my case class that inherits from the same trait, the current trait looks like that

import NodeSerialization.{nodeDecoder, nodeEncoder} 

trait CustomEndpointForNode[T <: Node] {
  implicit private def ct: ClassTag[T] = implicitly[ClassTag[T]]

  def getEndpointDefinition: Endpoint[NodeQueryParams, String, Seq[T], Any]

  def postEndpointDefinition: Endpoint[T, String, T, Any]

  def getBasePath: String
  def getEndpointDescription  = "Get given endpoint type"
  def postEndpointDescription = "Post new given endpoint type"

  def getEndpointDefinition: Endpoint[NodeQueryParams, String, Seq[T], Any] =
    endpoint.get
      .in(getBasePath)
      .in(query[Option[String]]("source_id"))
      .out(jsonBody[Seq[Node]])
      .errorOut(stringBody)
      .description(getEndpointDescription)

  def postEndpointDefinition: Endpoint[T, String, T, Any] =
    endpoint.post
      .in(getBasePath)
      .in(jsonBody[Node])
      .out(jsonBody[Node])
      .errorOut(stringBody)
      .description(postEndPointDescription)

  def getAllEndpointDefinitions: List[Endpoint[_, _, _, _]] = List(getEndpointDefinition, postEndpointDefinition)

  def getItems(params: NodeQueryParams): Future[Either[String, Seq[T]]]
  def postItem(item: T): Future[Either[String, T]]

  def allRoutes(): Route = {
    AkkaHttpServerInterpreter.toRoute(
      List(
        getEndpointDefinition.serverLogic(getItems),
        postEndpointDefinition.serverLogic(postItem)
      )
    )
  }
}

Note that in the jsonBody I use a serialization for the trait Node and not for a given T, this is because the serialization is done automatically using Circe which only provided a Decoder/Encoder for the trait and not for indiviual case classes . is there a way, given the ClassTag[T] to filter (similar to collect) only instances of T in order to retrieve a Seq[T] instead of Seq[Node] ?

thanks !

8 replies
Krzysztof Piszko
@LazyBun
Hello. I'm using SttpBackendOptions to setup a proxy, but I'd like to use this proxy only for specific hosts. Unfortunately, it seems that SttpBackendOptions only supports nonProxyHosts. Before I write an issue and try to implement this, I wanted to make sure that I'm not missing something that's implemented already. So, anybody knows about something like nonProxyHosts opposite?
3 replies
Tim Pigden
@TimPigden

Hi, I'm on 2.2.9. I have a problem that websocket frame size is too small

io.netty.handler.codec.http.websocketx.CorruptedWebSocketFrameException: Max frame length of 10240 has been exceeded.

I'm using zio async backend. How do I change the frame length?

Tim Pigden
@TimPigden
Never mind fixed it:
def bigBuffer = {
    val config = new DefaultAsyncHttpClientConfig.Builder()
      .setWebSocketMaxFrameSize(1024 * 1024 * 2) 
      .build()
    AsyncHttpClientZioBackend.usingConfig(config).toLayer
  }
Joshua Portway
@jportway
Hi - I'm trying to Stub a Websocket, but as soon as I try to open the websocket using the stub backend the websocket terminates immediately - my websocket code never even seems to receieve the initial message. The actual code itself seems to work fine when using a real live backend, it only fails with the stubbed backend. Here is my stubbing code :
        val backend = HttpClientZioBackend.stubLayer

        val webSocketStub = WebSocketStub
          .initialReceive(
            List(WebSocketFrame.text("Hello from the server!"))
          )
          .thenRespond{ _ => List(WebSocketFrame.text("nothing more to say here")) }

        val stubEffect:URIO[SttpClientStubbing with Logging, Unit]  = for {
          _ <- log.info("stubbing")
          _ <- whenAnyRequest.thenRespond(webSocketStub)
        } yield ()
Joshua Portway
@jportway

Here is my send code:

send(basicRequest.get(uri"wss://website.com/blah").response(responseType)).tap(res => log.info(s"websocket request fibre terminated with $res")).fork

When using a normal backend, that send task will keep going until the websocket gets closed. But when I use the stubbed backend it seems to close immediately and I get this logged :

websocket request fibre terminated with Response(sttp.ws.testing.WebSocketStub@7ce08c7f,200,OK,List(),List(),RequestMetadata(GET,http://example.com,List()))
Joshua Portway
@jportway
I still have no joy with the problem I described above - as far as I can tell I'm following the documentation for stubbing pretty much exactly and no amount of blind fiddling seems to produce a different result. When the stubbed backend returns a WebSocketStub the request seems to end immediately with the WebsocketStub itself as the body of the response. Should I file a bug for this, or am I completely misunderstanding something?
7 replies
Jonathan Neufeld
@jonathan-neufeld-asurion
I'm running into a dependency conflict and trying to sort it out, what version of Sttp model is supposed to pair with Sttp core 2.2.9?
3 replies
Jonathan Neufeld
@jonathan-neufeld-asurion
Ok I think I'm officially in JAR hell w.r.t. Sttp model / core
Circe client wants v1.1.4 but Tapir wants 1.3.4 (of model)
5 replies
William Haw
@williamhaw
hi, I have some legacy clients that are using a Future-based backend, however I would like to use a ZIO backend instead.
so far I have this:
class FutureToZioBackend[S](delegate: SttpBackend[Task, S]) extends SttpBackend[Future, S] {

  override def send[T, R >: S with capabilities.Effect[Future]](request: Request[T, R]): Future[Response[T]] =
    Runtime.default.unsafeRunToFuture(delegate.send(request))

  override def close(): Future[Unit] = Runtime.default.unsafeRunToFuture(delegate.close())

  override def responseMonad: MonadError[Future] = ???
}
  1. Is there a better way than to use unsafeRunToFuture in the send() method?
  2. how do I convert delegate.responseMonad which is of type MonadError[Task] into MonadError[Future]?
5 replies
Fernando Mora
@fernandomora

Taking a look to https://sttp.softwaremill.com/en/latest/responses/body.html#streaming it says
"The “unsafe” variants return the stream directly to the user, and then it’s up to the user of the code to consume and close the stream, releasing any resources held by the HTTP connection."

How can I "close" the returned stream?

I have checked the code for the http4s backend and It seems ResponseAsStreamUnsafe is discarding the closing Stream function https://github.com/softwaremill/sttp/blob/8dcc4878b9f400fb45c10d80e5878f5a3d605830/core/src/main/scala/sttp/client3/internal/BodyFromResponseAs.scala#L54
so the signalBodyComplete effect seems never run https://github.com/softwaremill/sttp/blob/8dcc4878b9f400fb45c10d80e5878f5a3d605830/http4s-backend/src/main/scala/sttp/client3/http4s/Http4sBackend.scala#L60
and the Deferred responseBodyCompleteVar would be never got https://github.com/softwaremill/sttp/blob/8dcc4878b9f400fb45c10d80e5878f5a3d605830/http4s-backend/src/main/scala/sttp/client3/http4s/Http4sBackend.scala#L74

Could this be problematic? How can I assure the connection is being released?

2 replies
harrylaou
@harrylaou:matrix.org
[m]

Hi ,
I am migrating a service that was using akka-http to sttp-zio

By using AsyncHttpClientZioBackend.layer() I am getting a java.io.IOException: Too many open files
My current understanding is that this will create many "backends" and these reserve tcp connections.
Whats is the recommendation? should I just increase ulimit in the deployment or try to use a single backend and inject it to the layer , something like

private val sttpClient: AsyncHttpClient = asyncHttpClient()
  val sttpClientLayer: Layer[Throwable, SttpClient] = AsyncHttpClientZioBackend.layerUsingClient(sttpClient)

(I assume that this just creates a singleton backend in the service)

???

2 replies
mvillafuertem
@mvillafuertem
Hi there @adamw, I am trying to test a simple code using sttp and zio with repeat and I get this In Suite "SttpRequestWithRepeatPolicySpec$", test "Test 1" has taken more than 1 m to execute. If this is not expected, consider using TestAspect.timeout to timeout runaway tests for faster diagnostics.
9 replies

object SttpRequestWithRepeatPolicySpec extends DefaultRunnableSpec {

  // @see https://requestbin.com/r/ene80m1n53nb
  implicit val customConfig: Configuration                        = Configuration.default
  val backend: Task[SttpBackend[Task, Nothing, WebSocketHandler]] = AsyncHttpClientZioBackend()

  case class Response(success: Boolean)
  private val uri        = "https://ene80m1n53nb.x.pipedream.net/"
  private val requestGET = basicRequest.get(uri"$uri").response(asJson[Response])

  override def spec: Spec[TestEnvironment, TestFailure[Throwable], TestSuccess] =
    suite(getClass.getSimpleName)(
      testM(s"Test 1")(
        assertM(
          backend.flatMap { implicit backend =>
            for {
              fiber <- requestGET
                         .send()
                         .map(_.body)
                         .absolve
                         .repeat(
                           (Schedule.spaced(2.second) >>>
                             Schedule.recurWhile[Long](_ < 5))
                             .tapOutput[Console](n => putStr(n.toString + " ").exitCode) *>
                             Schedule
                               .collectAll[Response]
                               .tapInput[Console, Response](response => putStrLn(response.toString).exitCode)
                         )
                         .catchAll(a =>
                           zio.console.putStr(a.getMessage) >>>
                             RIO.effect(Chunk(Response(false)))
                         )
                         .fork
              _     <- TestClock.adjust(20.seconds)
              _     <- fiber.join.ensuring(backend.close().ignore)
            } yield ()
          }
        )(equalTo(()))
      )
    )
}
Here the full code https://raw.githubusercontent.com/mvillafuertem/scala/4533aac8dcace8ebc7ec28ef480c00df575fdae8/modules/script/SttpRequestWithRepeatPolicy.sc
Glen Marchesani
@fizzy33
Anyone know the status on cats effect 3 ? I can see it in master and unless the naming convention has changed I don't see any ce3 artifacts out there... thanks in advance
2 replies
catostrophe
@catostrophe
@adamw http4s team is planning to release 0.23.x as a "stable" long-term series compatible with CE3, leaving 1.0.0-Mxx for further work until 1.0.0 is released (which will take quite a long time I suppose).
I think it makes sense to downgrade http4s to 0.23.x when it's published in both sttp and tapir. what do you think?
2 replies
harrylaou
@harrylaou:matrix.org
[m]

Hi ,

I am using sttp-zio and when running some tests with stubbing I am getting

Fiber failed.
An unchecked error was produced.
java.lang.NullPointerException

I can recover with catchAllCause, I have the stack trace but cannot see why this is failing. Any ideas? Has anyone seen this before when stubbing?

harrylaou
@harrylaou:matrix.org
[m]

:point_up: Edit: Hi ,

I am using sttp-zio and when running some tests with stubbing I am getting

Fiber failed.
An unchecked error was produced.
java.lang.NullPointerException

I can recover with catchAllCause, I have the stack trace but cannot see why this is failing.
Using it with unsafeRunToFuture

Any ideas? Has anyone seen this before when stubbing?

10 replies
Glen Marchesani
@fizzy33
Is there a way to build a backend that can stream and do websockets ?
Glen Marchesani
@fizzy33
oh my issue is in scala js. Is there any way to do websockets in scala js seems a better way to ask that.
Glen Marchesani
@fizzy33
looks like the answer is no, I guess in scala js just use websocket api directly
17 replies
Glen Marchesani
@fizzy33
okay I am at a loss on this one
          request
            .response(asStream[F, fs2.Stream[F,ByteBuffer]])
            .readTimeout(Duration.Inf)
            .post(uri)
            .send()
1 reply
error message is
wrong number of type parameters for method asStream: [F[_], T, S](s: sttp.capabilities.Streams[S])(f: s.BinaryStream => F[T]): sttp.client3.ResponseAs[Either[String,T],sttp.capabilities.Effect[F] with S]
            .response(asStream[F, fs2.Stream[F,ByteBuffer]])
I suspect my issue is I ideally want to use the same sttp backend for streaming bytes (for server sent events) and for websockets. Not sure if I can do that
the sttp backend that is present is not a specific container it is
  val instance: F[sttp.client3.SttpBackend[F, WebSockets]]
oof that error message is wrong the error messsage is
/Users/glen/code/accur8/remote-api/client/shared/src/main/scala/a8/remoteapi/client/DefaultSttpBackend.scala:31:16
Cannot prove that sttp.capabilities.WebSockets with sttp.capabilities.Effect[[_]F[_]] <:< Nothing.
          .send(backend)
Glen Marchesani
@fizzy33
okay I worked around that by having two backends one for WebSockets and one for Fs2Streams
2 replies
abstract class DefaultSttpBackend[F[_] : Async] {

  val instance: F[sttp.client3.SttpBackend[F, WebSockets]] = ...
  val instance2: F[sttp.client3.SttpBackend[F, Fs2Streams[F]]] = ...

  ...

}
is there a way to get both capabilities with a single backend instance ?
Glen Marchesani
@fizzy33
sorry for all the noise... I took on a large refactoring of a scala js and jvm app using monix to using cats and fs2. I see most of my issues stem now from fs2 not being in scala js (where monix is in both).
I have restarted and doing just jvm first.
Glen Marchesani
@fizzy33
I have my core question which is this, do I need three separate backends to do streams, websockets and regular or can I do all those with the same one ?
  val backendN: SttpBackend[F,Nothing] = ???
  val backendStream: SttpBackend[F,Fs2Streams[F]] = ???
  val backendWs: SttpBackend[F,WebSockets] = ???
In this use case I am AsyncHttpClientBackend so I can minimally re-use asyncHttpClient: AsyncHttpClient
across all three
Glen Marchesani
@fizzy33
oh snap @adam sorry I just saw your reply now to this.
i.e. -- just use with: SttpBackend[F, WebSockets with Fs2Streams[F]]
Glen Marchesani
@fizzy33
that is all working now thanks @adamw
Glen Marchesani
@fizzy33

one last one. I worked around this with a strategically placed .asInstanceOf[] but have created an isolated reproduction to see if/what I am missing


import cats.effect.IO
import sttp.client3.impl.cats.FetchCatsBackend
import sttp.client3._

object CannotGetThereFromHere {

  val backend = FetchCatsBackend[IO]()

  val responseIO = basicRequest.send(backend)

}

That gives a compile error of.

Cannot prove that sttp.client3.Empty[Unit] =:= sttp.client3.Identity[Unit].
  val responseIO = basicRequest.send(backend)
3 replies
Shouldn't that "just work (tm)", am I missing an import or something
William Haw
@williamhaw

ok figured it out w.r.t delegating from Future backend to ZIO backend:

class FutureToZioBackend[S](val delegate: SttpBackend[Task, S])(implicit ec: ExecutionContext)
    extends SttpBackend[Future, S] {

  override def send[T, R >: S with Effect[Future]](request: Request[T, R]): Future[Response[T]] =
    Runtime.default.unsafeRunToFuture(
      delegate.send(
        MapEffect.apply[Future, Task, Identity, T, S](
          request: Request[T, S with Effect[Future]],
          futureToTask,
          taskToFuture,
          this.responseMonad,
          delegate.responseMonad
        )
      )
    )

  private val futureToTask = new FunctionK[Future, Task] {
    override def apply[A](fa: Future[A]): Task[A] = ZIO.fromFuture(implicit ec => fa)
  }

  private val taskToFuture = new FunctionK[Task, Future] {
    override def apply[A](ta: Task[A]): Future[A] = Runtime.default.unsafeRunToFuture(ta)
  }

  override def close(): Future[Unit] = Runtime.default.unsafeRunToFuture(delegate.close())

  override val responseMonad: MonadError[Future] = new FutureMonad()
}

thanks @adamw

Daniel Vigovszky
@vigoo
hi! is there any special reason why slf4j-backend is not released for Scala 3? - if not I'd open an issue / PR
2 replies
vonchav
@voonchav_gitlab
@adamw I think Circe 0.14.1 can be used by Scala 2.x too. Any thoughts?
1 reply
Gerard Downes
@GDownes
Hi all, quite new to this.
I've been using the sttp client with different backends. I'd like to be able to modify/intercept the request and response of a request regardless of backend.
I have tried wrapping the SttpBackend send method, but the F type has made it difficult accessing the response. Would anyone have any ideas for implementing this type of interceptor, I'd like to be able to modify outgoing and receive incoming headers?
4 replies