Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Dec 03 10:34
    micossow synchronize #1197
  • Dec 03 10:34

    micossow on fix-curl-logging-secret-headers

    fix scala3 build (compare)

  • Dec 03 10:07
    mergify[bot] labeled #1200
  • Dec 03 10:04
    scala-steward opened #1200
  • Dec 03 09:34
    micossow synchronize #1197
  • Dec 03 09:34

    micossow on fix-curl-logging-secret-headers

    fix 2.11 build (compare)

  • Dec 03 09:25
    micossow synchronize #1197
  • Dec 03 09:25

    micossow on fix-curl-logging-secret-headers

    refactor (compare)

  • Dec 03 09:21
    micossow synchronize #1197
  • Dec 03 09:20

    micossow on fix-curl-logging-secret-headers

    custom sensitive headers (compare)

  • Dec 03 09:11
    micossow synchronize #1197
  • Dec 03 09:11

    micossow on fix-curl-logging-secret-headers

    custom sensitive headers (compare)

  • Dec 02 22:31

    mergify[bot] on master

    Update armeria to 1.13.4 Merge pull request #1199 from s… (compare)

  • Dec 02 22:31
    mergify[bot] closed #1199
  • Dec 02 22:05
    mergify[bot] labeled #1199
  • Dec 02 22:04
    scala-steward closed #1189
  • Dec 02 22:04
    scala-steward commented #1189
  • Dec 02 22:04
    scala-steward opened #1199
  • Dec 02 20:09
    Pask423 synchronize #1154
  • Dec 02 20:09

    Pask423 on auto-decompression-turning

    Update mdoc, sbt-mdoc to 2.2.24 Merge pull request #1151 from s… Update dependencies and 73 more (compare)

Anirudh Vyas
@AnirudhVyas
ok I figured it out @adamw I was thinking maybe a good idea to also add an example that receives more than one message maybe - I know for some folks flatMapLoop on task maybe a given but still ...
uri.flatMap[Response[Either[String, Vector[String]]]] { uri =>
      AsyncHttpClientMonixBackend()
        .flatMap { backend =>
          basicRequest
            .get(Uri(uri))
            .body(msg)
            .response(
              asWebSocket((f: WebSocket[Task]) =>
                f.sendText(msg) *> f.receiveText().flatMapLoop(Vector.empty[String]) { (each, acc, continue) =>
                  val nAcc = acc :+ each
                  println(nAcc)
                  if (terminateOn(nAcc)) {
                    Task.pure(nAcc)
                  } else continue(nAcc)
                }
              )
            )
            .send(backend)
        }
    }
this works - but unsure if this is elegant - maybe i can let messages arriving be an observable i will see about that
Anirudh Vyas
@AnirudhVyas
still do not know how to repeatedly call receive text to get many messages btw ... maybe someone could help there.
4 replies
jaredmdobson
@jaredmdobson
I migrated to sttp 3 and my upload performance has degraded, my guess is i'm using the sttp client incorrectly and was wondering if anyone sees anything glaring in my method. :grinning:
  def uploadFile[T](
    url: String,
    file: File,
    headers: Map[String, String],
    responseMapper: String => T
  ): Task[T] =
    for {
      request <- Task.effect(
        basicRequest
          .body(file)
          .readTimeout(2.minutes)
          .put(uri"$url")
          .headers(headers)
      )

      result <- send(request)
        .provideLayer(sttpBackendLayer)
        .flatMap { response =>
          if (response.isSuccess)
            response.body.fold(
              error => Task.fail(new RuntimeException(error)),
              stringRes => Task.effect(responseMapper(stringRes))
            )
          else Task.fail(new RuntimeException(response.statusText))
        }
    } yield result
8 replies
Anirudh Vyas
@AnirudhVyas
hi does sttp support connection pooling?
Anirudh Vyas
@AnirudhVyas
i understand that its not one of its goals but just want to know if it can OOB
4 replies
Imran Javaid
@imranjavaid
Hi, I am in the process of writing a library that uses sttp and am currently expecting an asynchronous backend to be passed to it. Is there a straightforward way for me to aslo support being passed a synchronous backend? In other words, how can I add a layer for the synchonous support by resuing the code that will have for the asynchronous backend?
1 reply
Tim Pigden
@TimPigden
Hi, I have computation service I'm calling that supports multiple simultaneous requests - and I have multiple computations. Can I call the same backend from multiple zio parallel fibres?
1 reply
Vitaly Filinov
@vitaly.filinov_gitlab

Hi guys! I have a service with WebSocket stream created as

basicRequest
      .get(uri)
      .response(asWebSocketStream(Fs2Streams[F])(pipe))
      .send(backend)

where backend is SttpBackend[F, Fs2Streams[F] with capabilities.WebSockets]
Any chance to create a stub for testing stream?
Thanks in advance

1 reply
antonkulaga
@antonkulaga:matrix.org
[m]

I am having a huge pain with documentation.
You often have:

import sttp.client3.httpclient.zio.HttpClientZioBackend

HttpClientZioBackend().flatMap { backend => ??? }

However I have no idea what I should put instead of ??? in the flatmap.
Also, I do not understand how can I retrieve the zio runtime for the sttp client so I can just do runtime.unsafeRun(myRequest)

3 replies
antonkulaga
@antonkulaga:matrix.org
[m]
From what I understand I cannot run zio.Runtime.default because the type for the request assumes httpclient in the environment
at least compiler did not allow me to call unsafeRun from default runtime on requests
So, did I get it right that I have to flatMap backend whenever I send any requests?
it was not clearly explained in docs (or at least I did not get it)
ex0ns
@ex0ns

I was wondering what was the best way to handle error with STTP and the ZIO backend.
The send method returns Throwable with (as explained in the doc) known error converted to SttpClientException.
Addind to that the circe decoding exception in a ResponseException, this become really tedious to deal with.
Do you have any recommendation ?
Let's assume the following piece of code:

        override def send[T](
            request: Request[MatrixResponse[T], Any]
        ): ZIO[Any, ResponseException[ResponseError, io.circe.Error], T] = {
          for {
            result <- sendRequest(request).provide(client) // Throwable (low level error, network error, ...)
            json <- ZIO.fromEither(result.body) // ResponseException (API error OR circe error)
          } yield json
        }

It does not compile because (sendRequest an alias to STTP send) sendRequest can fail with Throwable.
I would like to avoid forwarding this Throwable up to the caller, because it's confusing and does not make the type signature very useful.
Another solution would be to "flatten" those error (network error, api error and serialization errors) by creating an ADT to wrap them, is it a good solution ?

5 replies
reibitto
@reibitto

I'm writing a VCR lib with sttp which means I have to wrap any HTTP backend in a general way and have it still work. One thing I'm struggling with is converting an arbitrary response T to a String. I have this dirty hack currently that I want to get rid of:
https://github.com/reibitto/be-kind-rewind/blob/a2385dbd96904b39b554770518955f076efcf44c/client-sttp/src/main/scala/bekindrewind/sttpclient/VcrBackend.scala#L103-L108

Is there a typeclass I missed or some other mechanism to handle this? Any hints where to look would be appreciated 🙂

7 replies
Jakub Kozłowski
@kubukoz:matrix.org
[m]
hello there, is there a way to match the body text in a fake backend? In particular, I want to ensure a particular field is present in the body when decoded as a JSON object, but if I'm able to get the body as a string I'll be fine from that point
1 reply
@reibitto: you probably don't want to go to string but to a stream of bytes
2 replies
since users don't necessarily have to send utf8 strings
kinda weird that it's a generic T with no constraints but ideally you'd want something with an encoder/decoder typeclass constraint - these/one of these are probably included in the request's description of the response?
Brandon G
@gannicottb
Is Sttp supposed to wrap timeout exceptions arising from readTimeouts? I'm noticing that connectionTimeouts are properly wrapped, but readTimeouts throw unwrapped.
1 reply
Anirudh Vyas
@AnirudhVyas

hi there, For payload responses that are very large, I am running into :

12:01:32.294 [ERROR] o.a.n.h.WebSocketHandler - null
java.lang.NullPointerException: null
    at org.asynchttpclient.netty.channel.Channels.getAttribute(Channels.java:31)
    at org.asynchttpclient.netty.request.NettyRequestSender.abort(NettyRequestSender.java:467)
    at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.exceptionCaught(AsyncHttpClientHandler.java:201)
    at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302)
    at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281)
    at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273)
    at io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:143)

unsure what needs to be done there - I am using AsyncMonix handler for Websockets.

5 replies
I may have response frames/messages in Megs at times - I cannot receive streams etc. since this is more of a black box testing
 implicit val scheduler: Scheduler = Scheduler.global
      AsyncHttpClientMonixBackend()
        .flatMap { backend =>
          basicRequest
            .get(Uri(uri))
            .body(msg)
            .response(
              asWebSocket { (f: WebSocket[Task]) =>
                val observable = Observable
                  .repeatEvalF(f.receiveText()) // repeatedly call receive text to grab messages.
                  .dump("Incoming Msg")         // dumps message to println (can pass logger here)
                  .takeWhileInclusive { e =>    // take all values until condition is met + also include value where
                    // where condition is not met for the first time (for us this would be till completion)
                    println(s"is condition met for $e ${terminateOn(e)}")
                    !terminateOn(e) // negative to accumulate all messages.
                  }
                f.sendText(msg) *> observable.toListL // toListL transforms to a Task.pure(List[String])
              }
            )
            .send(backend)
thats my sample code
Octav Zaharia
@octavz
hi, where is the sttp version set ?
5 replies
Akshay Sachdeva
@asachdeva

Hi...so stack is
val http4s = "0.21.22"
val tapir = "0.18.0-M5"

val request = SttpClientInterpreter.toRequestThrowDecoderFailures(makeCall, config.server.externalApi.some)

        val response = request
          .apply(
            (
              param1.get.toString,
              param2.get.toString,
              config.server.apiKey.toString(),
              extraParam
            )
          )
          .send(backend)

where makeCall is an endpoint like

val makeCall = endpoint.get
        .in("")
        .in(query[String]("param1"))
        .in(query[String]("param2"))
        .in(query[String]("apiKey"))
        .in(query[String]("extraParam"))
        .out(jsonBody[Report])
        .errorOut(statusCode)

Everything looks good and all of the query Params get applied but I see that the request looks like https://api.blah.com/?param1= and so on...essentially with a slash after the external api uri and before the question mark for the query parameters. I have confirmed that the externalAPI does not have a trailing slash. Am I missing something obvious here? Thanks in advance for any help

2 replies
heksesang
@heksenlied:matrix.org
[m]
Is there a reason why async-http-client-backend-zio isn't released for 3.0.0-RC1 and/or -RC2?
1 reply
heksesang
@heksenlied:matrix.org
[m]
1.0.5 was released for 3.0.0-RC1, and I believe 1.0.6 is released for 3.0.0-RC2.
dev.zio:zio_3.0.0-RC2:1.0.6 and dev.zio:zio_3.0.0-RC1:1.0.5
Adam Warski
@adamw
ah, well it wasn't at the moment of the release :) I'll try to re-enable before the next one
dan-ilin
@dan-ilin

hi. i'm trying to write a function to execute requests using an SttpBackend[Future, Any], but i keep running into this error on compilation:

Cannot prove that Any with sttp.capabilities.Effect[scala.concurrent.Future] <:< Nothing with Any.
[error]         .send(backend)
[error]              ^

this is with version 3.2.3 of the library

2 replies
almoehi
@almoehi
Hey! I've moved an existing project to a new machine with java 11 & latest version of intelliJ - now it won't find the basicRequest anymore in my IDE. Although I do have import sttp.client3._ - are there any special/hidden settings I need to make in intelliJ ??? I'm pretty puzzled ...
2 replies
heksesang
@heksenlied:matrix.org
[m]
@adamw: sttp-shared 1.2.2 is missing for 3.0.0-RC2 – but Scala 3 was reenabled in the build.sbt before that release. Did something go wrong? Will it be out soon?
1 reply
The zio one, I mean.
I need com.softwaremill.sttp.shared:zio_3.0.0-RC2:1.2.2 in an attempt to see if I can compile the tapir zio integrations for Scala 3 (with 2.13.5 core), it's the only dependency I am missing.
Joshua Portway
@jportway
Hi - perhaps I'm misunderstanding something here, but is there an effectful version of asWebSocketStreamAlways ? The "pipe" required by this seems to have to be a simple function from an input stream to an output stream, but with ZIO (for instance, haven't tried other libraries) it is often not possible to create a stream without running an effect.
8 replies
several stream operations (for instance, partitioning a stream) don't produce a stream directly, but return an effect that produces the stream. But the "pipe" funtion required by asWebSocketStream needs to be pure - so there is no way to run the effect
Joshua Portway
@jportway
Unless I'm misunderstanding something, this seems to severely limit the possibilities of what can be done with streams - basically it looks like I can only do simple mapping and transformations and not anything else that might itself require an IO type effect. Is this right?
heksesang
@heksenlied:matrix.org
[m]
Odd, it's there now, it didn't show earlier when I looked up at search.maven.org.
Sergei Kharchikov
@callmestech
Hey! Is there a way to create AsyncHttpClientZioBackend.layer() but with enabled logging?
6 replies
Kisaragi
@KisaragiEffective
How can I convert non-literal String into Uri ? Just uri"""$someNonLiteral""" ?
2 replies
Paweł Kaczor
@pawelkaczor

Hello! I'm processing UTF-8 with BOM encoded json at the client side in the following way:

  basicRequest.response(asString.mapWithMetadata(myRespone).showAsJson)

  def myResponse = {
    ResponseAs.deserializeRightCatchingExceptions(deserializeJson[MyResponse].compose(_.stripPrefix("\uFEFF")))
  }

Is there a simpler way to do it ? (to remove initial character from the json string) ?

1 reply
Thijs Broersen
@ThijsBroersen
I am trying to generate a client from an OpenAPI-spec and it looks like it is fixed to the old sttp.client and not sttp.client3, while readthedocs clearly have client3 in their examples. I cannot get it to generate client3 code, is this correct?
2 replies
barthorre
@barthorre

Hi,

am I correct in assuming that SttpbackendStub does not support websocket streams?

9 replies
basing this on case ResponseAsWebSocketStream(_, _) => None in tryAdjustResponseBody
Zdenek Farana
@byF
Hey, does anyone have multipart/mixed response deserializer implementation by any chance? 😅
2 replies
catostrophe
@catostrophe
@adamw tapir-0.18.0-M8 wasn't published to sonatype
1 reply
Matt Smith
@00-matt
Hi, how can I catch network failures and make them a future instead? I'm using the Java 11 HttpClient backend. I've tried this but the exception doesn't get caught and it ends up stopping the whole application:
    try {
      basicRequest
        .get(uri)
        .response(asJson[StatsResponse])
        .send(backend)
    } catch {
      case e: SttpClientException.ConnectException => Future.failed(e)
    }
3 replies
Felix Dietze
@fdietze
Hi, I'm trying to use FetchCatsBackend[IO] with ScalaJS, but I'm getting this error message, what am I missing?
import sttp.client3.impl.cats.FetchCatsBackend
import cats.effect._
import sttp.client3._

  implicit val ec: scala.concurrent.ExecutionContext =
    scala.concurrent.ExecutionContext.global
  implicit val contextShift: ContextShift[IO]        = IO.contextShift(ec)

  // https://sttp.softwaremill.com/en/latest/backends/javascript/fetch.html?highlight=fetch#node-js
  val g         = scalajs.js.Dynamic.global.globalThis
  val nodeFetch = g.require("node-fetch")
  g.fetch = nodeFetch
  g.Headers = nodeFetch.Headers
  g.Request = nodeFetch.Request

  val sttpBackend = FetchCatsBackend[IO]()
[error]      could not find implicit value for evidence parameter of type cats.effect.kernel.Async[cats.effect.IO] (Could not find an instance of Monad for cats.effect.IO)
[error]      Error occurred in an application involving default arguments.
[error]      L59:   val sttpBackend = FetchCatsBackend[IO]()
[error]                                                   ^
Felix Dietze
@fdietze
solved it, I'm working with Cats 2, where the latest sttp release defaults to Cats 3