Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 27 15:22

    mergify[bot] on master

    Update upickle to 1.5.0 Merge pull request #1281 from s… (compare)

  • Jan 27 15:22
    mergify[bot] closed #1281
  • Jan 27 15:16

    mergify[bot] on master

    Update armeria to 1.14.0 Merge pull request #1282 from s… (compare)

  • Jan 27 15:16
    mergify[bot] closed #1282
  • Jan 27 14:45
    mergify[bot] labeled #1282
  • Jan 27 14:45
    mergify[bot] labeled #1281
  • Jan 27 14:45
    scala-steward opened #1282
  • Jan 27 14:45
    scala-steward opened #1281
  • Jan 27 13:39
    Kobenko edited #1279
  • Jan 27 13:39
    Kobenko edited #1279
  • Jan 27 11:13

    mergify[bot] on master

    Update http4s-blaze-client, htt… Merge pull request #1278 from s… (compare)

  • Jan 27 11:13
    mergify[bot] closed #1278
  • Jan 27 11:10

    Pask423 on master

    Update scala-compiler, scala-li… Fix compilations issues for Asy… Merge pull request #1255 from s… (compare)

  • Jan 27 11:10
    Pask423 closed #1255
  • Jan 27 10:37
    adamw synchronize #1255
  • Jan 27 10:37
    mergify[bot] commented #1255
  • Jan 27 10:36
    adamw synchronize #1278
  • Jan 27 10:36
    mergify[bot] commented #1278
  • Jan 27 10:36
    Pask423 commented #1255
  • Jan 27 10:35
    Pask423 commented #1278
Jonas Adler
@jadlr
Is there a way to request String urls until this is fixed?
Jonas Adler
@jadlr
@adamw thanks a ton for the fix! Great support!
2 replies
Anirudh Vyas
@AnirudhVyas
Hi there - I am looking for some suggestions / RFCs etc to design a websocket API, I cannot find any concrete material, in REST world this is fairly well established and many good documents to follow, can someone suggest something for websocket API design? like should the focus be on resource entities and operations constraints as in rest? should it be entity_operation as a single message and that template repeated
2 replies
I know this has more to do with sttp, but I thought with so many knowledgeable people here might chance it :)
Sam Desborough
@desbo
Hi all. I've noticed that when using an http4s backend with gzip middleware enabled, requesting gzipped resources will fail with Not in gzip format. This is presumably because STTP is attempting to decompress what has already been decompressed by the http4s middleware. This suggests that backends shouldn't do their own decompression, is that true?
2 replies
Harmeet Singh(Taara)
@harmeetsingh0013
Hey All, I am using STTP client with ZIO. Currently using send method from sttp.client3.asynchttpclient.zio._ package to make the request. But when I am trying to write unit test cases for that, I found https://sttp.softwaremill.com/en/latest/testing.html send(backEnd) method with passing backend. I check, there is stubbing in sttp.client3.asynchttpclient package, but not sure, is it used for unit test our not? Another question, Do we have predefined ZIO backend or we need to define our own? for passing into the send(...) method.
5 replies
Vasyl Zhurba
@vasyl-zhurba

Hi all. I couldn't find in sttp documentation if it is possible to describe a google batch request with sttp.
https://developers.google.com/gmail/api/guides/batch#batch-example-request

POST /batch/farm/v1 HTTP/1.1
Authorization: Bearer your_auth_token
Host: www.googleapis.com
Content-Type: multipart/mixed; boundary=batch_foobarbaz
Content-Length: total_content_length

--batch_foobarbaz
Content-Type: application/http
Content-ID: <item1:12930812@barnyard.example.com>

GET /farm/v1/animals/pony

--batch_foobarbaz
Content-Type: application/http
Content-ID: <item2:12930812@barnyard.example.com>

PUT /farm/v1/animals/sheep
Content-Type: application/json
Content-Length: part_content_length
If-Match: "etag/sheep"

{
  "animalName": "sheep",
  "animalAge": "5"
  "peltColor": "green",
}

--batch_foobarbaz
Content-Type: application/http
Content-ID: <item3:12930812@barnyard.example.com>

GET /farm/v1/animals
If-None-Match: "etag/animals"

--batch_foobarbaz--

Is there a way to call google batch api with sttp client?

5 replies
From the documentation, I read that it was possible to use multipartBody, but I couldn't find a way how to set an url for each part
Harmeet Singh(Taara)
@harmeetsingh0013
I am using STTP with Cats and adding Dependencies like
```
"com.softwaremill.sttp.client3" %% "circe" % "3.1.1"
"com.softwaremill.sttp.client3" %% "async-http-client-backend-cats" % "3.1.1"
Harmeet Singh(Taara)
@harmeetsingh0013
Hey all, I am creating a URL via Uri class, my test cases run, but when I used `uri"some string", my test cases failed. Even during print, I can see the same values.
1 reply
vito-c
@vito-c
is there a way to have all of my execute traffic routed through a socks5 proxy?
also I'm trying to figure out what would cause this error:
java.net.UnknownHostException: my.valid.host.com : nodename nor  servname provided, or not known
vito-c
@vito-c
I'm trying to hit the server over a proxy if that helps
vito-c
@vito-c
AsyncHttpClient/async-http-client#1682 this looks related to my issue
1 reply
Brandon G
@gannicottb
Hi, I'm using sttp3's AsyncHttpClientZioBackend. If I use the .layer methods to provide SttpClient to my effects, how am I supposed to customize the backend stub for tests? The .stubLayer method makes it impossible to add any .whenRequest-like methods to a stub (since it creates its own stub internally and maintains it via a Ref). What am I missing here?
1 reply
Brandon G
@gannicottb
IntelliJ 2020.3.1 can't figure out the return type of the whenRequest methods from sttp.client3.asynchttpclient.zio.stubbing._, and I can't even coerce with .asInstanceOf. Which means I can't chain them or even use them. Any ideas?
Lukasz Trojanowski
@lstrojanowski_twitter
Does sttp implement a way to fetch images? I couldn't find any examples in the documentation
1 reply
Anirudh Vyas
@AnirudhVyas
Hey there, I wish to write some basic sttp client websocket test suite, the idea is to connect -> send a request --> receive n messages back -> filter and as soon as I get response I expect mark it successful. I am looking for a complete example but cannot find that does this sort of thing - any suggestions on where to look?
willing to explore monix, fs2 or akka - doesnt matter, what matters is simplicity in API so adoption is widespread within our team
Anirudh Vyas
@AnirudhVyas

ok I did a bit of investigation and here's where I am at the moment:

No WebSocketUpgradeHandler but scheme is ws
java.lang.IllegalArgumentException: No WebSocketUpgradeHandler but scheme is ws
    at org.asynchttpclient.netty.request.NettyRequestSender.validateWebSocketRequest(NettyRequestSender.java:571)
    at org.asynchttpclient.netty.request.NettyRequestSender.sendRequest(NettyRequestSender.java:95)
    at org.asynchttpclient.DefaultAsyncHttpClient.execute(DefaultAsyncHttpClient.java:259)
    at org.asynchttpclient.DefaultAsyncHttpClient.executeRequest(DefaultAsyncHttpClient.java:228)
    at org.asynchttpclient.BoundRequestBuilder.execute(BoundRequestBuilder.java:35)
    at sttp.client3.asynchttpclient.AsyncHttpClientBackend.$anonfun$sendRegular$1(AsyncHttpClientBackend.scala:59)
    at sttp.client3.impl.monix.TaskMonadAsyncError$.$anonfun$async$1(TaskMonadAsyncError.scala:18)
    at cancelable @ sttp.client3.impl.monix.TaskMonadAsyncError$.async(TaskMonadAsyncError.scala:17)

I get that ^^ my code is:

uri.flatMap[Vector[String]] { uri =>
      AsyncHttpClientMonixBackend()
        .flatMap { backend =>
          basicRequest
            .get(Uri(uri))
            .body(msg)
            .send(backend)
            .flatMapLoop(Vector.empty[String]) { (each, acc: Vector[String], continue) =>
              Task
                .fromEither[String, String] { e: String => new Exception(e) }(each.body)
                .flatMap { response =>
                  val newList = acc :+ response
                  if (terminateOn(newList))
                    continue(newList)
                  else Task.pure(newList)
                }
            }
            .guarantee(backend.close())
        }
    }
pretty simple with a terminate on condition, unsure what am I missing
Anirudh Vyas
@AnirudhVyas
ok I figured it out @adamw I was thinking maybe a good idea to also add an example that receives more than one message maybe - I know for some folks flatMapLoop on task maybe a given but still ...
uri.flatMap[Response[Either[String, Vector[String]]]] { uri =>
      AsyncHttpClientMonixBackend()
        .flatMap { backend =>
          basicRequest
            .get(Uri(uri))
            .body(msg)
            .response(
              asWebSocket((f: WebSocket[Task]) =>
                f.sendText(msg) *> f.receiveText().flatMapLoop(Vector.empty[String]) { (each, acc, continue) =>
                  val nAcc = acc :+ each
                  println(nAcc)
                  if (terminateOn(nAcc)) {
                    Task.pure(nAcc)
                  } else continue(nAcc)
                }
              )
            )
            .send(backend)
        }
    }
this works - but unsure if this is elegant - maybe i can let messages arriving be an observable i will see about that
Anirudh Vyas
@AnirudhVyas
still do not know how to repeatedly call receive text to get many messages btw ... maybe someone could help there.
4 replies
jaredmdobson
@jaredmdobson
I migrated to sttp 3 and my upload performance has degraded, my guess is i'm using the sttp client incorrectly and was wondering if anyone sees anything glaring in my method. :grinning:
  def uploadFile[T](
    url: String,
    file: File,
    headers: Map[String, String],
    responseMapper: String => T
  ): Task[T] =
    for {
      request <- Task.effect(
        basicRequest
          .body(file)
          .readTimeout(2.minutes)
          .put(uri"$url")
          .headers(headers)
      )

      result <- send(request)
        .provideLayer(sttpBackendLayer)
        .flatMap { response =>
          if (response.isSuccess)
            response.body.fold(
              error => Task.fail(new RuntimeException(error)),
              stringRes => Task.effect(responseMapper(stringRes))
            )
          else Task.fail(new RuntimeException(response.statusText))
        }
    } yield result
8 replies
Anirudh Vyas
@AnirudhVyas
hi does sttp support connection pooling?
Anirudh Vyas
@AnirudhVyas
i understand that its not one of its goals but just want to know if it can OOB
4 replies
Imran Javaid
@imranjavaid
Hi, I am in the process of writing a library that uses sttp and am currently expecting an asynchronous backend to be passed to it. Is there a straightforward way for me to aslo support being passed a synchronous backend? In other words, how can I add a layer for the synchonous support by resuing the code that will have for the asynchronous backend?
1 reply
Tim Pigden
@TimPigden
Hi, I have computation service I'm calling that supports multiple simultaneous requests - and I have multiple computations. Can I call the same backend from multiple zio parallel fibres?
1 reply
Vitaly Filinov
@vitaly.filinov_gitlab

Hi guys! I have a service with WebSocket stream created as

basicRequest
      .get(uri)
      .response(asWebSocketStream(Fs2Streams[F])(pipe))
      .send(backend)

where backend is SttpBackend[F, Fs2Streams[F] with capabilities.WebSockets]
Any chance to create a stub for testing stream?
Thanks in advance

1 reply
antonkulaga
@antonkulaga:matrix.org
[m]

I am having a huge pain with documentation.
You often have:

import sttp.client3.httpclient.zio.HttpClientZioBackend

HttpClientZioBackend().flatMap { backend => ??? }

However I have no idea what I should put instead of ??? in the flatmap.
Also, I do not understand how can I retrieve the zio runtime for the sttp client so I can just do runtime.unsafeRun(myRequest)

3 replies
antonkulaga
@antonkulaga:matrix.org
[m]
From what I understand I cannot run zio.Runtime.default because the type for the request assumes httpclient in the environment
at least compiler did not allow me to call unsafeRun from default runtime on requests
So, did I get it right that I have to flatMap backend whenever I send any requests?
it was not clearly explained in docs (or at least I did not get it)
ex0ns
@ex0ns

I was wondering what was the best way to handle error with STTP and the ZIO backend.
The send method returns Throwable with (as explained in the doc) known error converted to SttpClientException.
Addind to that the circe decoding exception in a ResponseException, this become really tedious to deal with.
Do you have any recommendation ?
Let's assume the following piece of code:

        override def send[T](
            request: Request[MatrixResponse[T], Any]
        ): ZIO[Any, ResponseException[ResponseError, io.circe.Error], T] = {
          for {
            result <- sendRequest(request).provide(client) // Throwable (low level error, network error, ...)
            json <- ZIO.fromEither(result.body) // ResponseException (API error OR circe error)
          } yield json
        }

It does not compile because (sendRequest an alias to STTP send) sendRequest can fail with Throwable.
I would like to avoid forwarding this Throwable up to the caller, because it's confusing and does not make the type signature very useful.
Another solution would be to "flatten" those error (network error, api error and serialization errors) by creating an ADT to wrap them, is it a good solution ?

5 replies
reibitto
@reibitto

I'm writing a VCR lib with sttp which means I have to wrap any HTTP backend in a general way and have it still work. One thing I'm struggling with is converting an arbitrary response T to a String. I have this dirty hack currently that I want to get rid of:
https://github.com/reibitto/be-kind-rewind/blob/a2385dbd96904b39b554770518955f076efcf44c/client-sttp/src/main/scala/bekindrewind/sttpclient/VcrBackend.scala#L103-L108

Is there a typeclass I missed or some other mechanism to handle this? Any hints where to look would be appreciated 🙂

7 replies
Jakub Kozłowski
@kubukoz:matrix.org
[m]
hello there, is there a way to match the body text in a fake backend? In particular, I want to ensure a particular field is present in the body when decoded as a JSON object, but if I'm able to get the body as a string I'll be fine from that point
1 reply
@reibitto: you probably don't want to go to string but to a stream of bytes
2 replies
since users don't necessarily have to send utf8 strings
kinda weird that it's a generic T with no constraints but ideally you'd want something with an encoder/decoder typeclass constraint - these/one of these are probably included in the request's description of the response?
Brandon G
@gannicottb
Is Sttp supposed to wrap timeout exceptions arising from readTimeouts? I'm noticing that connectionTimeouts are properly wrapped, but readTimeouts throw unwrapped.
1 reply
Anirudh Vyas
@AnirudhVyas

hi there, For payload responses that are very large, I am running into :

12:01:32.294 [ERROR] o.a.n.h.WebSocketHandler - null
java.lang.NullPointerException: null
    at org.asynchttpclient.netty.channel.Channels.getAttribute(Channels.java:31)
    at org.asynchttpclient.netty.request.NettyRequestSender.abort(NettyRequestSender.java:467)
    at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.exceptionCaught(AsyncHttpClientHandler.java:201)
    at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302)
    at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281)
    at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273)
    at io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:143)

unsure what needs to be done there - I am using AsyncMonix handler for Websockets.

5 replies
I may have response frames/messages in Megs at times - I cannot receive streams etc. since this is more of a black box testing
 implicit val scheduler: Scheduler = Scheduler.global
      AsyncHttpClientMonixBackend()
        .flatMap { backend =>
          basicRequest
            .get(Uri(uri))
            .body(msg)
            .response(
              asWebSocket { (f: WebSocket[Task]) =>
                val observable = Observable
                  .repeatEvalF(f.receiveText()) // repeatedly call receive text to grab messages.
                  .dump("Incoming Msg")         // dumps message to println (can pass logger here)
                  .takeWhileInclusive { e =>    // take all values until condition is met + also include value where
                    // where condition is not met for the first time (for us this would be till completion)
                    println(s"is condition met for $e ${terminateOn(e)}")
                    !terminateOn(e) // negative to accumulate all messages.
                  }
                f.sendText(msg) *> observable.toListL // toListL transforms to a Task.pure(List[String])
              }
            )
            .send(backend)
thats my sample code
Octav Zaharia
@octavz
hi, where is the sttp version set ?
5 replies
Akshay Sachdeva
@asachdeva

Hi...so stack is
val http4s = "0.21.22"
val tapir = "0.18.0-M5"

val request = SttpClientInterpreter.toRequestThrowDecoderFailures(makeCall, config.server.externalApi.some)

        val response = request
          .apply(
            (
              param1.get.toString,
              param2.get.toString,
              config.server.apiKey.toString(),
              extraParam
            )
          )
          .send(backend)

where makeCall is an endpoint like

val makeCall = endpoint.get
        .in("")
        .in(query[String]("param1"))
        .in(query[String]("param2"))
        .in(query[String]("apiKey"))
        .in(query[String]("extraParam"))
        .out(jsonBody[Report])
        .errorOut(statusCode)

Everything looks good and all of the query Params get applied but I see that the request looks like https://api.blah.com/?param1= and so on...essentially with a slash after the external api uri and before the question mark for the query parameters. I have confirmed that the externalAPI does not have a trailing slash. Am I missing something obvious here? Thanks in advance for any help

2 replies
heksesang
@heksenlied:matrix.org
[m]
Is there a reason why async-http-client-backend-zio isn't released for 3.0.0-RC1 and/or -RC2?
1 reply
heksesang
@heksenlied:matrix.org
[m]
1.0.5 was released for 3.0.0-RC1, and I believe 1.0.6 is released for 3.0.0-RC2.
dev.zio:zio_3.0.0-RC2:1.0.6 and dev.zio:zio_3.0.0-RC1:1.0.5