pepegar/hammock

Anthony Cerruti

@srnb_gitlab

Trying to use .getBytes on said strings results in ef bf bd for UTF-8, unicode's "unknown character" filler, and for ISO and ASCII, 3f: ascii's ?

Anthony Cerruti

@srnb_gitlab

I ended up writing my own interpreter; copied the existing apache one and changed a few lines

I'll submit a PR tomorrow that does proper validation checking

Anton Kulaga

@antonkulaga

Looks like last minor version breaks the code

are there any plans to develop hammock further? Like adding multipart requests support and fixing the bug for txt results?

I am avaluating if I should switch to another lib or stay with hammock

Anthony Cerruti

@srnb_gitlab

@antonkulaga What's your usecase

Hammock is currently the only lib with both scala and scalajs support as far as I know

Anton Kulaga

@antonkulaga

@srnb_gitlab not really, there are several libs

I think at least 3 scala http clients support both jvm and js

I currently use 2 clients inside my https://github.com/antonkulaga/cromwell-client/ because hammock does not support multipart

I also use other scalajs client in another app, because hammock cannot process txt output (that is common for gene expression omnibus) properly (it ignores new lines there)

Michał Płachta

@miciek

what's broken with the current version? I will have a look at this tomorrow.

I am also planning to add multipart

Pepe García

@pepegar

Thanks @miciek for your help!

@antonkulaga lately I didn't have much time to put into the development of Hammock

PRs are open though, I'll review anything you want to contribute

Alice

@fp-alice

Anyone around?

I'm having this problem trying to use Hammock to make a simple get request

I'm trying to get plain text from this url and circe says it has a parsing error which makes me think it's being treated as json

Exception in thread "main" hammock.CodecException: expected whitespace or eof got '.48.16...' (line 1, column 8)
    at hammock.CodecException$.withMessageAndException(Codec.scala:10)
    at hammock.circe.HammockDecoderForCirce.$anonfun$decode$1(CirceCodec.scala:17)
    at scala.util.Either$LeftProjection.map(Either.scala:580)
    at hammock.circe.HammockDecoderForCirce.decode(CirceCodec.scala:17)
    at hammock.circe.HammockCodecForCirce.decode(CirceCodec.scala:29)
    at hammock.marshalling$$anon$1.apply(marshalling.scala:35)
    at hammock.marshalling$$anon$1.apply(marshalling.scala:33)
    at cats.data.EitherK.$anonfun$fold$2(EitherK.scala:84)
    at scala.util.Either.fold(Either.scala:125)
    at cats.data.EitherK.fold(EitherK.scala:84)
    at cats.arrow.FunctionK$$anon$2.apply(FunctionK.scala:47)
    at cats.arrow.FunctionK$$anon$2.apply(FunctionK.scala:47)
    at cats.free.Free.$anonfun$foldMap$1(Free.scala:155)
    at cats.StackSafeMonad.tailRecM(StackSafeMonad.scala:15)
    at cats.StackSafeMonad.tailRecM$(StackSafeMonad.scala:15)
    at cats.effect.IOLowPriorityInstances$IOEffect.tailRecM(IO.scala:745)
    at cats.free.Free.foldMap(Free.scala:153)
    at cats.free.Free.$anonfun$foldMap$1(Free.scala:156)
    at cats.StackSafeMonad.tailRecM(StackSafeMonad.scala:15)
    at cats.StackSafeMonad.tailRecM$(StackSafeMonad.scala:15)
    at cats.effect.IOLowPriorityInstances$IOEffect.tailRecM(IO.scala:745)
    at cats.StackSafeMonad.$anonfun$tailRecM$1(StackSafeMonad.scala:16)
    at cats.effect.internals.IORunLoop$.liftedTree3$1(IORunLoop.scala:225)
    at cats.effect.internals.IORunLoop$.step(IORunLoop.scala:225)
    at cats.effect.IO.unsafeRunTimed(IO.scala:321)
    at cats.effect.IO.unsafeRunSync(IO.scala:240)
    at wtf.shekels.alice.proxyscraper.Main$.main(Main.scala:7)
    at wtf.shekels.alice.proxyscraper.Main.main(Main.scala)
Caused by: io.circe.ParsingFailure: expected whitespace or eof got '.48.16...' (line 1, column 8)

This is how I'm making the request basically

  def request(location: String): IO[List[String]] = {
    Hammock.request(Method.GET, uri"$location", Map())
      .as[List[String]]
      .exec[IO]
  }

Pepe García

@pepegar

Hey @fp-alice

Alice

@fp-alice

Hello :D

Pepe García

@pepegar

so, what you're receiving from $location is plain text, not JSON?

Alice

@fp-alice

correct

Pepe García

@pepegar

Ok, then, the problem you're having is that the as method is using a circe codec, and circe tries to parse the response as a JSON

the error comes from there, I believe

If you want to get just the plain text you could:

Pepe García

@pepegar

scala> import hammock._, hammock.asynchttpclient._, Hammock._, AsyncHttpClientInterpreter._, cats._, cats.implicits._, cats.effect._
import hammock._
import hammock.asynchttpclient._
import Hammock._
import AsyncHttpClientInterpreter._
import cats._
import cats.implicits._
import cats.effect._

scala> val response: HttpResponse = Hammock.request(Method.GET, uri"http://google.com", Map()).exec[IO].unsafeRunSync
response: hammock.HttpResponse =
HttpResponse(Status(301,Moved Permanently,This and all future requests should be directed to the given URI.),Map(Location -> http://www.google.com/, Server -> gws, X-Frame-Options -> SAMEORIGIN, X-XSS-Protection -> 0, Expires -> Sun, 25 Aug 2019 18:47:14 GMT, Content-Length -> 219, Cache-Control -> public, max-age=2592000, Content-Type -> text/html; charset=UTF-8, Date -> Fri, 26 Jul 2019 18:47:14 GMT),StringEntity(<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>
,hammock.ContentType$$anon$2@2e40703e))

scala> val entity = response.entity
entity: hammock.Entity =
StringEntity(<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>
,hammock.ContentType$$anon$2@2e40703e)

scala> val text = entity.cata(_.body, Function.const(""), Function.const(""))
text: String =
"<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>
"

(BTW, do not unsafeRunSync directly, control your effects in IO)

but in this case, what we're doign is first getting the Entity out of the HttpResponse. Entity is the datatype that wraps the content of the response and it can be of several different types, StringEntity for plain text, ByteArrayEntity for bytes, etc

Alice

@fp-alice

I'm still very new to using IO, so learning, but thanks a lot

Pepe García

@pepegar

then, since you want to focus explicitly on the plain text, you can call cata on your Entity. cata is like a pattern match, and we pass three lambdas to it. The first lambda is the one Hammock will use to extract data from the StringEntity, your case. In my example, I'm just extracting the body with _.body. For the other two cases I'm just returning a plain empty string.

Alice

@fp-alice

How would I perform this operation before evaluating the IO?

like without the unsafeRunsync

.map after the .exec i assume

That did it! Thanks so much

didn't expect people to actually be here, I appreciate the help

and good work on your library :)

Pepe García

@pepegar

Thank you, glad it helps ;)

Alice

@fp-alice

Hm actually I have a new problem now ;o

This may or may not be your problem I may have fucked it up myself

I'm trying to scrape proxies, and when I do a curl request it responds a list of IPs with newlines between them to separate

using the function you gave me the newlines are stripped out

My code as per your suggestion now looks like


    Hammock.request(Method.GET, uri"$location", Map())
      .exec[IO]
      .map(_.entity.cata(_.body, Function.const(""), Function.const("")))

Anton Kulaga

@antonkulaga

Any updates on 2.13 support?

Michał Płachta

@miciek

I am trying to bring more people to hammock as it's one of the nicest way to start contributing to FP Scala projects. Hopefully will find somebody for 2.13 support.

Pepe García

@pepegar

That's awesome @miciek, great to hear that :)

Giuseppe Cannella

@gekomad

does hammock use sock proxy?

Where communities thrive

People

Repo info

Activity