I think at least 3 scala http clients support both jvm and js
I currently use 2 clients inside my https://github.com/antonkulaga/cromwell-client/ because hammock does not support multipart
I also use other scalajs client in another app, because hammock cannot process txt output (that is common for gene expression omnibus) properly (it ignores new lines there)
I am also planning to add multipart
@antonkulaga lately I didn't have much time to put into the development of Hammock
PRs are open though, I'll review anything you want to contribute
I'm having this problem trying to use Hammock to make a simple get request
I'm trying to get plain text from this url and circe says it has a parsing error which makes me think it's being treated as json
Exception in thread "main" hammock.CodecException: expected whitespace or eof got '.48.16...' (line 1, column 8)
at hammock.CodecException$.withMessageAndException(Codec.scala:10)
at hammock.circe.HammockDecoderForCirce.$anonfun$decode$1(CirceCodec.scala:17)
at scala.util.Either$LeftProjection.map(Either.scala:580)
at hammock.circe.HammockDecoderForCirce.decode(CirceCodec.scala:17)
at hammock.circe.HammockCodecForCirce.decode(CirceCodec.scala:29)
at hammock.marshalling$$anon$1.apply(marshalling.scala:35)
at hammock.marshalling$$anon$1.apply(marshalling.scala:33)
at cats.data.EitherK.$anonfun$fold$2(EitherK.scala:84)
at scala.util.Either.fold(Either.scala:125)
at cats.data.EitherK.fold(EitherK.scala:84)
at cats.arrow.FunctionK$$anon$2.apply(FunctionK.scala:47)
at cats.arrow.FunctionK$$anon$2.apply(FunctionK.scala:47)
at cats.free.Free.$anonfun$foldMap$1(Free.scala:155)
at cats.StackSafeMonad.tailRecM(StackSafeMonad.scala:15)
at cats.StackSafeMonad.tailRecM$(StackSafeMonad.scala:15)
at cats.effect.IOLowPriorityInstances$IOEffect.tailRecM(IO.scala:745)
at cats.free.Free.foldMap(Free.scala:153)
at cats.free.Free.$anonfun$foldMap$1(Free.scala:156)
at cats.StackSafeMonad.tailRecM(StackSafeMonad.scala:15)
at cats.StackSafeMonad.tailRecM$(StackSafeMonad.scala:15)
at cats.effect.IOLowPriorityInstances$IOEffect.tailRecM(IO.scala:745)
at cats.StackSafeMonad.$anonfun$tailRecM$1(StackSafeMonad.scala:16)
at cats.effect.internals.IORunLoop$.liftedTree3$1(IORunLoop.scala:225)
at cats.effect.internals.IORunLoop$.step(IORunLoop.scala:225)
at cats.effect.IO.unsafeRunTimed(IO.scala:321)
at cats.effect.IO.unsafeRunSync(IO.scala:240)
at wtf.shekels.alice.proxyscraper.Main$.main(Main.scala:7)
at wtf.shekels.alice.proxyscraper.Main.main(Main.scala)
Caused by: io.circe.ParsingFailure: expected whitespace or eof got '.48.16...' (line 1, column 8)
This is how I'm making the request basically
def request(location: String): IO[List[String]] = {
Hammock.request(Method.GET, uri"$location", Map())
.as[List[String]]
.exec[IO]
}
the error comes from there, I believe
If you want to get just the plain text you could:
scala> import hammock._, hammock.asynchttpclient._, Hammock._, AsyncHttpClientInterpreter._, cats._, cats.implicits._, cats.effect._
import hammock._
import hammock.asynchttpclient._
import Hammock._
import AsyncHttpClientInterpreter._
import cats._
import cats.implicits._
import cats.effect._
scala> val response: HttpResponse = Hammock.request(Method.GET, uri"http://google.com", Map()).exec[IO].unsafeRunSync
response: hammock.HttpResponse =
HttpResponse(Status(301,Moved Permanently,This and all future requests should be directed to the given URI.),Map(Location -> http://www.google.com/, Server -> gws, X-Frame-Options -> SAMEORIGIN, X-XSS-Protection -> 0, Expires -> Sun, 25 Aug 2019 18:47:14 GMT, Content-Length -> 219, Cache-Control -> public, max-age=2592000, Content-Type -> text/html; charset=UTF-8, Date -> Fri, 26 Jul 2019 18:47:14 GMT),StringEntity(<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>
,hammock.ContentType$$anon$2@2e40703e))
scala> val entity = response.entity
entity: hammock.Entity =
StringEntity(<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>
,hammock.ContentType$$anon$2@2e40703e)
scala> val text = entity.cata(_.body, Function.const(""), Function.const(""))
text: String =
"<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>
"
(BTW, do not
unsafeRunSync directly, control your effects in IO)
but in this case, what we're doign is first getting the
Entity out of the HttpResponse. Entity is the datatype that wraps the content of the response and it can be of several different types, StringEntity for plain text, ByteArrayEntity for bytes, etc
then, since you want to focus explicitly on the
plain text, you can call cata on your Entity. cata is like a pattern match, and we pass three lambdas to it. The first lambda is the one Hammock will use to extract data from the StringEntity, your case. In my example, I'm just extracting the body with _.body. For the other two cases I'm just returning a plain empty string.
like without the unsafeRunsync
.map after the .exec i assume
That did it! Thanks so much
didn't expect people to actually be here, I appreciate the help
and good work on your library :)
This may or may not be your problem I may have fucked it up myself
I'm trying to scrape proxies, and when I do a curl request it responds a list of IPs with newlines between them to separate
using the function you gave me the newlines are stripped out
My code as per your suggestion now looks like
Hammock.request(Method.GET, uri"$location", Map())
.exec[IO]
.map(_.entity.cata(_.body, Function.const(""), Function.const("")))
Hello Hammock folks! I noticed that there are out-of-the-box interpreters for many common http clients, but I didn't see one for Http4s' client. If I were using Hammock in a project that is using http4s, is there a recommended interpreter that I should use? Is it recommended / generally OK to mix http clients? Or should I plan on implementing my own interpreter using the Http4s client?
@rlmark I think one project should use one HTTP client generally. Of course there will be exceptions. However I feel that you can mix different HTTP server and HTTP client in the same project, as those are different concerns IMHO.
Hi! I am sorry for a very basic question... I would like to use Hammock to send a get response. I took the example from https://github.com/pepegar/hammock. I modified code a bit to make it an IOApp. Here is the code that I use:
import cats.effect.{ExitCode, IO, IOApp}
import hammock._
import hammock.marshalling._
import hammock.apache.ApacheInterpreter
import hammock.circe.implicits._
import cats.syntax.functor._
object HttpClient extends IOApp {
implicit val interpreter: InterpTrans[IO] = ApacheInterpreter.instance[IO]
val response: IO[List[String]] = Hammock
.request(Method.GET, uri"https://api.fidesmo.com/apps", Map())
.as[List[String]]
.exec[IO]
override def run(args: List[String]): IO[ExitCode] =
response.flatMap(r => IO(println(r))).as(ExitCode.Success)
}Here is my build.sbt. I took dependencies from https://github.com/pepegar/hammock (I added hammock-circe for hammock.circe.implicits._):
lazy val root = (project in file("."))
.settings(
name := "hammock-test",
version := "0.0.1-SNAPSHOT",
scalaVersion := "2.12.12",
libraryDependencies ++= Seq(
"com.pepegar" %% "hammock-core" % "0.10.0",
"com.pepegar" %% "hammock-apache-http" % "0.10.0",
"com.pepegar" %% "hammock-circe" % "0.10.0"
),
)
scalacOptions ++= Seq("-deprecation", "-encoding", "UTF-8", "-language:higherKinds", "-language:postfixOps", "-feature", "-Xfatal-warnings")When I try to compile it I get the following error:
.../hammock-test/src/main/scala/HttpClient.scala:14:8
not enough arguments for method as: (implicit D: hammock.Decoder[List[String]], implicit M: hammock.marshalling.MarshallC[[γ$0$]cats.data.EitherK[hammock.HttpF,hammock.marshalling.MarshallF,γ$0$]])cats.free.Free[[γ$1$]cats.data.EitherK[hammock.HttpF,hammock.marshalling.MarshallF,γ$1$],List[String]].
Unspecified value parameter M.
.as[List[String]]Could you suggest how I fix that please? Thank you!
