hyperium/hyper

An HTTP library for Rust

Jim Baca

@desertjim_gitlab

Can someone explain what's wrong here?

rkfox

@rkfox

The error message tells you exactly what you should try.

= note: the following trait is implemented but not in scope; perhaps add a use for it:
use futures_util::future::try_future::TryFutureExt;

Jim Baca

@desertjim_gitlab

I tried that but got this error:
error[E0603]: module try_future is private
--> service.rs:4:27
|
4 | use futures_util::future::try_future::TryFutureExt;
| ^^^^^^^^^^

rkfox

@rkfox

use futures_util::try_future::TryFutureExt;

I think that should work.

Jim Baca

@desertjim_gitlab

It gives me the same error

Jim Baca

@desertjim_gitlab

The versions I'm using are:
futures = "0.3.5"
futures-util = "0.3.5"
hyper = "0.13.5"

rkfox

@rkfox

Hmm... maybe try "use futures::prelude::TryFutureExt" or "use futures::future::TryFutureExt"

aohan237

@aohan237

let s2 = Arc::new(RwLock::new(c_router));

let new_service = make_service_fn(move |_| async {
    let m_router = s2.clone();
    Ok::<_, GenericError>(service_fn(move |req| Router::handle_url(m_router, req)))
});

c_router is a struct for url matching

why it shows error

cant impl copy for Arc?

how to make this happen

aohan237

@aohan237

cannot infer an appropriate lifetime for lifetime parameter in function call due to conflicting requirements

Jim Baca

@desertjim_gitlab

@rkfox thanks! "use futures::future::TryFutureExt;" worked.

rkfox

@rkfox

@desertjim_gitlab Sure thing. Some types are that appear in the compiler are private. Usually because they are within a private module and you have to use the module one level up to include it.

aohan237

@aohan237

how to share struct data state in hyper?

any clone or rwlock or reference will have life time issue?

using async/await

rkfox

@rkfox

that is the big question...

aohan237

@aohan237

in the combinator era of hyper , when async/await is unavailable, i can share data using "hyper::rt::run(future::lazy" but in async/await it suddenly dont work

Jeffrey Smith

@jeffsmith82

i'm trying to add headers to a RequestBuilder if i create it and then try and add extra headers with req.header() it errors with "move occurs because req has type reqwest::blocking::request::RequestBuilder, which does not implement the Copy trait" what am i doing wrong ?

Jeffrey Smith

@jeffsmith82

anyone ? is there a better place to ask the question for reqwest ?

Michal 'vorner' Vaner

@vorner

This sounds like a more general Rust question than reqwest-related one, but if you could share the code, maybe someone could help you more.

But I guess what you're trying to do is something like let builder = client.builder(); builder.headers(...); builder.send() and if you look at the signature of headers, you see that it consumes self and returns a new RequestBuilder.

So you should do something like this instead: client.builder().headers().send() ‒ each one (except the last) will take a builder as an input and produce a builder on the output, not modify an existing builder.

Jeffrey Smith

@jeffsmith82

let req = self
.client
.post(&url)
.body(body)
.header("Content-Type", "text/xmlmc");

then i do this as i only want to add the header if that is true.

if self.jsonresp == true {
req.header("Accept", "text/json");
}

Michal 'vorner' Vaner

@vorner

Hmm, so you probably want somethnig like: let mut req = self.client....; if json { req = req.header(...); }

btw, you really don't need to compare bools to true in conditionals. They are already booleans enough ;-)

Jeffrey Smith

@jeffsmith82

that works perfectly. cheers for the help. still wrapping my head around ownership in rust.

Mayank Shah

@mayankshah1607

Good day, everyone.
I'm trying to contribute to the library by working on this issue - hyperium/hyper#2178
I understand the context of the issue, but I'm a little confused about where this AddOrigin utility must be implemented (I'm very new to Rust and this is the first time I'll be contributing to a Rust project).
I'd really appreciate if anyone could help me out a little. Thanks! :)

Mayank Shah

@mayankshah1607

Hello. Is there a better place to ask the above question? - I had a couple of questions regarding the issue. I commented on the issue thread but haven't gotten any response.

Thanks again! :)

Jasper

@jbg

@mayankshah1607 my advice would be to make a PR implementing your idea of how it should be done

there isn't a single "right answer", and until there is code to look at, the discussion is quite abstract and thus less likely to get engagement from other people involved in the project

Peter Mertz

@trezm

qq: how would I go about sending trailers on an http2 response from the server? Was trying to figure it out from the docs but can't seem to find how to add them to a Body.

Peter Mertz

@trezm

FWIW I made a wrapper around Body and implemented the HttpBody trait where I just manually track and return trailer values

Jeffrey Smith

@jeffsmith82

I'm trying to get the full body of a request using
let whole_body = hyper::body::to_bytes(req.into_body()).await?;

I then try and pass &req into another function and it says it cant as it was moved by into_body(). Can i split these into two so i could just pass the headers through to the function ?

Jeffrey Smith

@jeffsmith82

so looks like
let (parts, body) = req.into_parts();
then pass the body into
let whole_body = hyper::body::to_bytes(body).await?;
and then can pass the parts bit around.

Jasper

@jbg

yes, that's correct

it's a common idiom in rust that methods named into_*() take self by value (in other words, they move/consume what they're called on)

Frank Prößdorf

@endor

I'm trying to parse a (big) XML file, do some aggregation and then send the resulting data as JSON to another service. What I did so far is parse the XML into data structures, turn the data structures into JSON, send the JSON to the other service, all one after the other. That was using quite a bit of memory and also wasn't too fast. I thought using hyper::Body::channel could help with that. So I've been using it somewhat like this: https://gist.github.com/endor/4aab0dc39f844af634b6dbf37f9ad731.

I imagine this is probably not the most efficient way to use both hyper and tokio, but I haven't really understood the concepts in tokio yet.

First I tried to send every little bit of data I just processed with sender.send_data().await, but that seemed to be very slow even though it was using less memory. Then I tried to compile larger chunks first and send those the same way and that was faster (as in total request/processing time), but was using a bit more memory.

My question is whether I'm using hyper::Body::channel correctly in combination with tokio and whether there is some way to make this still faster. For reference, the XML I'm parsing is 147MB, the peak memory usage is about 430MB and the time it takes from start to finish is 3min, which seems quite long.

Jasper

@jbg

@endor did you do any profiling (or manual timing of bits of code) to figure out where the time is spent?

it seems to me that streaming out the JSON (which sounds like what you're doing with your use of Body::channel()) is probably the least of your worries

parsing and aggregating the XML is probably where the time is spent..?

Frank Prößdorf

@endor

I did some a while ago, but just to make sure I did one again. The XML parsing and aggregation part takes about 37s. This could likely also be improved, but it still leaves ~2.5min for the JSON sending. Or rather for the JSON building from internal data structures and sending.

I was thinking that the sending takes so long, because it makes a difference in how long the whole thing takes whether I send very small pieces or whether I first build larger JSON pieces and then send those.

Frank Prößdorf

@endor

Part of that remaining 2.5min goes also to the receiving service processing the JSON, but when checking on the memory usage, it seems like the JSON building and sending part takes at least 1.5min.

rkfox

@rkfox

It's difficult to say. It seems like you have the right approach. Are you sure there isn't a network limitation sending a large amount of data?

Frank Prößdorf

@endor

These results are from running both services on localhost, so I would assume there aren’t any network limitations. It’s hood to hear that the approach is good. I will try to measure a bit more when making the pieces of data I’m sending bigger and smaller. Does it make a difference that the queue size of the hyper channel is 1 vs something bigger? Is it to be expected that sending many smaller pieces would create a noticable overhead?

Where communities thrive

People

Repo info

Activity