Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 31 2019 07:07
    kud1ing edited #1757
  • Jan 31 2019 07:07
    kud1ing edited #1757
  • Jan 30 2019 11:57
    kud1ing edited #1757
  • Jan 30 2019 08:23
    ubnt-intrepid commented #1758
  • Jan 29 2019 19:00
    kud1ing edited #1757
  • Jan 29 2019 15:31
    ddosoff commented #1513
  • Jan 29 2019 12:19
    kud1ing edited #1757
  • Jan 29 2019 11:24
    surajpayu opened #1758
  • Jan 29 2019 07:42
    kud1ing commented #1757
  • Jan 29 2019 07:42
    kud1ing edited #1757
  • Jan 28 2019 21:44
    kud1ing commented #1757
  • Jan 28 2019 19:39
    kud1ing commented #1757
  • Jan 28 2019 19:39
    kud1ing edited #1757
  • Jan 28 2019 19:38
    kud1ing commented #1757
  • Jan 28 2019 19:08
    sfackler commented #1757
  • Jan 28 2019 19:07
    sfackler commented #1757
  • Jan 28 2019 18:55
    kud1ing edited #1757
  • Jan 28 2019 18:52
    kud1ing opened #1757
  • Jan 28 2019 07:55
    ddosoff commented #1752
  • Jan 28 2019 07:55
    ddosoff commented #1752
Jim Baca
@desertjim_gitlab
I tried that but got this error:
error[E0603]: module try_future is private
--> service.rs:4:27
|
4 | use futures_util::future::try_future::TryFutureExt;
| ^^^^^^^^^^
rkfox
@rkfox
use futures_util::try_future::TryFutureExt;
I think that should work.
Jim Baca
@desertjim_gitlab
It gives me the same error
Jim Baca
@desertjim_gitlab
The versions I'm using are:
futures = "0.3.5"
futures-util = "0.3.5"
hyper = "0.13.5"
rkfox
@rkfox
Hmm... maybe try "use futures::prelude::TryFutureExt" or "use futures::future::TryFutureExt"
aohan237
@aohan237
let s2 = Arc::new(RwLock::new(c_router));

let new_service = make_service_fn(move |_| async {
    let m_router = s2.clone();
    Ok::<_, GenericError>(service_fn(move |req| Router::handle_url(m_router, req)))
});
c_router is a struct for url matching
why it shows error
cant impl copy for Arc?
how to make this happen
aohan237
@aohan237
cannot infer an appropriate lifetime for lifetime parameter in function call due to conflicting requirements
Jim Baca
@desertjim_gitlab
@rkfox thanks! "use futures::future::TryFutureExt;" worked.
rkfox
@rkfox
@desertjim_gitlab Sure thing. Some types are that appear in the compiler are private. Usually because they are within a private module and you have to use the module one level up to include it.
aohan237
@aohan237
how to share struct data state in hyper?
any clone or rwlock or reference will have life time issue?
using async/await
rkfox
@rkfox
that is the big question...
aohan237
@aohan237
in the combinator era of hyper , when async/await is unavailable, i can share data using "hyper::rt::run(future::lazy" but in async/await it suddenly dont work
Jeffrey Smith
@jeffsmith82
i'm trying to add headers to a RequestBuilder if i create it and then try and add extra headers with req.header() it errors with "move occurs because req has type reqwest::blocking::request::RequestBuilder, which does not implement the Copy trait" what am i doing wrong ?
Jeffrey Smith
@jeffsmith82
anyone ? is there a better place to ask the question for reqwest ?
Michal 'vorner' Vaner
@vorner
This sounds like a more general Rust question than reqwest-related one, but if you could share the code, maybe someone could help you more.
But I guess what you're trying to do is something like let builder = client.builder(); builder.headers(...); builder.send() and if you look at the signature of headers, you see that it consumes self and returns a new RequestBuilder.
So you should do something like this instead: client.builder().headers().send() ‒ each one (except the last) will take a builder as an input and produce a builder on the output, not modify an existing builder.
Jeffrey Smith
@jeffsmith82

let req = self
.client
.post(&url)
.body(body)
.header("Content-Type", "text/xmlmc");

then i do this as i only want to add the header if that is true.

if self.jsonresp == true {
req.header("Accept", "text/json");
}

Michal 'vorner' Vaner
@vorner
Hmm, so you probably want somethnig like: let mut req = self.client....; if json { req = req.header(...); }
btw, you really don't need to compare bools to true in conditionals. They are already booleans enough ;-)
Jeffrey Smith
@jeffsmith82
that works perfectly. cheers for the help. still wrapping my head around ownership in rust.
Mayank Shah
@mayankshah1607
Good day, everyone.
I'm trying to contribute to the library by working on this issue - hyperium/hyper#2178
I understand the context of the issue, but I'm a little confused about where this AddOrigin utility must be implemented (I'm very new to Rust and this is the first time I'll be contributing to a Rust project).
I'd really appreciate if anyone could help me out a little. Thanks! :)
Mayank Shah
@mayankshah1607
Hello. Is there a better place to ask the above question? - I had a couple of questions regarding the issue. I commented on the issue thread but haven't gotten any response.
Thanks again! :)
Jasper
@jbg
@mayankshah1607 my advice would be to make a PR implementing your idea of how it should be done
there isn't a single "right answer", and until there is code to look at, the discussion is quite abstract and thus less likely to get engagement from other people involved in the project
Peter Mertz
@trezm
qq: how would I go about sending trailers on an http2 response from the server? Was trying to figure it out from the docs but can't seem to find how to add them to a Body.
Peter Mertz
@trezm
FWIW I made a wrapper around Body and implemented the HttpBody trait where I just manually track and return trailer values
Jeffrey Smith
@jeffsmith82

I'm trying to get the full body of a request using
let whole_body = hyper::body::to_bytes(req.into_body()).await?;

I then try and pass &req into another function and it says it cant as it was moved by into_body(). Can i split these into two so i could just pass the headers through to the function ?

Jeffrey Smith
@jeffsmith82
so looks like
let (parts, body) = req.into_parts();
then pass the body into
let whole_body = hyper::body::to_bytes(body).await?;
and then can pass the parts bit around.
Jasper
@jbg
yes, that's correct
it's a common idiom in rust that methods named into_*() take self by value (in other words, they move/consume what they're called on)
Frank Prößdorf
@endor

I'm trying to parse a (big) XML file, do some aggregation and then send the resulting data as JSON to another service. What I did so far is parse the XML into data structures, turn the data structures into JSON, send the JSON to the other service, all one after the other. That was using quite a bit of memory and also wasn't too fast. I thought using hyper::Body::channel could help with that. So I've been using it somewhat like this: https://gist.github.com/endor/4aab0dc39f844af634b6dbf37f9ad731.

I imagine this is probably not the most efficient way to use both hyper and tokio, but I haven't really understood the concepts in tokio yet.

First I tried to send every little bit of data I just processed with sender.send_data().await, but that seemed to be very slow even though it was using less memory. Then I tried to compile larger chunks first and send those the same way and that was faster (as in total request/processing time), but was using a bit more memory.

My question is whether I'm using hyper::Body::channel correctly in combination with tokio and whether there is some way to make this still faster. For reference, the XML I'm parsing is 147MB, the peak memory usage is about 430MB and the time it takes from start to finish is 3min, which seems quite long.

Jasper
@jbg
@endor did you do any profiling (or manual timing of bits of code) to figure out where the time is spent?
it seems to me that streaming out the JSON (which sounds like what you're doing with your use of Body::channel()) is probably the least of your worries
parsing and aggregating the XML is probably where the time is spent..?
Frank Prößdorf
@endor

I did some a while ago, but just to make sure I did one again. The XML parsing and aggregation part takes about 37s. This could likely also be improved, but it still leaves ~2.5min for the JSON sending. Or rather for the JSON building from internal data structures and sending.

I was thinking that the sending takes so long, because it makes a difference in how long the whole thing takes whether I send very small pieces or whether I first build larger JSON pieces and then send those.

Frank Prößdorf
@endor
Part of that remaining 2.5min goes also to the receiving service processing the JSON, but when checking on the memory usage, it seems like the JSON building and sending part takes at least 1.5min.
rkfox
@rkfox
It's difficult to say. It seems like you have the right approach. Are you sure there isn't a network limitation sending a large amount of data?
Frank Prößdorf
@endor
These results are from running both services on localhost, so I would assume there aren’t any network limitations. It’s hood to hear that the approach is good. I will try to measure a bit more when making the pieces of data I’m sending bigger and smaller. Does it make a difference that the queue size of the hyper channel is 1 vs something bigger? Is it to be expected that sending many smaller pieces would create a noticable overhead?
rkfox
@rkfox
I think that the chunk size could have a substantial impact on the performance, yes, but it sounds like a mysterious issue. It shouldn't make a huge difference locally..
I'm not really sure what kind of overhead there is with the channel, though.
Jordan Doyle
@w4
hey guys, i've got a protocol here which doesn't exactly abide by http standards in that the client requires that a status line is sent before the body is streamed to the server, how would i go about taking control of the sockets in hyper? or at least being able to flush a response without closing the 'req.body()'