crystal-lang/crystal

The Crystal programming language | http://crystal-lang.org | Fund Crystal's development: http://is.gd/X7PRtI | Docs: http://crystal-lang.org/docs/ | API: http://crystal-lang.org/api/

David Blurton

@davidblurton

I guess I can do the same with the HTTP::Client response.body

use the version with a block to read into a slice and carefully convert to a string

Chris Watson

@watzon

Why does Array not have a filter method? Is there something similar that's just named differently?

From IRC (bridge bot)

@FromIRC

<oprypin> yeah, definitely don't keep arbitrary bytes in a string, it just happens to work but you cant rely on it because all string methods assume utf-8

<oprypin> but yeah String.new(slice_of_bytes) is the way to blindly stuff things into a string

David Blurton

@davidblurton

But the data will round trip fine if I were to convert it back to binary?

From IRC (bridge bot)

@FromIRC

<oprypin> @watzon, select

<oprypin> @davidblurton, if you put data into a string, don't modify it, and then obtain raw data from it, yes, it will probably be fine. why would you do that though?

<oprypin> by the way if you read a string from a http endpoint producing arbitrary bytes, that will NOT be fine

<oprypin> probably, i dunno. really unreliable in any case

David Blurton

@davidblurton

Right, doesn't make any sense.

I'll stick to slices

Can just read a byte at a time and see if it's utf8

From IRC (bridge bot)

@FromIRC

<oprypin> @davidblurton, ...uh no?

<oprypin> it has to be known in advance, defined by a protocol, guessing never works

David Blurton

@davidblurton

I won't be guessing. But I do need to parse the data. Some of it will be strings, some of it bytes.

From IRC (bridge bot)

@FromIRC

<oprypin> davidblurton so you have an IO and call this when you expect utf-8 encoded text https://crystal-lang.org/api/0.23.1/IO.html#read_string%28bytesize%3AInt%29%3AString-instance-method and other methods otherwise

<oprypin> likely https://crystal-lang.org/api/0.23.1/IO.html#read%28slice%3ABytes%29-instance-method or more advanced variants

David Blurton

@davidblurton

def get_raw_body(url) 
  HTTP::Client.get url do |response|
    length = response.headers.get("content_length")[0].to_i

    slice = Slice(UInt8).new(length)
    response.body_io.read_fully(slice)
    slice
  end
end

@oprypin How can I improve this?

From IRC (bridge bot)

@FromIRC

<oprypin> i'm at a loss, can't see an improvement

David Blurton

@davidblurton

Just leave the data in the an Array? I feel like slices are nicer though

From IRC (bridge bot)

@FromIRC

<hightower2> Hey, isn't behavior of Dir.entries( "/directory") a bit weird given that it returns "." and ".." among the results, in unclear sort order? I called it and it returned ["three", "four", "one", ".", "two", ".."]

<oprypin> but maybe you don't actually want to turn it into a slice and leave it in the IO

David Blurton

@davidblurton

That's true, I could pass the IO directly to my body parser

From IRC (bridge bot)

@FromIRC

<oprypin> hightower2, the sort order is not weird, but the .. stuff is indeed silly; unusable

<oprypin> so with the IO you can read a few bytes to a slice, then read a few bytes as a string

<oprypin> which seems to be what you want

<oprypin> but it's always weird how inconvenient it is to go from IO to slice. i guess they don't want people to be wasteful with allocations

<oprypin> like, instead of making a loop creating a new slice every time this encourages reuse

David Blurton

@davidblurton

I guess I was expecting the client response to have a raw_body property or something

From IRC (bridge bot)

@FromIRC

<oprypin> @davidblurton, but it allows streaming, it does not contain the whole response

<oprypin> abstracting streaming is the whole reason for IO to exist in addition to Slice

David Blurton

@davidblurton

right, I'm looking for a simple way to read the whole body though

I can see that IO is a nice abstraction for reading and writing, but I don't really care about streams here

From IRC (bridge bot)

@FromIRC

<oprypin> @davidblurton, but it's also a nice abstraction for protocol parsing

David Blurton

@davidblurton

I agree, I was using Char::Reader before, until I realised some of the response was binary

so should be easy to swap that for an IO

John Jansen

@johnjansen

anyone got a nice way to timeout an http get

ive done it before, now i cant find the code, or remember what i did

forget i asked … stupid me ;-/

From IRC (bridge bot)

@FromIRC

<RX14> @johnjansen you need to use HTTP::Client.new then set the timeout on that client instance

<RX14> HTTP::Client needs some love to be honest

John Jansen

@johnjansen

yep … didnt think to look first, did i ;-)

From IRC (bridge bot)

@FromIRC

<hightower2> Hey, if I need to write a 16bit unsigned int in little endian order into some buffer, then 1) What buffer do I use? StringBuilder? and 2) How do I write something like that?

John Jansen

@johnjansen

@RX14 might have an answer to that, he was here a little while ago

From IRC (bridge bot)

@FromIRC

<RX14> hightower2, you should look at the write_bytes method on IO

<RX14> String.build yields you an IO instance

<RX14> so you can use that

<RX14> however writing a raw integer to a string is unlikely to be what you want

<RX14> strings shouldn't contain binary data

Where communities thrive

People

Repo info

Activity