by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jun 17 01:49
    ioquatix synchronize #56
  • Jun 17 01:49

    ioquatix on scheduler

    `@ios` can be nil if the reacto… (compare)

  • Jun 16 12:26
    ioquatix synchronize #56
  • Jun 16 12:26

    ioquatix on scheduler

    Add support for native schedule… Defer fiber creation to reactor. Playing around with `Kernel::Fi… and 1 more (compare)

  • Jun 16 12:25

    ioquatix on master

    Add bake as development depende… (compare)

  • Jun 16 12:22
    ioquatix synchronize #56
  • Jun 16 12:22

    ioquatix on scheduler

    Add support for native schedule… Defer fiber creation to reactor. Playing around with `Kernel::Fi… and 1 more (compare)

  • Jun 16 12:22

    ioquatix on master

    Comment out local gems. (compare)

  • Jun 16 12:19

    ioquatix on master

    Remove unused `Rakefile`. (compare)

  • Jun 16 12:16
    ioquatix synchronize #56
  • Jun 16 12:16

    ioquatix on scheduler

    Add support for native schedule… Defer fiber creation to reactor. Playing around with `Kernel::Fi… and 1 more (compare)

  • Jun 16 12:15

    ioquatix on master

    Move bake-* development gems to… Remove unused `.yardopts`. (compare)

  • Jun 16 12:04
    ioquatix synchronize #56
  • Jun 16 12:04

    ioquatix on scheduler

    Fix scheduler specs. (compare)

  • Jun 16 03:06
    ioquatix synchronize #56
  • Jun 16 03:06

    ioquatix on scheduler

    Add support for native schedule… Defer fiber creation to reactor. Playing around with `Kernel::Fi… (compare)

  • Jun 11 14:26
    ioquatix commented #71
  • Jun 11 14:22
    ioquatix commented #71
  • Jun 11 14:21
    wyhaines commented #71
  • Jun 11 14:12

    ioquatix on master

    Remove unused `event.gemfile`. (compare)

Scott Tadman
@tadman
I solved my immediate problem with caching, but the real issue here is that when you exhaust filehandles from creating UDP sockets it's SecureRandom that fails (can't open /dev/urandom), so it's hard to tell what went wrong and where. Increasing file-handle counts is a temporary fix as well.
Samuel Williams
@ioquatix
Haha sorry I find that quite funny
Probavly
async-dns needs to cache sockets per-resolver
its quite an old code base so... probably need to refactor it
Scott Tadman
@tadman
When it comes to things that seems like missing features is it best to raise here first, or as an issue?
There's a number of things that seem needlessly opaque, lacking reflection tools.
For example.
Samuel Williams
@ioquatix
Issues on GitHub are best
what do you mean reflection tools?
Scott Tadman
@tadman
For example, it's really hard to get internal information from Async::IO::HostEndpoint. Does that describe a UDP endpoint? It won't tell you.
@specification is private and only exposed through things like to_s in a human-readable form.
Like you can get it through address, but you need to know the specific order of things, too.
Little things like endpoint.family or endpoint.udp? would be super handy.
Samuel Williams
@ioquatix
The only design requirements of Endpoint is to have connect, bind, accept, and each.
not all kinds of addresses support the things you are asking for, but certainly happy to add specific ones.
You cannot rely on a generic endpoint supporting udp?
Scott Tadman
@tadman
Well, as in Endpoint.udp produces a thing, and that thing should udp? presumably.
When I was digging into Async DNS it became obvious it had no easy way to differentiate short of connecting and finding out after the fact, which seems odd.
Or something obscure like endpoint.address[2]
It's a handy container which makes it a logical place for those sorts of utility methods.
Samuel Williams
@ioquatix
What should be Async::HTTP::Endpoint.udp??
Hey, is being able to read arbitrary data from the heap a security issue?
Scott Tadman
@tadman
Depends on who's doing the reading.
Samuel Williams
@ioquatix
haha yeah
but it would say, make a RCE worse
If you are running untrusted code, it could be an issue'
Scott Tadman
@tadman
RCE is always bad. If you can do that, you could coredump the process.
So it's like "infinity plus one" bad if you can read the heap too.
It'd be a problem if you're trying to make a sandbox. JavaScript perpetually battles stuff like that.
Scott Tadman
@tadman
Is this how to spin up Falcon inside an existing Async application but without using an external config.ru file? https://gist.github.com/tadman/bcbd560e507e0f0f5ddb25267b9e1775
Samuel Williams
@ioquatix
Do you only want a single process/thread?
Scott Tadman
@tadman
Yeah, looking to have it all running in one reactor. Thanks!
What wasn't clear was the requirement for Falcon::Server.middleware which is the thing that actually "rackifies" the app.
Samuel Williams
@ioquatix
You don't have to use it, but yes, it does apply the correct adaptors
Scott Tadman
@tadman
If you don't use that it complains about missing method [] on Http1 requests.
Not sure if that's supposed to work or not, but couldn't get it to without the middleware wrapper.
Samuel Williams
@ioquatix
You still need to use the rack adapter
Can Eldem
@eldemcan

:wave: I use async and async-http gem. Great performance improvements but I am struggling to run one of my tests. I am getting following error.

1 warn: Async::HTTP::WebMockClient [oid=0x3fcd94b74250] [pid=10106] [2020-06-04 17:25:06 +0100]
               | Waiting for Async::HTTP::Protocol::HTTPS pool to drain: #<Async::Pool::Controller(121/∞) >
  error: Async::Task [oid=0x3fcdac9769b8] [pid=10106] [2020-06-04 17:25:06 +0100]
               |   Errno::EMFILE: Too many open files - socketpair(2)

a lot of Too many open files - socketpair(2) error

https://github.com/spandx/spandx/pull/25/files => this is the MR that introduces async gem.
https://github.com/spandx/spandx/blob/master/spec/unit/cli/scan_spec.rb => this one fails and hangs in github ci.

My assumption is since we create asycn task for each http request I am hitting limit in my os for creating socket . I don't understand why this is happening with tests though since we don't actually do http request.

~/spandx [async●] » ulimit -a
-t: cpu time (seconds)              unlimited
-f: file size (blocks)              unlimited
-d: data seg size (kbytes)          unlimited
-s: stack size (kbytes)             8192
-c: core file size (blocks)         0
-v: address space (kbytes)          unlimited
-l: locked-in-memory size (kbytes)  unlimited
-u: processes                       5568
-n: file descriptors                256

if this is too complicated for chat I can create issue on project. (I am quite sure it is something related with me not with project)

Samuel Williams
@ioquatix
@eldemcan you need to increase the maximum number of file descriptors - in the output you give me the maximum number is 256 which is pretty low - try setting it to 50,000
Also you might want to check if you are putting some request limit using a semaphore so you don't just consume all available resources
Can Eldem
@eldemcan
@ioquatix thanks for the tip. I was using tmux and ulimit -a gives different result when you use it in normal terminal. I will try using semaphores next because I can't change those parameters in CI. Another interesting thing is that this problem occurs in rspec test, when I run same code as normal ruby process I don't get this error even though I am doing same amount of http call. (macos) However I had problems in raspberry pi process was stalling most probably again related with amount of FD I can open.
Can Eldem
@eldemcan

I wrote a script which basically reads url addresses from files and makes http requests. I used semaphore but when I run this script ruby VM crashes with error message. Can anybody point my mistake, https://gist.github.com/eldemcan/4b1bbf208262538ba8ad54585a111cb7
main.rb -> script
results.txt -> urls that script makes http request
error.txt -> error output

I tried same script in two different machines just to be safe but I got similar result

Samuel Williams
@ioquatix
I can take a look
it's odd to see it segfault
Samuel Williams
@ioquatix
@eldemcan I added a working version of your code in a comment on the same gist
Can Eldem
@eldemcan
@ioquatix thank you so much :clap: It would be nice to know what causing error, between gist that you provide and gist that I wrote I don't see major difference (maybe I am mistaken)
Samuel Williams
@ioquatix
My guess is your internet.close is outside any async block and thus causing problems.
internet.close does quite a bit of stuff w.r.t. http/2