Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Dec 05 22:58
    mperham commented #2678
  • Dec 05 22:51
    mperham commented #2678
  • Dec 05 21:56
    SampsonCrowley commented #2678
  • Dec 05 21:50
    SampsonCrowley commented #2678
  • Dec 05 21:49
    SampsonCrowley commented #2678
  • Dec 05 21:47
    mperham commented #2678
  • Dec 05 21:41
    SampsonCrowley commented #2678
  • Dec 05 18:00
    mperham commented #2678
  • Dec 05 17:35
    SampsonCrowley commented #2678
  • Dec 05 16:39
    mperham commented #4397
  • Dec 05 16:39
    mperham closed #4397
  • Dec 05 08:24
    riqwan opened #4397
  • Dec 04 23:32
    jason-yardley closed #4392
  • Dec 04 23:31
    jason-yardley commented #4392
  • Dec 03 17:43
    mperham closed #4396
  • Dec 03 17:43
    mperham commented #4396
  • Dec 03 17:36
    aayush2610 edited #4396
  • Dec 03 17:36
    aayush2610 commented #4396
  • Dec 03 16:59
    mperham commented #4396
  • Dec 03 16:51
    aayush2610 commented #4396
Bulat-Gumerov
@Bulat-Gumerov
Is there any sidekiq status http page/api endpoint which could return 200 OK or other response to make sure that it is running OK? I need this to get sidekiq readiness status in kubernetes
Mike Perham
@mperham
@Bulat-Gumerov Search the issues. mperham/sidekiq#3648
Bulat-Gumerov
@Bulat-Gumerov
How about HTTP GET on /sidekiq/stats?
Mike Perham
@mperham
The Web UI runs in puma, is independent of a Sidekiq process.
Bulat-Gumerov
@Bulat-Gumerov
Thanks, I’ve looked into this issue, I don’t want to install sidekiq_alive gem, is there any options for it?
Mike Perham
@mperham
Copy the alive code into your app. Reimplement it yourself. Sidekiq does not provide anything like that feature.
Mikael Henriksson
@mhenrixon
@mperham do you have any idea what might be causing an error like: Redis INFO err: write tcp 10.134.213.53:35188->10.134.213.53:6379: use of closed network connection? I am thinking it might somehow be connection/pool related but I am grasping at straws.
Mike Perham
@mperham
no clue
Jason Witkowski
@jwitko
Hey All, I have this really weird intermittent error appearing in my sidekiq logs. I get it about 5-10 times a day

2019-03-13T18:14:17.253Z 10 TID-w4o6y RefreshFeedItemWorker JID-3f55b473c8477fdf0adcd048 INFO: done: 0.444 sec
E, [2019-03-13T18:14:17.304391 #10] ERROR -- : #<Faraday::Response:0x007f1158da8908>
It always appears to start with this INFO level message (sometimes done, sometimes start, and then this error with no context and what looks to be a faraday object
Anyone ever seen this before or can offer advice?
Andrey Kryachkov
@kryachkov
Hi @here. What may be the downsides of setting average_scheduled_poll_interval to 1 second?
Mike Perham
@mperham
more polling traffic to/from redis
Andrey Kryachkov
@kryachkov
No performance or reliability drawbacks?
Mike Perham
@mperham
more traffic is a perf drawback, especially if your Redis is very busy. :-)
Andrey Kryachkov
@kryachkov
Thanks Mike!
Mikael Henriksson
@mhenrixon
Regarding my question over a month ago the error I posted then was just a fallout of the following: MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.. Out of Sync in other words.
Mike Perham
@mperham
@mhenrixon Thanks for following up, TIL!
Bulat-Gumerov
@Bulat-Gumerov
Hey @mperham what limits best for sidekiq pods cpu/ram usage in k8s?
chetan-verticalfox
@themainguru
@Bulat-Gumerov Depends on what you are trying to achive. We had huge background jobs + Sidekiq pods assigned for them. In events of high ram usage(crossing 4gb or 8gb), the pods restarted and jobs were lost.On the basis of trial and error we were able to set a limit.
Also refactored the code :)
Ilya Vassilevsky
@vassilevsky
Hello :) We need an in-memory cache inside the Sidekiq process. We plan to use some thread-safe map implementation. Just put it in a singleton and use it from all jobs. Is this a viable plan? Any pitfalls to avoid? Thank you!
Mike Perham
@mperham
@vassilevsky If you only have one Sidekiq process, sure. You can use Redis or memcached if you need something that is accessible from all processes.
Ismael Correa Castro
@Iarfen
hi! how I can open sidekiq? I've already runned bundle exec sidekiq but it doesn't open the browser tab
The Web UI runs in your app server so you need to start puma and go to /sidekiq.
Mikael Henriksson
@mhenrixon
@mperham any gotchas with on hooking into Sidekiq.on(:heartbeat)? Anything I should have in mind if I do?
Garrett T
@film42
I noticed v6 is planning on removing the server/active_record middleware. Is there documentation somewhere showing what has changed in rails?
Mike Perham
@mperham
the middleware has not been necessary since rails 5.
Garrett T
@film42
:+1: do you know if there's an issue that references the behavior changes in rails? I'm currently tracking down some optimizations around the clear_active_connections! path for rails 4
Mike Perham
@mperham
connection cleanup is handled by ActiveSupport::Executor now.
there's been a dozen issues around Executor over the last 2 years.
that class wraps both web request processing (in puma) and job execution (in sidekiq).
Garrett T
@film42
Awesome! Thank you. I didn't realize there was a new pattern.
We had started to move clear_active_connections to an async timer task allowing the request path to run without connection mgmt overhead. I noticed that wasn't in place for sidekiq, but I guess at this point, it doesn't even matter :)
Mark Cheshier
@mark.cheshier_gitlab
Anyone here using k8s+istio w/sidekiq? Having all sorts of trouble getting the sidekiq UI to display properly from a Sinatra app. The resources are there but the UI keeps sending everything to HOST/sidekiq, even if I try and explicitly rewrite from either the virtual service or in the Rack command.
Works great in ECS, just hoping someone has already gone through the pain of migrating to k8s and has some pointers :)
Mark Cheshier
@mark.cheshier_gitlab
In case anyone runs into this, the solution ended up being kind of hacky: make a dummy route in the virtual service and tell Rack to URLMap sidekiq to that spot. The key is to put your dummy route first as istio matches in order.
Kadu Ribeiro
@duduribeiro

Hey folks 👋

I'm using sidekiq (4.2.10) and I have this configuration


Sidekiq.configure_client do |config|
  config.client_middleware do |chain|
    chain.add MyModule::SidekiqLoggingMiddleware
  end
end

to add this SidekiqLoggingMiddleware as a middleware. this middleware relies on another class (FilterArguments)

module MyModule
  class SidekiqLoggingMiddleware
    def call(worker_class, msg, *)
      yield
    ensure
      filtered_args = FilterArguments.new(msg['args']).filter
      Rails.logger.info "Enqueued #{worker_class}##{msg['jid']} with args: #{filtered_args}"
    end
  end
end

this works fine..

but in development, when I change some code, (probably the hot reload) crashes something, and I get this error

ArgumentError in Api::ApplicationPlansController#masterize A copy of MyModule::SidekiqLoggingMiddleware has been removed from the module tree but is still active!

and I need to restart my server

do you folks have any idea why?
Kadu Ribeiro
@duduribeiro

I think that I fixed explicitly adding the module for the class that the middleware relies:

 filtered_args = MyModule::FilterArguments.new(msg['args']).filter

instead of :

filtered_args = FilterArguments.new(msg['args']).filter
Mike Perham
@mperham
Please upgrade to Sidekiq 5.
dev mode code reloading works much better in sidekiq 5.
William Flanagan
@wflanagan
hey all.. i've got a question.. I've got logs that look like this...
William Flanagan
@wflanagan
basically, my JSON is intermixed with non JSON messages that have "start" and "done XXX.xx" in them. I'm not sure how to get the start and done to stop.. without cutting off all INFO messages all together.. all other cases for the most part are handled and return JSON-formatted log entries..