Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Dec 04 03:15

    mhenrixon on gh-pages

    Update documentation (compare)

  • Dec 04 03:14

    mhenrixon on v7.0.0.beta9

    (compare)

  • Dec 04 03:14

    mhenrixon on master

    Bump sidekiq-unique-jobs to 7.0… (compare)

  • Dec 03 14:04
    serg-kovalev commented #411
  • Dec 03 13:00

    delete-merged-branch[bot] on improve-reliability

    (compare)

  • Dec 03 13:00

    mhenrixon on master

    Various changes to test and ver… (compare)

  • Dec 03 13:00
    mhenrixon closed #463
  • Dec 03 12:15
    mhenrixon synchronize #463
  • Dec 03 12:15

    mhenrixon on improve-reliability

    Upgrade the rubocop channel (compare)

  • Dec 03 12:01
    mhenrixon labeled #463
  • Dec 03 12:01
    mhenrixon labeled #463
  • Dec 03 12:01
    mhenrixon opened #463
  • Dec 03 12:01
    mhenrixon labeled #463
  • Dec 03 12:01
    mhenrixon assigned #463
  • Dec 03 12:00

    mhenrixon on improve-reliability

    Various changes to test and ver… (compare)

  • Dec 03 09:12
    mhenrixon closed #411
  • Dec 03 09:01
    joscal-nl commented #411
  • Nov 29 16:25
    mhenrixon commented #379
  • Nov 29 16:20
    KevinColemanInc commented #379
  • Nov 29 16:20
    KevinColemanInc commented #379
Alex Matchneer
@machty
[54] pry(main)> CheckoutBeaconRefreshLocationWorker.perform_async(2)
=> "95fa3812506f2b0afcd8a9bc"
[55] pry(main)> CheckoutBeaconRefreshLocationWorker.perform_async(2)
 SidekiqUniqueJobs::Client::Middleware DIG-uniquejobs:96d85bb9beaebfa5dfbb531691b2372d INFO: skipping job with id (70b94bf9cc88ddd5d7063092) because unique_digest: (uniquejobs:96d85bb9beaebfa5dfbb531691b2372d) already exists
=> "70b94bf9cc88ddd5d7063092"
i'm trying to understand the behavior of on_conflict: log
according to docs
This strategy is intended to be used with UntilExecuted and UntilExpired. It will log a line about that this is job is a duplicate of another.
so, does it actually skip the job?
because it appears the sidekiq job is still enqueued
based on 1. it still returns a JID 2. there's a sidekiq job enqueued in the Redis queue
@mhenrixon ^
Alex Matchneer
@machty
this appears to be what i'm running into: mhenrixon/sidekiq-unique-jobs#408
but i still really don't understand what's going on here and can't tell if the documentation is wrong
Alex Matchneer
@machty
i'm on the latest version and this issue does not appear to be fixed
much like #355, i'm expecting 1 job to start running, 1 to be enqueued, and 1 to be dropped
Alex Matchneer
@machty
instead, only the first performs
Mikael Henriksson
@mhenrixon
Yup @machty it is still a problem in the latest version. If you want you can give master (v7) a try. Problem should be fixed there.
Alex Matchneer
@machty
@mhenrixon which commit do you think fixed it?
Alex Matchneer
@machty
ok now i'm running into issues with some collision between the Sidekiq::Logging declared in sidekiq repo vs the one declared in unique-jobs
i guess Sidekiq::Logging had to be moved over as part of version 7?
Alex Matchneer
@machty
i don't want to needlessly pile on some of the negativity i've seen in the issues but as much work has gone into this gem, there are so many problems, outstanding decisions, and questionable decisions (why did it make sense to whole-sale copy Sidekiq::Logging from sidekiq core to this gem, especially given the risk of collisions and dependency hell?)
but there's really no choice but to move on to enterprise
I understand how difficult it is to maintain a concurrency library correctly, but I wish the README were more upfront with just how Alpha everything still is
Mikael Henriksson
@mhenrixon

@machty great attitude dude. With this gem you pay nothing, sidekiq enterprise is 180 USD per month and upwards. Unless you can contribute in a constructive way I suggesed you just pay the money to Mike Perham. You expect me to maintain as high quality for free as you get for sidekiq enterprise support? Then you really need to reevaluate your life man.

Yes, there has been a lot of work going into the gem. There have also been a lot of problems but if you find one, then create a PR to fix it.

I have a family, a full time job and two dogs ontop of a multitude of other interests. Unless people start paying me 180 EUR per month for this they shold NOT expect it to be perfect.

The logging is there to be able to test older versions. It is only required in tests and nowhere else
Markov Aleksey
@Piton4eg_twitter
hello, anybody knows does the gem work with sidekiq retry jobs?
I used the old version of sidekiq-unique-jobs and it worked
but I upgraded gem to the new version and it doesn't work now
without retry
Mikael Henriksson
@mhenrixon
Hi @Piton4eg_twitter you might want to use the master branch for now. It is pretty close to a pre-release and if you give it a try and run into any problems I will fix them immediately. Unfortunately version 6 was a terrible terrible idea. I will blog about it soon as well. I should be ashamed for releasing such terrible quality. I've put a lot of work into version 7 to fix those problems. Let me know and I'll prioritize getting v7 ready. Been a really tough time with becoming a father again in February, managing my consulting business and learning frontend development to build myself a website (required to avoid legal issues in Germany).
Markov Aleksey
@Piton4eg_twitter
@mhenrixon thanks for the answer, but no, it doesn't work too
2019-07-09T12:28:05.587Z 90053 TID-1s0 CardInteraction::Mastercard::StatementCreditsFileCreator JID-4d79a4d95ae211bc1a6af83d INFO: start
2019-07-09T12:28:05.589Z 90053 TID-1s0 uniquejobs-server DIG-uniquejobs:2c2652f0bf6ba3f8fcb5c545b8bb2c12 WARN: couldn't unlock digest: uniquejobs:2c2652f0bf6ba3f8fcb5c545b8bb2c12 4d79a4d95ae211bc1a6af83d
2019-07-09T12:28:05.590Z 90053 TID-1s0 INFO: done: 0.003 sec
it just silently returns
without retrying
I'll rollback to 4.0.17
it worked with this version
will think about other ways how to lock jobs
Mikael Henriksson
@mhenrixon
That is really interesting! I would love to get to the bottom of this so that I can fix the problem you are facing. Could you send me a gist with the configuration of sidekiq, unique jobs and the worker that you are trying?
I want to also provide a really simple way to manually handle locking directly in the worker much like sidekiq enterprise does with the limiter. I've basically built in a limiter directly into the locking mechanism but I want to also provide the same functionality without middleware.
Fixing the bug you are facing would be top priority though.
Markov Aleksey
@Piton4eg_twitter
I can send you in direct message
Mikael Henriksson
@mhenrixon
Yes please! Making unique jobs super reliable and non-problematic is one of my highest priorities outside of the family stuff.
I'm basically just waiting for some people to help me with testing at this point.
Markov Aleksey
@Piton4eg_twitter
@mhenrixon hello, I’ve created an issue with details about the problem which I wrote about couple weeks ago
please let me know if you need any additional details or help to debug the issue
Praveen
@metapraveen
Hi @mhenrixon thank you for this amazing gem and your time for creating this. I recently started using this gem at workplace and wanted to contribute to this in my free time. Wanted to understand how does lib/sidekiq-unique-jobs.rb is loaded on the start of sidekiq inside my_app. Sorry if this is dumb question.
Mikael Henriksson
@mhenrixon

@metapraveen no such thing as a stupid question. If anyone ever told you that they are arrogant! The filename you are asking about sidekiq-unique-jobs.rb is required by default when the gem is required. It has been a good few years since I investigated this but I believe that requiring the gem automatically attempts a require of a file in lib named the same thing as the gem. I haven't really tried to do anything about it but as you can see I've moved pretty much everything away from that file because I want to use lib/sidekiq_unique_jobs.rb which is how ruby likes files to be named.

So with that piece of background: when you require the gem (usually by an autoloader if rails or manually when building a grape app) , that is the actual file that gets loaded. From there on I do load some sidekiq middleware etc and setup some convenient default configuration values for you.

Even though I recently did improve this part a little it could probably be trimmed a bit more. I think your question also proves that I should provide some such information in a readme or wiki page. I think it would be really helpful for a lot of people out there to better understand how the gem is wired up.