ruby on rails - Working with stale data when performing asynchronous jobs with Sidekiq -
in order process events asynchronously , create activity feed, i'm using sidekiq , ruby on rails' global id.
this works types of activities, of them require data change time job performed.
here's made-up example:
class movie < activerecord::base include redis::objects value :score # stores integer in redis has_many :likes def popular? likes.count > 1000 end end
and sidekiq worker performing job every time movie updated:
class movieupdatedworker include sidekiq::worker def perform(global_id) movie = globalid::locator.locate(global_id) movieupdatedactivity.create(movie: movie, score: movie.score) if movie.popular? end end
now, imagine sidekiq lagging behind and, before gets chance perform job, movie's score
updated in redis, users unliked movie , popular
method returns false.
sidekiq ends working updated data.
i'm looking ways schedule jobs while making sure required data won't change when job performed. few ideas:
1/ manually pass in required data , adjust worker accordingly:
movieupdatedworker.perform_async( movie: self, score: score, likes_count: likes.count )
this work require reimplementing/duplicating methods rely on data such score
, popular?
(imagine app more these two/three movable pieces).
this doesn't scale either since serialized objects take lot of room in redis.
2/ stubbing methods on record passed in worker:
movieupdatedworker.perform_async( global_id, stubs: { score: score, popular?: popular? } ) class movieupdatedworker include sidekiq::worker def perform(global_id, stubs: {}) movie = globalid::locator.locate(global_id) # inspired rspec stubs.each |message, return_value| movie.stub(message) { return_value } end movieupdatedactivity.create(movie: movie, score: movie.score) if movie.popular? end end
this isn't functional, can imagine convenience of dealing actual record, not having reimplement existing methods, , dealing actual data.
do see other strategies "freeze" object data , asynchronously process them? think these two?
i wouldn't data stale since have newest version of it, no longer popular. sounds want stale version.
if don't want data have changed need cache somehow. either say, pass data job directly, or can add form of versioning of data in database , pass reference old version.
i think passing on data need redis reasonable way. serialize attributes care about, score.
Comments
Post a Comment