Rails, database connection pool and sidekiq concurrency.

This should be quite easy for everyone, but not everyone understands how connection pooling and concurrency cooperate and blame the part of the system which shouldn’t be blamed in the first place.

So here it goes.

Connection pool in Rails config/database.yml must have a larger value than the Concurrency level set in config/sidekiq.yml

Why? Because when you run rails app and sidekiq together on one node sidekiq is using connection pool settings from Rails app. You need to have some part of the pool for Rails app and some part of the pool for sidekiq tasks.

Simple as that. If you see timeouts in sidekiq saying that it couldn’t obtain database connection in time specified by timeout value (default is 5 secs) then most probably it means that you have misconfigured your application.

Rails, database connection pool and sidekiq concurrency.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.