Bug #17781: Resolv::DNS RequestID table allocations are never freed, causing DNS lookups to eventually hang - Ruby master - Ruby Issue Tracking System
https://bugs.ruby-lang.org/issues/17781
Sidekiqを長時間走らせておくと処理が止まる原因はこれかもわね…
not reproducible on:
- ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [x86_64-linux-musl] (ruby:2.7.2-alpine docker image)
- ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [x86_64-linux]
reproducible on:
- ruby 2.7.3p183 (2021-04-05 revision 6847ee089d) [x86_64-linux-musl] (ruby:2.7.3-alpine docker image)
- ruby 2.7.3p183 (2021-04-05 revision 6847ee089d) [x86_64-linux]
- ruby 3.0.0p0 (2020-12-25 revision 95aff21468) [x86_64-linux]
- ruby 3.0.1p64 (2021-04-05 revision 0fb782ee38) [x86_64-linux]
@zundan Oh!!!
Is this the problem everyone's been having?! How did you manage to find it?
And I never noticed it because I'm still using 2.6.6 in production...
@Gargron I've just noticed this Ruby's bug in a PR to a work related code, while I personally haven't been able to reproduce Sidekiq hanging after a while as Heroku restarts processes every 24 hours.
@Gargron @spla Yep. Reproduced locally on Ubuntu 20.04 and ruby 2.7.3p183 (2021-04-05 revision 6847ee089d) [x86_64-linux]
$ ruby /tmp/test.rb
Ran 65536 times
and top shows 99-100 %CPU for the ruby.
Also on a Hobby dyno on Heroku (waiting for a few minutes):
$ heroku run bash
~ $ which ruby
/app/bin/ruby
~ $ ruby --version
ruby 2.7.3p183 (2021-04-05 revision 6847ee089d) [x86_64-linux]
~ $ cat <<_END > test.rb
> require 'resolv'
> 65536.times { Resolv::DNS.new.getresource('www.example.net', Resolv::DNS::Resource::IN::A) }
> puts "Ran 65536 times"
> Resolv::DNS.new.getresource('www.example.net', Resolv::DNS::Resource::IN::A)
> puts "Ran 65537 times"
> _END
~ $ ruby test.rb
Ran 65536 times
^Z
[1]+ Stopped ruby test.rb
~ $ bg
[1]+ ruby test.rb &
~ $ top
:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
11 u6876 20 0 93216 34284 6100 R 99.7 0.1 1:59.79 ruby
: