Golang Concurrency in Ruby
@sudhindraRao
Sudhindra Rao
(Uses Spaces not Tabs)
Follow me: @sudhindraRao
sudhindraRao.com
We are going to talk about Jim Carrey
Ruby and Concurrency
Mechanisms for Protection
Our application
Code
Threads all over ...
def start
super
@running = true
@thread = Thread.new(&method(:run))
end
def shutdown
@running = false
@thread.join
super
end
while true
# Grab the batch of records
...
# limit max workers to thread count to prevent overloading xray.
thread_pool = Thread.pool(thread_count)
thread_pool.process {
for xray_violation_url in xray_violation_urls_list do
pull_violation_details(xray_violation_url)
end
}
thread_pool.shutdown
end
Even Matz feels this way
As we experience the crashes ...
I know how to do this in Golang...
And if we can do this in Ruby...
Concurrency in Go - A Primer
Goroutines
Channels
Do not communicate by sharing memory; instead, share memory by communicating.
Pipelines
concurrent-ruby gem
why concurrent-ruby gem
Modern concurrency tools for Ruby. Inspired by Erlang, Clojure, Scala, Haskell, F#, C#, Java, and classic concurrency patterns.
Concurrent Ruby makes one of the strongest thread safety guarantees of any Ruby concurrency library, providing consistent behavior and guarantees on all four of the main Ruby interpreters (MRI/CRuby, JRuby, Rubinius, TruffleRuby).
It is critical to remember, however, that Ruby is a language of mutable references. No concurrency library for Ruby can ever prevent the user from making thread safety mistakes
Concurrent Ruby is also the only Ruby library which provides a full suite of thread safe and immutable variable types and data structures.
Why not Ruby 3.0
Concurrency / Parallel
It’s multi-core age today. Concurrency is very important. With Ractor, along with Async Fiber, Ruby will be a real concurrent language. — Matz
Ractor (experimental)
Ractor is an Actor-model like concurrent abstraction designed to provide a parallel execution feature without thread-safety concerns.
Rails, Sidekiq already use concurrent-ruby
Abstractions Solve Problems
Don’t forget the basics
Concurrent Ruby in Practice
def run
. . .
xray = Xray.new(@jpd_url, @username, @apikey, @wait_interval, @batch_size, @pos_file_path, router, @tag)
violations_channel = xray.violations(date_since)
xray.violation_details(violations_channel)
. . .
end
def violations(date_since)
violations_channel = Concurrent::Channel.new(capacity: @batch_size)
timer_task = Concurrent::TimerTask.new(execution_interval: @wait_interval, timeout_interval: 30) do
. . .
resp = get_violations(xray_json)
. . .
resp['violations'].each {|v| violations_channel = process(v, violations_channel) }
. . .
. . .
timer_task.execute
violations_channel
end
def violation_details(violations_channel)
violations_channel.each do |v|
Concurrent::Promises.future(v) do |v|
pull_violation_details(v['violation_details_url'])
pos_file = PositionFile.new(@pos_file_path)
pos_file.write(v)
end
end
end
it "creates a future for every violation" do
xray = Xray.new(@jpd_url, @username, @apikey, @wait_interval, @batch_size, @pos_file_path, @router, @tag)
(1..5).each do |i|
violations << i
end
promises = class_double("Concurrent::Promises").as_stubbed_const(:transfer_nested_constants => true)
expect(promises).to receive(:future).exactly(5).times
xray.violation_details(violations)
end
https://github.com/jfrog/fluent-plugin-jfrog-siem
References
THANK YOU! @sudhindraRao