Various Ways to Run Shell Commands in Ruby

When building web applications, often you need to interact with the underlying operating system to run shell commands to accomplish specific tasks. For example, I recently wanted to combine two PDFs and needed to invoke a PDF processing library written in Go that handles it efficiently.

Ruby provides different ways to interact with the shell and run shell commands from the code. In this article, we'll learn them all, from the simplest to the most advanced.

Before proceeding, few things to keep in mind:

1. Do you really need to run a bash command? Can you do this task in pure Ruby, either with the core classes or the standard library? e.g. If you want to copy/move a file or directory, check out FileUtils library.
2. Are you running a command which was constructed with input from the users?
3. Are you running the command just for its side effects, or are you interested in the standard output (stdout)? Will you need standard error (stderr), as well?
4. Will you need the result codes?
5. How large is the result set? Do you want to stream the output while the command is still running in the background? or do you want to fetch everything at once and hold it in memory?
6. Do you want to run the command as a separate process and access that process as a Ruby object? Do you want to kill that process later, if it's taking too long?

Important Reminder: Make sure you don't call any commands that you receive from the end users without sanitizing them. Similar to SQL injection, these methods have potential security vulnerabilities.

The solution is the same: sanitize, escape, and parameterize your arguments. Instead of running ls #{directory}, provide the directory name separately, as an argument to the command.

With that out of the way, let's look at the different ways you can execute shell commands in Ruby.

`cmd` (backticks)

The simplest way to call a shell command in Ruby is to wrap the command in backticks, like ls -la. The Ruby interpreter will execute the command and return the output (as a string) that you can store in a variable.

files = `ls docs`

p files # "demo\ndocs\nimages\n"

%x{ cmd }

The %x{ cmd } works in exactly the same way as backticks.

files = %x{ ls docs }

p files # "demo\ndocs\nimages\n"

With both the above methods, we get the standard output (stdout), but not the standard error (stderr)

Reminder: Do not use the above two techniques if you're running user-generated commands. All the techniques below accept arguments separately, so they're safe to use.

system(cmd, args)

The Kernel#system method executes the command in a subshell. It returns true if the command returns a zero exit code, false for non-zero exit code, and nil if it fails.

result = system 'ls', '-l', '.'

puts "result: #{result}"

# Output

# total 16
# -rw-r--r--  1 akshay  staff  1014  4 Apr 08:51 draft.md
# -rw-r--r--  1 akshay  staff    57 28 Apr 17:52 scratch.rb
# result: true

If you aren't familiar with Unix exit codes, 0 means a command executed successfully, and non-zero means that it returned an error.

Note that you won't receive the output of the command. You'll only know if the command succeeded or not.

exec(cmd, args)

The Kernel#exec method replaces the current process and runs the provided command. Any code that comes after it is not executed as the current process was replaced and never continues.

files = exec 'ls'

# control won't reach here,
# as the current process was replaced
puts "result = #{files}"

# Output
# demo                    docs                    images                 

Now you might be wondering what exactly does it mean to 'replace the current process'? To understand it, try running the exec command in an IRB session.

➜ irb

irb(main):001:0> exec 'ls'
demo			images			scratch.rb

➜  scratch

Notice how it ended (replaced) the current IRB session and put you back in the terminal?

spawn(cmd)

The Kernel#spawn command executes the specified command and also returns the process ID as output.

pid = spawn("tar xf ruby.tar.bz2")
Process.wait pid

Note: This is similar to Kernel#system but won't wait for the command to finish. Hence you've to call Process.wait on the process.

IO.popen(cmd)

The IO.popen method runs the given command as a subprocess, connects the standard input and standard output of the command to a new IO stream, and returns that stream as a result.

You can also pass a block to this method, which received the IO stream as a parameter. The good thing about the block version is that the stream is closed when the block returns.

This stream is open for reading, writing, or both depending on which mode you pass to the method.

IO.popen('ls') do |pipe|
  puts pipe.readlines
end

This method gives you more control over the command, its input, and the output. Check out the docs to learn more. However, you still don't have the standard error. For this, use the next method.

Open3.popen3(cmd, args)

The popen3 method provided by the Open3 library gives you full control over the shell command. You can access stdin, stdout, stderr, and also a thread to wait for the child process when you're running another program.

require 'open3'

Open3.popen3("curl", "https://api.rubyonrails.org/") do |stdin, stdout, stderr, thread|
   pid = thread.pid
   5.times { puts stdout.readline }
end

# Output

# <!DOCTYPE html>
# <html lang="en">
# <head>
#     <title>Ruby on Rails API</title>
#     <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" # />

This command is the most powerful of them all.

To learn more about IO#popen3, read the official documentation and also check out this blog post for some examples.

That's a wrap. I hope you have a better understanding of how to run shell commands from your Ruby code, and also what to consider before you pick a solution.

If you're interested in learning more, I encourage you to read the following post from Tim Cuthbertson, which goes in-depth, providing additional nuances about these techniques.

Running a child process in Ruby (properly)

We use Ruby a lot here at Zendesk, and mostly it works pretty well for us. But one thing that sucks is when it makes the wrong solution…

Zendesk EngineeringTim Cuthbertson

I hope you liked this article and you learned something new.

As always, if you have any questions or feedback, didn't understand something, or found a mistake, please leave a comment below or send me an email. I look forward to hearing from you.

If you'd like to receive future articles directly in your email, please subscribe to my blog. If you're already a subscriber, thank you.