Using Regular Expressions in Ruby (Part 1 of 3)

Nell Shamrell works as a Software Development Engineer for Blue Box. She also sits on the advisory board for the University of Washington Certificate in Ruby Programming. She specializes in Ruby, Rails, and Test Driven Development.  Prior to entering the world of software development, she studied and worked in the field of Theatre. The world of Theatre prepared her well for the dynamic world of creating software applications. In both, she strives to create a cohesive and extraordinary experience. In her free time she enjoys practicing the martial art Naginata.

This is the first installation of a three part series on regular expressions in Ruby.  Read the second and third parts.

Introduction

I’ll be honest, the first time I saw a regular expression I was intimidated. It seemed so cryptic and mysterious. I couldn’t even begin to comprehend how I might understand it, let alone write something like it. Until recently I only used regular expressions when absolutely necessary – maybe an email validation here, a basic substitution there. This kept me from even scratching the surface of regular expressions and Ruby. Ruby works with regular expressions in a harmony, a symphony of code. In order to fully harness Ruby, I had to move beyond my intimidation, my fear of regular expressions.

I have overcome this fear. Like anything in life, regular expressions are overwhelming until you break them down. I want to help you overcome your fear of regular expressions. So I’m going to break it down, step by step, and take you through the concepts of regular expressions in Ruby and on to advanced techniques. My hope is that you’ll see the beauty of regular expressions and be able to move beyond the fear and intimidation and embrace them in your code.

Regular Expressions

A regular expression is just a pattern. It’s a pattern that a string either fits or it doesn’t. Programming Ruby 1.9 by Dave Thomas (more commonly known as the pickaxe book) sums up what you do with regular expressions in three words–test, extract, change. You can test a string to see if it matches your pattern. You can extract a string (or part of a string) that matches your pattern. And you can change a string, replacing parts that match your pattern with other text. Test, extract, change. So simple, yet so powerful.

Regular Expressions in Ruby

Ruby lets us take regular expressions to further heights. In Ruby, everything is an object. This includes regular expressions. Since you can send messages to objects, you can also send messages to regular expressions. You can also assign them to variables, pass them to methods, and more.

As of Ruby 1.9, Ruby uses the Oniguruma regular expressions library. It provides all the standard regular expressions features that of other Regular regular Expressions expressions library, but adds additional features and extensions. It handles complex, multi-byte characters (such as Japanese text) very well. A feature I particularly like is I can use /h and /H as shorthand for hexadecimal digits.

I recently found out Ruby 2.0 use a different regular expressions library, Onigmo. Onigmo is forked off of Oniguruma and will add even more features that can be harnessed by Ruby. It will be quite interesting to see what it brings.

Regardless of what changes Onigmo makes, it won’t change the fundamental thing you do with a regular expression. You match your regular expression to a string. You compose a pattern and match a string to it.

Basic Matching

In most of my Ruby regular expressions, I use the =~ operator. This is Ruby’s basic pattern matching operator. When I use this operator, I’m asking Ruby “Does this string contain this pattern?”

Let’s go through an example:

/force/ =~ “Use the force”

On the left side I have a regular expression, which in this case is the literal word force. On the other side I have a quote from one of my very favorite movies, Star Wars. “Use the force.” When I run this, I’m asking Ruby if my pattern, on the left, exists in the string on the right.

What’s nice is I can format it the opposite way if I want to.

“Use the force” =~ /force/

I can have the string on the left and the regular expression on the right. I’m running the same function, just in different words – does my string contain my regular expression? Some people find this a little more readable.

When I run this, it returns the character number where my match starts. In this case, my match for the pattern /force/ begins on the eight character of my string.

“Use the force” =~ /force/
=> 8

I can also test if a string DOES NOT match a pattern by using the operator !~. This just returns a Boolean of true or false.

/dark side/ !~ “Use the force”

If I run this, asking whether the regular expression /dark side/ is not contained in my string “Use the force.” In this case, it returns true.

/dark side/ !~ “Use the force”
=> true

Operators are great for basic matching, knowing whether my string matches my regular expression or does not. But Ruby provides me much more information about my match. All I have to do is ask.

MatchData

The secret is to make my match a MatchData object. I create this object using the match method.

Let’s say I have a string.

string = “The force will be with you always”

And I want to find out if this string contains the word force. I can use this regular expression:

/force/

I call the match method on my regular expression and pass it my string. When I run this, it returns an instance of the MatchData class for my particular match.

my_match = /force/.match(string)
=> #<MatchData "force">

As of Ruby 1.9, I actually don’t have to start my match at the beginning of my string. I can pass a second argument, an integer, which means start my match on that character number.

my_match = /force/.match(string,5)
=> nil

In this case, it returns nil. In order to find a match for the word force, it needs to start matching earlier in the string.

For this example, however, I’m just going to pass my string:.

my_match = /force/.match(string)
=> #<MatchData "force">

I have access to methods which give me MUCH more information about my match because my match is an instance of the MatchData class.

If I call to_s on my match, it will return the entire text of my match.

my_match.to_s
=> “force”

If I call pre_match on my match, it will return the portion of my string that comes BEFORE my match:

my_match.pre_match
=> “The ”

If I call post_match on my match, it will return the portion of my string that comes AFTER my match:

my_match.post_match
=> “ will be with you”

All these methods (and there are several more) are quite useful. However, MatchData truly shines when it comes to capture groups, though. Capture groups are subexpressions within your regular expression. Let’s look at an example.

/(.*)force(.*)/

In order for a string to match this regular expression, it has to have any character appearing any number of times – that’s what the .* means- followed by the word force, followed by any character appearing any number of times.

Notice that I have the first and last part of this expression in parentheses. These are called groups. When I run this match on my string, it will store matches for my groups in memory. I can access these groups and use the in other places in my code.

my_match = /(.*)force(.*)/.match(string)

If I want to see all of these groups, my capture groups, I can call the captures method on my match.

my_match.captures
=> [“The “, “ will be with you always”]

MatchData objects are a lot like arrays. I can access individual captures using bracket notation, similar to how I access individual elements of an array.

If I run my_match[1], I get my first capture group, “The ”.

Similarly, my_match[2] returnd my second capture group, “ will be with you always”.

Notice I didn’t start with my_match[0], like I would with the first element of an array. If I run my_match[0], rather than getting my first match, I get the full string I ran the match on.

my_match[0]
=> “The force will be with you always”

It’s important to remember that although MatchData objects are similar to Arrays, they’re not actually arrays.

If I try to call an array method like each on my MatchData object, I get back a no method error.

my_match.each do |m|
 puts m.upcase
end
=> NoMethodError

I can easily fix this, however, by converting my my MatchData to an array using the to_a method.

my_match.to_a.each do |m|
 puts m.upcase
end
=> THE FORCE WILL BE WITH YOU ALWAYS
THE
WILL BE WITH YOU ALWAYS

Check back next week for the second installment in this series.