Last time we defined the meaning of a database and picked apart a few examples queries from the MySQL camp. To recap, we looked at CREATE TABLE, INSERT, UPDATE, ALTER TABLE, and SELECT statements. Just to refresh your memory, they use the following syntax (abbreviated listing): 

SELECT a, list, of, stuff  FROM tablename

ALTER TABLE tablename ADD name datatype other_parameters

UPDATE tablename SET colname = somevalue WHERE somecolumn = ‘some_value’

INSERT INTO tablename (list, of, colnames) VALUES (‘list’, ‘of’, ‘values’)

CREATE TABLE (colname_1 datatype, colname_2 datatype, …colname_N datatype) 

These are just a few of the basic commands or queries that are typically executed on a daily basis. Create Table is a little rarer in an established application since we are focusing on getting data into and out of an existing database. Applications that must dynamically create a database every run time are a niche product and do not represent typical behavior.

SQL Search Engine
Let’s say you want to make a SQL search engine that pulls data out from our members database. Suppose we want to generate a listing of all of the members, sorted alphabetically. We could construct the following query: 

mysql> SELECT * FROM members ORDER BY lastname ASC;

This will pull all of the data about each member and return a list that is sorted from A to Z. If we want the data returned from Z to A we would specify DESC (descending) after ORDER BY instead of ASC (ascending). At a minimum SQL will let you sort by an integer column, alphabetical (VARCHAR or TEXT), decimal or floating point, and date columns. Since the SQL engine doesn’t know the lexical ordering of raw binary data, BLOB columns aren’t typically sorted. Even TEXT columns, since they have an “unlimited” length, will only sort up to X amount of characters (this value can be changed at the cost of system efficiency). 

Grouping the Data
Well, that was fine and dandy, but I have over 25,000 members and that’s just too large of a list. Perhaps we could break it down a little more, let’s say by letter. We want to be able to create an interface like that below, where we can click on a letter and get all of the last names that begin with that letter, sorted alphabetically to return the following results:

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z 

Name                                      Email

Falwell, James                         jfalwell@gmail.com

Farney, Sarah Marie                sarah_baby@msn.net

Fardwood, Alexandra             cutsiepie99@hotmail.com

Fickleton, Martin                    martin_fickleton@cnn.com

Fiduciary, Dixon                     fdixon@gatech.edu

Fixington, Thomas                  thomas-the-tank@gmail.com

Flaherty, Timothy                   tflaherty@salvationarmy.org

We would construct the query:

mysql> SELECT lastname, firstname, email FROM members WHERE lastname LIKE “F%”;

 The syntax used here is a call to a SQL function “LIKE” that can take parameters such as the “F” and “%”. The % is a wildcard that matches any number of characters. There are different wildcards that can match single characters, or a specified number of characters. 

Perhaps you want a more advanced engine than can find parts of words (substrings) inside a given column. We can modify the above example by adding a “%” before the letter. 

mysql> SELECT lastname, firstname, email FROM members WHERE lastname LIKE “%F%”; 

Now we will get any and all members that have an “F” somewhere in their last name. The wildcards just say “find an f” even if there are letters to the left or right. Notice that % matches zero or more characters – the search will still return the members who’s last names begin with “F”.

Regular Expressions
If you want to do even more advanced searches you can use regular expressions. Regular expressions provide a concise and flexible means for identifying strings of text of interest, such as particular characters, words, or patterns of characters. Regular expressions are written in a formal language that can be interpreted by a regular expression processor, a program that either serves as a parser generator or examines text and identifies parts that match the provided specification. They can be very complex or very simple – returning results such as those above, or employed in an advanced data processing suite. Regular expressions are not SQL based, they are used in almost every developed programming language available. 

Regex based searches are generally very efficient and can be optimized for speed by an advanced programmer. As with most programming paradigms, there are many ways to employ regex strings to attack any one problem. There are “best-practices” to follow when using regular expressions, but in general you need to become familiar with the technology and syntax before you start optimizing statements. 

As exemplified by this XKCD comic, regular expressions are a very powerful tool that you will make you wonder what you ever did without them. Now, it’s not absolutely required that you start using regular expressions right now; in fact, it’s probably better for you to get accustomed to the SQL syntax before delving into this more complex topic. That being said, I’ll show you a few quick examples of what regex searches can do for you! 

Let’s start with the example above. We wanted to find all members whose last names started with the letter “F”. To make this into a regular expression based search, we’d simply type the following. 

mysql> SELECT lastname, firstname, email FROM members WHERE lastname REGEXP “^F”; 

or to be more explicit, 

mysql> SELECT lastname, firstname, email FROM members WHERE lastname REGEXP “^F.+$”;

If we break down the search string “^F” we have two components, the letter we are searching by and some other character “^”. This character just tells the regex engine to start looking at the beginning of the word. If we would omit the “^” we would get all results that had an “F” anywhere in the lastname column. 

The second example adds a few more characters that explicitly define our search as “starting at the beginning of the last name, search for an F and any other characters until the end.” The dollar sign “$” is a tag for the end of the string. “.+” just represents any number of any characters. The “.” specifier matches any character, including numbers and whitespaces, while the “+” symbol says to apply the previous specifier to any number of characters.

 This is a very basic example of regular expressions and the syntax used in more advanced SQL queries. I’ll be posting a simple table guide later so you don’t have to wade through an entire post to get the information you need! Let me know your thoughts in the comments!

Labels: Databases, Lesson, MySQL, queries, regular expressions, relational databases, SQL

This entry was posted on Monday, December 1, 2008 at 2:14 PM.