Java split String - Split a String into tokens with S...

http://www.devdaily.com/java/edu/pj/pj010006

rss | twitter | software | about | contact

devdaily.com
Java split String - Split a String into tokens with StringTokenizer
Submitted by alvin on July 28, 2007 - 12:49am

new posts
Design Patterns in Java Win a copy of Introducing HTML5 (free) Linux processor and memory information commands Drupal websites speed and performance tuning Huge Drupal Boost module performance improvements Use sed to modify files in place How to schedule an automatic Mac wake up time How to succeed as a consultant Apache RedirectMatch wildcard examples Two Java inheritance tests / interview questions more

tags: break

java

java

parse

split

string

stringtokenizer

Register for a chance to win Introducing HTML5

Java String splitting FAQ: Can you share some examples of how to split a String in Java? When working with any general-purpose programming language, it's often necessary to break a large string into smaller components. Whether you're working with Unix system files, older Windows' ".ini" files, or maybe flat files in a text database, you'll often read in a record of information, and then break that record up into smaller chunks. More recently, I've used this method to programmatically interpret the contents of HTML pages for web "robots". In this article we'll demonstrate how to split a Java String into smaller components, called tokens. We'll begin by breaking a simple well-known sentence into words, and then we'll demonstrate how to use the same technique to work with a flat-file database.

Java StringTokenizer - Breaking a sentence into words

One of the most famous sentences in American history begins with the words "Four score and seven years ago". For our purposes, suppose this sentence is stored in a variable named speech, like this:

String speech = "Four score and seven years ago";

A simple snippet of code to break that sentence into individual words using the Java StringTokenizer class looks like this:

String speech = "Four score and seven years ago"; StringTokenizer st = new StringTokenizer(speech); while (st.hasMoreTokens()) { println(st.nextToken()); }

categories
android (17) best practices (54) career (43) cvs (27) design (17) drupal (38) eclipse (6) gadgets (105) git (12) intellij (4) java (396) jbuilder (20) jdbc (21) swing (74) jsp (9) latex (26) linux/unix (241) mac os x (271) mysql (47) news (106)

In this example, the variable speech is passed into the Java StringTokenizer constructor method. Because StringTokenizer is not given a field separator value as an input parameter, it uses it's default field separator, and assumes that fields within the string are separated by whitespace characters (spaces, tabs, and carriage-return characters). Therefore, each time through the while loop a word is printed on a separate line, and the resulting output from this snippet of code looks like this:

1 of 6

06/24/2011 05:25 PM

Java split String - Split a String into tokens with S...

http://www.devdaily.com/java/edu/pj/pj010006

ooa/ood (10) page 1 (280) perl (152) php (85) postgresql (17) ruby (54) servlets (10) svn (7) technology (67) testing (14) uml (21)

Four score and seven years ago

The while loop test checks to see if there are any tokens left in the st object. As long as there are, the println statement is executed. Once there are no tokens remaining, the println statement is skipped and the while loop is exited. One quick note before continuing: If you try this example, you'll need to import the java.util.* package to run this example, like this:

import java.util.*;

A Java split string example using a text database file

In the example just shown, a text string was broken down into separate tokens. Of course in the English language we call these tokens "words", and these words are usually separated by whitespace. In our next example, we'll show how to break a database record (separated by colon characters) into tokens typically called "fields". The following two records are from a hypothetical customer database file named customer.db. Each record contains information about a customer, including their first name, last name, and the city and state of their address. Within a record, each field is separated by a colon character.

Homer:Simpson:Springfield:??? Hank:Hill:Arlen:Texas

Assuming that the first record (the Homer Simpson record) was read into a String variable named dbRecord, the record could be broken up into four fields and printed as follows:

// assume 'dbRecord' is assigned the value of the first database record StringTokenizer st = new StringTokenizer(dbRecord, ":"); String fname = st.nextToken(); String lname = st.nextToken(); String city = st.nextToken(); String state = st.nextToken(); println("First Name: println("Last Name: println("City: println("State: " + fname); " + lname); " + city); " + state);

In this example, we assume that the variable dbRecord already contains the entire first record of information from our database. Because we know that the fields of each record are separated by the colon character, we specify that the colon character should be the field delimiter (or field separator) when we call the StringTokenizer constructor, like this:

StringTokenizer st = new StringTokenizer(dbRecord, ":");

2 of 6

06/24/2011 05:25 PM

Java split String - Split a String into tokens with S...

http://www.devdaily.com/java/edu/pj/pj010006

Java Certification Guide www.epractizelabs.com Java Certification Training Lab Complete study material. Java Database www.versant.com High Scalability, High Availability Large Data Sets, High Performance Informix 4GL Outsourcing www.moredata.eu Meet a team with 25+ yrs experience developing 4GL & OSS applications Learn English Vocabulary LearnRealEnglish.com The Secrets To Learning Vocabulary Faster. Free Email Course.

After that, it's a simple matter to break the record into it's four fields using the nextToken() method of the StringTokenizer class. This technique is demonstrated completely in Listing 1, where the entire
customer.db text database is read (record-by-record), and printed.

// Listing 1: TokenTest.java import java.io.*; import java.util.*; class TokenTest { public static void main (String[] args) { TokenTest tt = new TokenTest(); tt.dbTest(); }

void dbTest() { DataInputStream dis = null; String dbRecord = null; try { File f = new File("customer.db"); FileInputStream fis = new FileInputStream(f); BufferedInputStream bis = new BufferedInputStream(fis); dis = new DataInputStream(bis); // read the first record of the database while ( (dbRecord = dis.readLine()) != null) { StringTokenizer st = new StringTokenizer(dbRecord, ":"); String fname = st.nextToken(); String lname = st.nextToken(); String city = st.nextToken(); String state = st.nextToken(); System.out.println("First Name: System.out.println("Last Name: System.out.println("City: System.out.println("State: } } catch (IOException e) { // catch io errors from FileInputStream or readLine() System.out.println("Uh oh, got an IOException error: " + e.getMessage()); } finally { // if the file opened okay, make sure we close it if (dis != null) { try { dis.close(); } catch (IOException ioe) { System.out.println("IOException error trying to close the file: " + e.getMessage()); " + fname); " + lname); " + city); " + state + "\n");

3 of 6

06/24/2011 05:25 PM

Java split String - Split a String into tokens with S...

} } // end if } // end finally } // end dbTest } // end class

http://www.devdaily.com/java/edu/pj/pj010006

Listing 1 (above): The file TokenTest.java program demonstrates a method to read every record in a text database file (customer.db), and break the data records into tokens (i.e., fields) using the StringTokenizer class. Although the code shown in Listing 1 is not a good example of object-oriented programming style, it does demonstrate our technique of reading a file and breaking it's records into fields using the StringTokenizer class.

Java split String examples - Summary

The simple method we've shown here is a powerful way of breaking a String into tokens. If you need a more powerful tokenizer, you might look at the StreamTokenizer class instead. The StreamTokenizer class can recognize various comment styles of programming languages, and offers a number of control flags that can be set to various states. If you'd like to download the source code shown in Listing 1, just click here, and the Java code will be displayed in your browser. Then just use the "File | Save As" option of your browser to save the source code to your system. To download the customer.db database file, just click here, and follow the same procedure to save the file to your local filesystem.

If this article was helpful, we appreciate your link:

Twitter

Facebook

StumbleUpon

Reddit

del.icio.us

Digg

related articles
Java StringTokenizer example Perl CSV file column extraction Opening and reading files with Java JDK 1.0.x Ruby CSV - An example of how to split CSV row data into fields Perl split function - how to process text data files

alvin's blog

Thanks
Submitted by Jwebuser (not verified) on November 10, 2009 - 9:58am.

Hi Alvin, thanks, this helped me much. ;-) reply

omg just perfect as with all

Submitted by Anonymous (not verified) on April 8, 2010 - 12:02pm.

omg just perfect as with all ur other tutorials..thanks so much reply

4 of 6

06/24/2011 05:25 PM

Java split String - Split a String into tokens with S...

http://www.devdaily.com/java/edu/pj/pj010006

I thank you for this. I've

Submitted by Ice-Man (not verified) on May 27, 2010 - 7:20pm.

I thank you for this. I've been looking for solution for my problem for days. reply

Post new comment

Your name:

Anonymous
E-mail:
The content of this field is kept private and will not be shown publicly.

Homepage:

Subject:

Comment: *

Notify me when new comments are posted All comments Replies to my comment

Preview

5 of 6

06/24/2011 05:25 PM

Java split String - Split a String into tokens with S...

http://www.devdaily.com/java/edu/pj/pj010006

java
java applets java faqs misc content java source code test projects lejos

unix
man (help) pages unix by example tutorials

misc
privacy policy terms & conditions subscribe unsubscribe wincvs tutorial function point analysis (fpa) fpa tutorial function point analysis software

other
living in alaska personal diary of selling a business mvp - alaska website design and programming codemee - java source code talkeetna, alaska

source code warehouse

bourne shell c/c++ examples css examples drupal examples php examples ruby examples

perl
perl faqs programs perl recipes perl tutorials perl source code

6 of 6

06/24/2011 05:25 PM