Test-First Java Concurrency for the Classroom

SIGCSE 2010 Mathias Ricken and Robert Cartwright Rice University March 12, 2009

Two Trends
Test-driven development Concurrent programming

Brian Goetz, Java Concurrency in Practice, Addison-Wesley, 2006

2

Unit Testing Benefits

Occurs early Automates testing Keeps the shared repository clean Prevents bugs from reoccurring Allows safe refactoring Serves as documentation
3

Unit Testing in Assignments

Hand out test cases to students
Improves confidence and understanding

Instill good practices

Require students to extend test suites

Automated grading
Part graded automatically, part by hand

Moores Law Requires Concurrency

Adopted from Sutter 2009

Concurrency Is Difficult

Unit testing not effective in multi-threaded programs

Existing Unit Testing Frameworks

JUnit, TestNG Dont detect test failures in threads other than main thread
Failures in event thread not detected either

Dont ensure that other threads terminate Tests that should fail may succeed
7

Sample JUnit Tests

public class SimpleTest extends TestCase { public void testException() { throw new RuntimeException("booh!"); } } public void testAssertion() { assertEquals(0, 1); } }
if (0!=1) throw new AssertionFailedError();

Both tests fail.

JUnit Test with Child Thread

public class SimpleTest extends TestCase { public void testException() { new Thread() { new Thread() { public void run() {{ public void run() throw new RuntimeException("booh!"); throw new RuntimeException("booh!"); throw new RuntimeException("booh!"); }} }.start(); }.start(); Uncaught exception, } test should fail but } does not!
end of test success! Child thread uncaught!
9

Main thread

Child thread

Main thread

spawns

ConcJUnit
Backward compatible replacement for JUnit Detects exceptions in all threads
Exception handler for all child threads and the event thread

Ensures that child threads have terminated and event thread is done
Enumerate live threads after test Inspect event queue

Requires all child threads to be joined

Analyze join graph
10

Thread Creation Coordinates

In Thread.start() record stack trace of
Thread.currentThread() Easy to find where a thread that caused a failure was started Also shows where threads that outlived the test were started

11

Creation Coordinates Example

class Helper extends Thread { void m() { if (b) Assert.fail(); } public void run() { m(); } private boolean b; // } class Main { void foo() { // which one? new Helper(true).start(); new Helper(false).start(); // ... } } AssertionError: at Helper.m(Helper.java:2) at Helper.run(Helper.java:3)

Started at: at Main.foo(Main.java:4) at Main.bar(Main.java:15) at Main.main(Main.java:25)

12

ConcJUnit Demo

13

Concurrency Examples
In-class discussion
Multi-threaded counter: data races Multi-threaded bank: deadlock

Homework
Bounded buffer Readers-writer lock Test suite handed out to help students

Multi-threaded Breakout
14

Example: Counter
Class that can increment an integer variable N times Write test first
public class CounterTest extends TestCase { final long PER_THREAD = 1000000; public void testSingle() { Counter c = new Counter(); c.incrementNTimes(PER_THREAD); assertEquals(PER_THREAD, c.getCount()); } }
15

Counter: Implementation
Write implementation
public class Counter { private long count = 0;
public long getCount() { return count; } public void incrementNTimes(long n) { for(long i=0; i<n; ++i) { ++count; } } }

Test passes!
16

Counter: Multi-threaded Test

Write multi-threaded test
public void testMulti() { final Counter c = new Counter(); for(int i=0; i<NUM_THREADS; ++i) { new Thread() { public void run() { c.incrementNTimes(PER_THREAD); } }.start(); } TestUtils.waitForOtherThreads(); assertEquals(NUM_THREADS*PER_THREAD,c.getCount()); }

Test fails (most likely)!

17

Shared Data
Why does the multi-threaded counter test fail?
The count field is shared among threads The ++count operation is not atomic Thread may be interrupted after reading count, but before writing back to count
count=0 regA=? regB=? 0 0 ? 0 0 0 0 1 0 1 1 0 1 1 1 1 1 1
18

A1

regA = count; B1 regB = count; A2 regA = regA + 1; A3 count = regA; B2 regB = regB + 1; B3 count = regB;

Data Races
Definition
Two threads access the same data At least one access is a write Nothing prevents the order from changing

Would like code to execute atomically (without interruption)

Java does not support atomicity (for general code)

19

Java Locks & Synchronized

Java provides lock objects and synchronized blocks
synchronized(lock) { ++count; }

Thread must compete for ownership of lock object before entering synchronized block Synchronized block is not atomic But once a thread has a lock object, no other thread can execute code protected by the same lock object
20

Counter: Re-Write
Rewrite implementation
// ... private Object lock = new Object();

public void incrementNTimes(long n) { for(long i=0; i<n; ++i) { synchronized(lock) { ++count; } } }

Test passes!
21

Concurrency Still Difficult

Even race-free, deadlock-free programs are not deterministic
Thread scheduling is essentially nondeterministic

Different schedules may compute different results

May or may not be acceptable, depending on the task
22

Multi-threaded Breakout
Uses ACM Java Task Force material
Based on Breakout - Nifty Assignment by Eric Roberts, SIGCSE 2006

Multiple balls, each in its own thread

Atomicity assumption when removing bricks Ends game before all bricks are removed

Other problems
X,Y coordinate changes not atomic X,Y coordinates not volatile or synchronized, event thread may never see the updates

Correctly synchronized version still not deterministic

23

Future Work
Testing all schedules is intractable

Insert random delays/yields before synchronization operations

Must consider volatile variable accesses to comply with Java Memory Model Re-run program several times Can detect a number of sample problems

Record schedule, replay if test fails

Makes failures reproducible if found
*3
24

Conclusion
Unit testing has important benefits in industry and in the classroom Concurrent programming is becoming more important, and its difficult ConcJUnit helps

www.concutest.org www.drjava.org
25

Notes

Notes
1. Also cannot detect uncaught exceptions in a programs uncaught exception handler (JLS limitation) 2. Only add edge if joined thread is really dead; do not add if join ended spuriously. 3. Have not studied probabilities or durations for sleeps/yields: One inserted delay may negatively impact a second inserted delay Example: If both notify() and wait() are delayed.
27

Spurious Wakeup
4.
public class Test extends TestCase { public void testException() { Thread t = new Thread(new Runnable() { public void run() { throw new RuntimeException("booh!"); } Loop since }); join() may end t.start(); spuriously while(t.isAlive()) { try { t.join(); } catch(InterruptedException ie) { } } } }
28

Image Attribution

Image Attribution
1.

2.
3. 4.

Left image on Two Trends: Test Driven Development, Damian Cugley. Right image on Two Trends: adapted from Brian Goetz et al. 2006, Addison Wesley. Graph on Moores Law: Adapted from Herb Sutter 2009 Image on Concurrency Is Difficult: Caption Fridays

30

Extra Slides

Changes to JUnit (1 of 3)
Thread group with exception handler
JUnit test runs in a separate thread, not main thread Child threads are created in same thread group When test ends, check if handler was invoked

Reasoning: Uncaught exceptions in all threads must cause failure

32

JUnit Test with Child Thread

public class Test extends TestCase { public void testException() { new Thread(new Runnable() { public void run() { throw new RuntimeException("booh!"); } }).start(); } } invokes

TestGroups Uncaught Exception Handler

33

JUnit Test with Child Thread

public class Test extends TestCase { public void testException() { new Thread() { public void run() { throw new RuntimeException("booh!"); } }.start(); } }
Main thread spawns and joins Test thread Child thread resumes end of test uncaught! invokes exception handler check exception handler

failure!

34

Child Thread Outlives Parent

public class Test extends TestCase { public void testException() { new Thread() { public void run() { throw new RuntimeException("booh!"); } }.start(); } } check exception
Main thread Test thread Child thread end of test handler

success!
Too late!

uncaught!

invokes exception handler 35

Changes to JUnit (2 of 3)
Check for living child threads after test ends
Reasoning: Uncaught exceptions in all threads must cause failure If the test is declared a success before all child threads have ended, failures may go unnoticed Therefore, all child threads must terminate before test ends
36

Check for Living Child Threads

public class Test extends TestCase { public void testException() { new Thread() { public void run() { throw new RuntimeException("booh!"); } }.start(); } check for living check groups }
child threads handler Main thread Test thread Child thread end of test

failure!

uncaught!

invokes groups handler

37

Correctly Written Test

public class Test extends TestCase { public void testException() { Thread t = new Thread() { public void run() { /* child thread */ } }; t.start(); t.join(); // wait until child thread has ended } check for living check groups }
child threads handler Main thread Test thread Child thread end of test

success!

*4

38

Changes to JUnit (3 of 3)
Check if any child threads were not joined
Reasoning: All child threads must terminate before test ends Without join() operation, a test may get lucky Require all child threads to be joined

39

Fork/Join Model
Parent thread joins with each of its child threads
Main thread

Child thread 1
Child thread 2

May be too limited for a general-purpose programming language

40

Example of Other Join Models

Chain of child threads guaranteed to outlive parent Main thread joins with last thread of chain
Main thread Child thread 1 Child thread 2 Child thread 3
41

Modifying the Java Runtime

Changing Thread.start()and join()
Need to modify Java Runtime Library Utility to process users rt.jar file Put new jar file on boot classpath:
-Xbootclasspath/p:newrt.jar

Still works without modified Thread class

Just does not emit lucky warnings
42

Join with All Offspring Threads

Main thread joins with all offspring threads, regardless of what thread spawned them
Main thread Child thread 1 Child thread 2

43

Generalize to Join Graph

Threads as nodes; edges to joined thread Test is well-formed as long as all threads are reachable from main thread
Main thread Child thread 1 Child thread 2 Child thread 3 MT CT1 CT2 CT3
44

Join Graph Examples

Main thread MT

Child thread 1
Child thread 2 Main thread Child thread 1 Child thread 2

CT1 CT2

MT CT1 CT2

45

Unreachable Nodes
An unreachable node has not been joined
Child thread may outlive the test

Main thread Child thread 1 Child thread 2

MT CT1 CT2

46

Constructing the Graph

// in mainThread childThread.start();

Add node for childThread

main Thread

MT

child Thread

CT

47

Constructing the Graph

// in mainThread childThread.join();

When leaving join(), add edge from mainThread to childThread

main Thread MT

child Thread

CT

*2

48

Example: Multi-threaded Bank

Program simulating checking accounts Account balances are shared data
To avoid data races, use synchronized Need access to two accounts for transfers
synchronized(locks[from]) { synchronized(locks[to]) { accounts[from] -= amount; accounts[to] += amount; } }

Test hangs!

49

Deadlock
Thread A transfers from account 0 to 1 Thread B transfers from account 1 to 0 Thread A gets interrupted after acquiring locks[0]
// thread A synchronized(locks[0]) { // thread B synchronized(locks[1]) { synchronized(locks[0]) // cant continue, locks[0] // is owned by thread A */

synchronized(locks[1]) // cant continue, locks[1] // is owned by thread B */

50

Lock Acquisition Order

No deadlock if both threads had attempted to acquire lock 0 first When acquiring more than one lock object, always acquire them in the same order
e.g. acquire lower accounts lock object first
synchronized(locks[Math.min(from,to)]) { synchronized(locks[Math.max(from,to)]) { accounts[from] -= amount; accounts[to] += amount; } }
51

Homework Assignment
Common structures students will see time and again
Bounded buffer Readers-writer lock

Grade correctness and efficiency, e.g.

Maximize concurrency Only wake up as few threads as possible

Provide students with test suites

52

Many Thanks To
My advisor
Corky Cartwright

My committee members
Walid Taha David Scott Bill Scherer

NSF and Texas ATP

For providing partial funding

53