Memory Planning in Java

by Rick Grehan and Paul McLachlan

All Compuware products and services listed within are trademarks or registered trademarks of Compuware. Java and all Java-based marks are the trademarks or registered trademarks of Sun Microsystems, Inc., in the United States and other countries. All other company or product names are trademarks of their respective owners.

Memory Planning in Java

Automatic Memory Management - Why ........................................................................ 3 Automatic Memory Management - How ........................................................................ 4
Reference Counting.................................................................................................................................... 5 Future of Garbage Collectors ..................................................................................................................... 9

Problems Survive............................................................................................................... 9
Some Control: Reference Objects ............................................................................................................ 10 Pacing and Finalizers ............................................................................................................................... 11

Memory Planning............................................................................................................ 12 No Honest sizeof()............................................................................................................ 14 Coping With The Garbage Collector ............................................................................ 15
Object Pooling.......................................................................................................................................... 15 Resource Pooling ..................................................................................................................................... 16 Object Unrolling ...................................................................................................................................... 16

Special Environments ..................................................................................................... 16

The Realtime Specification for Java (RTSJ)............................................................................................ 17 KJAVA .................................................................................................................................................... 19 JavaCard................................................................................................................................................... 20

Conclusions ...................................................................................................................... 22 Appendix: Profilers ......................................................................................................... 22 Further Reading .............................................................................................................. 23

Page 2 of 23

Memory Planning in Java

Automatic memory management has always been one of Javas strong points. The concept is both simple and powerful. The Java environment says to the programmer: You create and use objects as you need them. When youre done, dont worry about them. Ill clean up after you. On the surface and even some distance beneath automatic memory management is a wonderful thing. It appears to completely eliminate some of the more subtle (and nefarious) problems that have plagued programmers in both object-oriented and procedural languages alike. It is not, however, a panacea; not even in regards to the class of problems it seeks to abolish. In keeping with one of the fundamental laws of the universe to wit, you dont get something for nothing although automatic memory management solves one group of problems, other problems rush in to fill the void. (And some of the old problems simply change clothes and come back.) These new problems can, however, be avoided by someone armed with an understanding of how Javas automatic memory management works. The goal of this white paper is to explore some of the behind-the-scenes gymnastics that the JVM employs to provide automatic memory management. This will allow us to describe some of the consequences of those gymnastics that might result in unintended consequences, and thereby arm a programmer against such consequences. In addition, the paper will example some special situations; unconventional Java environments targeted at specific applications that are gaining in popularity. These environments are special in the sense that they define memory behavior unlike what is found in the standard desktop Java environment. These special environments include: Realtime Java KVM JavaCard

As an addendum, well provide a list of profiling tools commercial and free.

Automatic Memory Management - Why

In languages such as C and C++, memory management is overt. The programmer explicitly creates an object (which allocates the memory necessary to house that object) and, when the objects lifetime has ended, must explicitly deallocate the object. By comparison, although creating an object in Java is still explicit, deallocating the object is not. One of the jobs of the Java Virtual Machine (JVM) is to automatically remove objects that are no longer in use by the application, releasing the memory they consume. This is done implicitly: there is no free operation in Java. This is referred to as automatic memory management. The programmer doesnt have to worry about managing memory, the virtual machine does all the worrying. The foremost challenge that the virtual machine faces in this regard is how to determine when an application doesnt need a given object anymore. A secondary but important challenge is how to do it efficiently, in terms of time and space used managing the memory. We will discuss some of these strategies shortly.

Page 3 of 23

Memory Planning in Java

In Java, all objects are allocated on the heap, a region of memory set aside specifically for the purpose of hosting application objects. An object that has been identified by whatever means as unneeded by the application is considered garbage, and that part of the virtual machine whose job it is to identify and clean up garbage objects is called the garbage collector. The garbage collector typically runs in a background thread, periodically scanning the heap, identifying discardable objects, and releasing the memory they occupy so that the memory is available for future objects. Garbage collection is essential for clean Object Oriented Design (see http://www.elj.com/eiffel/bm/ot-gc/) and also solves two of the biggest debugging problems that arise out of manual memory management: Dangling pointers. This happens when the application releases an object for which a reference still exists somewhere. A dangling pointer points off to nowhere. If application code attempts to use the pointer as though it actually pointed to something, untold troubles unfold. The garbage collector, which doesnt suffer from the absent-mindedness that often besets programmers, doesnt create dangling pointers. An object is garbage collected only when the collector has determined that the object has no references to it. Memory leaks. A memory leak occurs when all references (pointers) to a piece of allocated memory are overwritten, cleared, or pass out of scope. The result is that the program simply forgets about that particular piece of memory. Unfortunately, the operating environment (usually an OS) is not aware of the applications amnesia. That memory is treated by the outside world as though it still belongs to the application. The memory is therefore completely unavailable; it has leaked. (In the worst case, the memory can become unavailable to all applications in the system, even if the application that created the leak is terminated. The memory can only be reclaimed by rebooting the system.)

Javas automatic memory management scheme handles the first problem dangling pointers by the simple fact that an application never explicitly releases any object that it creates. (That is, there is no method in Java that corresponds to the C++ free() operation.) Perhaps the Java garbage collectors main duty is to handle the second problem, memory leaks. In one sense, a Java application, while executing, necessarily creates lots of memory leaks. But Javas garbage collector, executing behind the scenes, locates all the leaks (all the objects that the application has forgotten) and reclaims the memory. In an ideal situation, all leaked objects are found and their memory reclaimed. But, as well discuss shortly, the real world is as usual far from the ideal.

Automatic Memory Management - How

Having given an overview of the whys of automatic memory management, we proceed to how it is accomplished. Though our focus here is on Java, we will take occasional side roads to other languages as a mechanism of illumination by comparison.

Page 4 of 23

Memory Planning in Java

Reference Counting Reference counting is one technique for providing automatic memory management in those systems where it would ordinarily be unavailable. In a nutshell, it works like this. Once an object is created, code that wants to use that object registers that intent explicitly. (For the sake of discussion, well call this registering process opening the object.) Some component of the system it could be the object itself, or the objects runtime environment maintains a counter of those tasks that have registered an open call on the object; each open call increments the counter. This counter is associated with the object. When a task has finished with the object, the task again expresses that explicitly: it issues a close call on the object. Again, whatever component is tracking the count decrements the objects associated counter for every close call on the object. When the objects associated counter reaches zero, the object (and its memory) can be reclaimed. Suppose for example, a program creates an object call it object A and opens a reference to that object. Object A would begin life with a reference count of 1. Now suppose that the program creates object B, and code within object B obtains a reference to object A. Now, object A has a reference count of 2. When object B closes its reference count to object A, object A decrements its reference count back to 1. Once an objects reference count drops to zero, that object can be released.

Figure 1. Circular references. (a) Object A holds a reference to Object B; Object B and Object C hold references to one another (the reference count is in parenthesis). (b) If Object As reference is removed, Objects B and C still reference one another, and both will forever have a non-zero reference count, which will prevent them from being collected.

There are plenty of real life examples of reference counting. In Microsofts COM, Visual Basic hides the underlying reference counting mechanics from the developer, but in C++ you have to manually call AddRef() and Release() as you start and finish with the COM object. Some developers use advanced C++ features to create smart pointers that automatically increment & decrement reference counts as the pointers are copied and

Page 5 of 23

Memory Planning in Java

assigned. Reference counting is also used in such varied places as the Unix file system, to determine when a file with hard links can be removed. Reference counting has a relatively high performance overhead in that it must often update 2 reference counts when pointers are assigned. This causes a relatively1 constant slowdown of the program being run. Reference counting has a problem with circular references (Figure 1). Suppose an application creates three objects. As the application executes, the three objects build references to one another as shown in Figure 1a. Object A references Object B, B references C, and C holds a reference back to B. In time, all the references to A might be cleared, bringing A to a reference count of 0. In the process of de-allocation, all of the references that A holds will be cleared, namely the reference to B. Now, however, both B and C have reference counts of 1, and will not be deallocated, even though the program is not capable of using either of these objects again as all external references to them are gone. Garbage Collection A tracing garbage collector avoids reference countings problems (circular references and the performance overhead during pointer updates) by approaching the problem in a fundamentally different way. Instead of attempting to notice immediately when an object becomes eligible for collection (when there are no remaining references), a garbage collector periodically scans through memory looking for objects that have no remaining references to them. This occurs automatically at intervals or when the program gets low on available memory. Java does provide a mechanism for asking the garbage collector to execute, although no guarantees are made on whether the virtual machine will comply (we shall soon examine a few reasons that it might not). It is very important to realize that finding objects that have no remaining references to them (that the program is not able to use again because it has lost the object) is different from locating objects that will not be used again. We will revisit this distinction later. Briefly, heres how a Java garbage collector works. We have simplified some of the details somewhat for clarity. All Java applications are made up of classes and threads. These are the root objects in the system. If there is a reference from a static variable in a class to an object, that object (and all the objects that it holds a reference to) is alive. By threads, we mean the local variables of methods that are currently executing. If an object is referred to by a local variable that is still in use it can obviously not be collected. Syntactically, the application is only capable of using objects that are referred to by one (or more) of these roots. If it isnt directly referenced by a local variable or a static
Other garbage collection techniques have unpredictable (and sometimes long) delays that are rarely encountered when using reference counting because objects are freed as soon as the last reference to them is gone rather than at intervals. Such behavior does occur with reference counting when the last pointer to a large set of objects (such as a linked list) is cleared: the program will pause as the algorithm recursively clears all of the child objects.
1

Page 6 of 23

Memory Planning in Java

variable, and it isnt indirectly referenced through an intermediate object, the programmer simply has no way to use that object we say that the object isnt reachable: that there isnt a path to it from one of the roots. When the application no longer needs an object, that path is (presumably) somehow cut. Either the reference is overwritten, the method exits and the local variable goes out of scope, or one of the internal references (in the case of an object that is reachable through intervening objects) is overwritten or discarded. In any case, the object becomes unreachable by the application. The Java garbage collector follows the root references into the heap, locating all of the objects that are reachable by the application. Depending on the complexity of the applications object structure, this could be a daunting task. Root references could lead to objects, which refer to other objects, which refer to other objects, and so on. Ultimately, the collector will discover all the objects that are reachable by the program. All other object are unreachable garbage and are candidates for being eliminated so that their memory can be freed for use by future objects. Garbage collector algorithms There are several different algorithms that a garbage collector might implement: Conservative (vs exact) - A conservative garbage collector scans memory, searching for values that look like pointers to objects in the heap. Since the collector is just searching for memory that contains values that might be addresses to objects, it can be fooled into not collecting an object that it should. For example, an integer value just happens to contain the same number as the address of an object on the heap could fool the collector into thinking there was a reference to that object. A conservative collector is so named because it can overestimate the amount of live objects in the system. JDK 1.0.2 and 1.1 used a partially conservative mark and sweep garbage collector. One advantage of a conservative collector is that it doesnt rely on any support from the underlying language it can even be used to provide garbage collection in C and C++. As a side effect, back in the early versions of Java, the collector could actually detect and not free an object if it was being used by native code. In comparison, an exact garbage collector has knowledge of the layout of objects in memory and cannot confuse an int for a pointer. Mark and Sweep - A mark and sweep collector is named for the way it progresses through the collection cycle. Such a collector will scan through the heap (starting at root references) and mark all objects that are accessible from the program (live objects). It then sweeps the heap, collecting all the unmarked objects. Compacting - Compacting is another algorithm for garbage collecting. The heap is broken into two halves, fromSpace and toSpace. All objects are initially in the fromSpace. The heap is scanned (recursively from roots), and each object that is alive is moved to the start of the toSpace. Obviously, as each object is moved, the references to that object have to be changed to point to the new location. At the end of the collection, the toSpace becomes the fromSpace (ready for the next collection), and the fromSpace will be reused as the next toSpace. There are several consequences of this:

Page 7 of 23

Memory Planning in Java

At the end of the collection, the toSpace is contiguous, there isnt any fragmentation of the heap. When the program wishes to allocate more memory, it doesnt have to search for a big enough slot: the new object is just placed at to top of the toSpace. This makes allocations even faster than in manual memory management (malloc and free). Twice as much memory is used! Memory locality is improved. Processors typically cache parts of main memory that will be used during processing. As objects that refer to each other will be copied after one another, chances are that more useful objects will be pre-cached when an object is accessed. This can have a large impact on the performance of the application.

Generational It turns out that most objects that are allocated are temporary they are not kept around for very long. Temporary objects are particularly prevalent in Java, which does not permit stack-allocated objects. Objects that a C++ programmer would have placed on the stack and which would come and go as their enclosing methods are entered and exited are, in Java, allocated in heap memory. A generational garbage collector leverages this observation and scans newly created (young) objects looking for dead ones more often than scanning the old objects. Simply put, objects tend to die young, so a garbage collectors time is better spent scanning only the youngest generation of objects in search of objects to collect. Often, each generation is collected differently: the nursery (for new objects) might be collected with a compacting collector, because if only a few objects survive (as we expect), the compacting algorithm, which concentrates on objects that are alive, does less work than the mark and sweep algorithm, which concentrates on objects that are dead. As the number of young objects will typically be much smaller than the size of the entire heap, a young generation collection can be performed much faster than a full collection. This means that collections can be run more frequently (to keep memory usage down) and that they wont cause such prominent GC pauses as is occasionally seen in Java GUI applications. One negative fallout of generational collection is that it causes some old objects to take an abnormally long time to be finalized and collected. In fact, if memory never gets low, it may be that the older generation is never scanned for dead objects. An object that has no references to it might never be actually collected. This is one of the reasons the Hotspot team (Hotspot is a generational collector) keeps reminding developers not to rely on System.gc() to actually do anything useful the object the developer is trying to get collected might never be, regardless of how many collections are run. Garbage collector properties Stopping - A stopping garbage collector pauses the entire program in order to scan the heap, thereby preventing other threads from changing references while the collector is doing its work. This provides the collector with a consistent view of the heap throughout the collection process. While simple to implement (at least, simpler than the alternatives) a stopping garbage collector can be very bad on a multiprocessor system. (The collector has to stop everything.)

Page 8 of 23

Memory Planning in Java

Incremental - An incremental garbage collector does its collecting work in increments. The collector interleaves its work with the execution of the program. This requires the collector to perform its work in such a way that each increment leaves the heap in a state that is usable by applications. Incremental garbage collectors prevent long pauses in program response as the system waits for the collector to finish. (The Hotspot virtual machines garbage collector can optionally operate as a incremental collector. The command-line switch -Xincgc enables the incremental collector.) Concurrent - A concurrent garbage collector allows the program to run concurrently with garbage collection operation. (Authors note: Microsofts .NET appears to use a compacting, generational, stopping garbage collector.) Future of Garbage Collectors In the future, tighter coupling between JITs and garbage collectors could lead to improvements in even the above algorithms. One such improvement is called escape analysis. A system that employs escape analysis can automatically detect objects that will never live beyond their allocating method (i.e., that do not escape the method). For example, a StringBuffer object that is allocated to build up a string that will be returned by toString() is never stored in a class variable or passed to another method as an object it is completely local to the method that allocates it. In other languages (e.g., C++ and to some extent .NET), the programmer could manually cause those objects to be allocated on the local variable stack, just like any other primitive. However, there is no reason that the JIT could not also determine that the object will never be used outside the method and also place the object on the stack. Escape analysis is attractive for a number of reasons. If objects are allocated on the stack the garbage collector does not have to spend its time collecting them. In addition, stack allocations and frees are much more efficient than using heap memory, usually as simple as an addition to or subtraction from a stack register. In the white paper Escape Analysis for Java (Jong-Deok Choi et al, IBM T.J. Watson Research Center, Nov. 1999) the authors present an algorithm for escape analysis in Java, and ran a number of benchmark programs to test the validity of the scheme. They report that, as part of their tests, 70% of the objects in 3 of the 10 benchmark programs they ran were candidates for allocation on the stack. (The median for all benchmark programs was 19%.) (Authors note: The realtime Java specification as advanced by the JConsortium defines the stackable keyword, which identifies objects to be created on the stack)

Problems Survive
Having given an introduction to garbage collectors, we might have left you with the idea that the automatic memory management that garbage collectors provide cures all memory-related programming ills. Unhappily, this is not the case.

Page 9 of 23

Memory Planning in Java

Amazingly, even though an exact garbage collector will locate and deallocate all objects that cannot be used again by the application, memory leaks still occur. The reason this happens has to do with the distinction between an object that cannot be used again, and an object that will not be used again. An object of the latter sort is still alive as far as the garbage collector is concerned (there are still references to it somewhere), but will remain unused by the application for the remainder of that applications lifetime. Strictly speaking, the object has been leaked. Only, in this case, it isnt the application that has forgotten the object, its the programmer. This forgetfulness is not as difficult to achieve as you might imagine. An object can be leaked by being placed in a container object at one point of the code, and ... well ... forgotten. There is no explicit reference to the object that the programmer will see in the source code; the reference is inside the container. From the garbage collectors perspective, however, the object is still alive, and will not be reclaimed. It becomes a classic out of sight, out of mind situation. And, as data structures within an application become more complex, the probability that objects will be leaked rises. At the extreme, leaked objects will cause the same calamity as before: the application will run out of memory. In such cases, a profiler is invaluable. A good profiler will allow the developer to mark a point in time (such as while waiting for a HTTP connection) and watch objects that get allocated (and not freed) after this point. If you run your program through a few operations, and then let it return to being idle, the profiler will be able to display the set of surviving objects those objects that the program created that were not freed. You can then weed through to determine which objects were intentionally created and which were accidentally leaked.

Some Control: Reference Objects

Javas garbage collector does its work under the covers. And the fact that the work is done covertly is a two-edged sword. On the one edge, the programmer is relieved from a great deal of explicit effort (keeping track of when an object becomes a candidate for collection). On the other edge, there is a class of problems that require more interaction with the memory management system than simply calling new. For instance, a caching mechanism might want to keep some objects in memory only until memory starts getting low at which point it would like to be able to move some of the objects off onto disk until they are needed again. Javas reference API (java.lang.ref) addresses this need. A reference is a proxy object that references a target object (called the referent) in such a way that the reference does not necessarily keep the object alive. That is, if the only references to an object are all soft, the garbage collector can decide if it needs to collect the memory or not. In practice, the proxy object has a get() method that will either return a reference to the target object, or NULL if the object has already been collected. There are three flavors of reference objects: soft, weak, and phantom. Soft references A soft reference to an object is a reference that will only be cleared by the garbage collector if it is low on memory. Obviously, if there are still normal references to the object, the object cannot be cleared.

Page 10 of 23

Memory Planning in Java

Weak references A weak reference to an object is a reference that doesnt stop the object from being collected. That is, it allows you to hold a reference that will just get set to NULL when there are no remaining normal (or soft) references to that object. Phantom references The other two references are both cleared before the object is finalized (meaning, in theory, that the object is actually still around and might be resurrected). Phantom references provide a way to be notified after an object has actually been finalized and there can never be any more references to it. The get() method of a phantom reference will always return NULL the only use for this reference is to get a callback after the object is finalized. This allows you to provide more robust cleanup than the finalize() method. Using finalize() is a poor way to do cleanup for (at the moment) two reasons: 1) A subclass can override finalize() and fail to call super.finalize(). 2) A finalize() method has access to the this reference and can resurrect an object by storing a reference to it in (for instance) a static variable, making that object reachable again. While the VM has a defined response to this, and while it is remarkably poor programming practice, it is nonetheless possible for a malicious, over-tired, or under-studied programmer to do. Such open doors are invariably walked through.

Pacing and Finalizers

In some cases, an application might be allocating temporary objects at such a rate that the garbage collector cannot collect the discarded objects quickly enough to make memory available for the allocation requests. This is a matter of pacing: the application is outrunning the garbage collector. It can happen with incremental garbage collectors and is more likely to occur when the application is under load. (The result, of course, is an OutOfMemoryError exception something that is very hard to recover from gracefully.) Fortunately, garbage collectors generally fall back to stop the world when memory actually runs out and do a full collection. In most cases, that will avert the out-of-memory catastrophe. But, as usual, the real world is often more complicated than wed like it to be. One example is that finalization creates a backlog of objects to be freed. Heres how: Javas finalization is implemented using weak references (see above). Once an object can be collected, that objects finalize() method can be run, and must be run before collection can take place. However, as described earlier, a malicious finalize() method can resurrect the object by storing this in another object. Hence, the garbage collector may have to perform as many as 3 passes on the object. The first scan identifies that the object is unreachable, the second pass in a finalization thread to run the finalize method, and a third pass is needed to confirm the object is still unreachable and to actually collect it. In addition, because the objects finalize() method is called from a Java thread (the finalizer thread), the objects finalize() method might not get scheduled quickly enough to help the garbage collector reclaim that objects memory. In the worst case, the application may have threads running at a higher priority than the finalizer thread. Page 11 of 23

Memory Planning in Java

It gets better (or, worse, depending on the strength of your sense of humor). Remember, generational garbage collectors dont scan all objects with each collection pass. Such collectors will typically only examine the young generation of objects. An old object might be waiting a very long time to get collected; perhaps, if you never get low on memory, it might wait forever. If that objects finalize() method actually does something like closing a connection or a file the program could run out of system resources (file descriptors, native graphics contexts, database connections) long before it runs out of memory. Cleanup of resources should always be done explicitly, not by relying upon automatic memory management to take care of it for you.

Memory Planning
There is an axiom that must be stated prior to discussing memory planning in Java. Put succinctly, that axiom might read: In Java, things are almost always bigger than they seem. Certainly, Java objects and arrays are larger than their equivalents in, say, C++. But, for certain Java data structures, there is additional (and hidden) overhead, which we will illustrate presently. If your application does make use of the Java Collections API, the single most important factor influencing your applications memory scalability is the choice of data structure you make for a particular problem. All collection objects grow more or less linearly, though the rates of growth vary. (See figure 2.) This linear growth is true for memory consumed as well as for the number of objects created. (Of course, you must weigh memory use against performance. How objects in a collection are to be accessed should weigh in your choice of collection object. For example, if the primary access is sequential, choose a linked list. If the primary access is random via a unique key, choose a HashMap. Etc.)

Arrays and Collections

45000 40000 35000 30000 25000 20000 15000 10000 5000 0 1DArray 2DArray1 2DArray2 Link e dLis t ArrayList Has hSe t Tre e Se t

Actual number of objects

4000

8000

20000

Number of objects requested

12000

16000

Page 12 of 23

Memory Planning in Java

Figure 2. Growth rates for several data structure.2DArray1 starts with an array of [2][1000] and increases the first index. 2DArray2 starts with an array of [1000][2] and increases the first index (note that it grows faster than 2Darray1).

Though growth for all collection objects is generally linear, note that HashTables, HashMaps, and HashSets all use the same underlying hashing mechanism, and grow in spurts (see figure 3). The growth amount depends on the tables initial size. At each growth spurt, the hash table roughly doubles in size (it grows to the next prime number greater than twice the tables current size).

Figure 3. Memory behavior of a HashSet. The lower (red) line in the graph shows the available memory. Notice that, for most of the programs execution, the line shows a linear descent (as new objects are added to the HashSet). However, at certain points, the line drops suddenly. These are the points at which the hash table is grown. Points at which the line rises suddenly indicate execution of the garbage collector. (The graph was generated using the MemoryMonitor application from Java Performance Tuning by Jack Shirazi, OReilly & Associates, 2000.)

Also, some collections grow by creating bigger versions of themselves, copying their current contents into the new version, then deleting the old. (ArrayList is like this) This can produce a memory consumption pattern with the creation of lots of objects followed by the destruction of lots of objects. Of course, the old object is not explicitly deleted: this is Java, after all. The collection forgets about it and it is left up to the garbage collector to reclaim the space. (Such collection objects can also cause your application to experience spurts in performance. A request to put an object in the collection will take an unusually long time to return because that request has triggered the create-copy-delete sequence.) A general recommendation for using collection objects is to pre-size the collection object when you create it so that it is slightly larger than what you anticipate it needs to be. This will eliminate the problems that occur when the collection object must grow.

Page 13 of 23

Memory Planning in Java

No Honest sizeof()
The C/C++ programming language has a sizeof() macro, which returns the size in bytes of its argument. The argument of sizeof() can be a datatype, a structure, a variable, or an object. The sizeof() macro is handy for calculating how much memory a given entity will consume. Java, however, has no sizeof() method. We can, however, use a profiler and the JVMPI (Java Virtual Machine Profiling Interface) as though Java did have sizeof() to determine the amount of space occupied by various objects.: sizeof( new Object() ) = 8 bytes sizeof( new int[1] ) = 16 bytes sizeof( new byte[1000] ) = 1016 bytes sizeof( new char[1000] ) = 2016 bytes We can use this to reveal the minimum number of bytes of overhead that will be consumed by an object (8 bytes) or a one-dimensional array (16 bytes, regardless of whats in the array). We can also use this to discover something that we really knew all along. Specifically, that arrays are objects, too: sizeof( new int[2][100] ) = 214 words (3 objects) sizeof( new int[100][2] ) = 704 words (101 objects) Remember that, in Java, a multi-dimensional array consists of arrays of internal pointers. So, a [2][100] array is actually a one-dimensional array consisting of two references, each reference pointing to a one-dimensional array of 100 elements. Meanwhile, a [100][2] array is a one-dimensional array consisting of 100 references, each reference pointing to a one-dimensional array of 2 elements. In terms of actual memory consumed (and whether you should be worried about it), this is probably only a big deal if you have lots of small, multi-dimensional arrays. In such cases, the ratio of internal references to actual data could be unacceptably high. Finally, using a profiler and JVMPI we can re-discover that finalization in Java is implemented with weak references. So, given the following code: class FinalEg { public void finalize() {} } For this code, sizeof(FinalEg) is 40 bytes (over several objects), not 8! (Which is perhaps yet one more reason not to use finalize().)

Page 14 of 23

Memory Planning in Java

Coping With The Garbage Collector

Now that we understand some of the limitations of Javas garbage collector, as well as a few of the more peculiar and unsavory aspects of Javas architecture (and how it effects memory use), we should take a brief look at techniques employed by programmers to avoid the resulting difficulties.

Object Pooling
The goal of object pooling is to minimize object creation and garbage collection costs. The technique is straightforward, and a close cousin of object pooling is often used in database applications to manage connections. (Though, in database connections, connection pooling as it is called is used to minimize the amount of time needed to create a new connection to the database.) Object pooling consists of pre-instantiating a set of objects, placing those objects in a pool usually some sort of container object and having the application draw the prebuilt objects out of the pool as they are needed. When the applications use of an object is finished, the object is returned to the pool. Pooling is best used when an application needs a large, but fixed, number of similar objects. Or, in situations where the creation or elimination of such objects is timeconsuming, and the developer would rather the application incur those costs up front, rather than have them recur at execution time. The advantage of this scheme is that it eliminates the repeated object creation and destruction costs (and the associated cost of garbage collecting destroyed objects). Objects are neither created nor destroyed (or, at least, they are created once at application startup, and destroyed once at application shutdown). Objects are borrowed from the pool when needed, and returned to the pool when no longer needed. The disadvantage is that, since drawing an object from and returning that object to the pool is a replacement for creating and discarding that object, what was once implicit (the discarding and garbage collecting) is now explicit (returning the object to the pool). Code which corresponded to the finalizer must now be associated with whatever method is used to return an object to the pool. And this code must be carefully crafted so as to avoid leaks. Because an object in the pool is still live, if it holds references to other objects, those other objects will not be garbage collected when the object is returned to the pool if the application doesnt remove those references from the object before putting it back in the pool. This is equivalent to the problem cited earlier, in which the program deposits an object in a collection and forgets about it. For the most part, however, object pooling for the purpose of managing object creation and garbage collection costs is unnecessary if youre using a generational garbage collector such as is found in the Hotspot VM. The Hotspot VMs collector is faster at clearing out objects than any Java code you can write. In addition, relying on the VM to manage the reclamation of objects permits the developer to write cleaner, less-cluttered code. The methods for properly managing the pool withdrawing and returning objects are unneeded.

Page 15 of 23

Memory Planning in Java

Resource Pooling
While object pooling as described above is unnecessary in a VM such as Hotspot, resource pooling is another matter. There may be instances when pooling objects for the purpose of managing a resource other than memory is worthwhile. For example, Hotspot cannot optimize the time spent executing an objects constructor. And some objects are expensive to construct and initialize: e.g., threads and database connections. In such cases, it may make sense to re-use objects via object pooling. Nevertheless, pooling can obscure your program logic. You should always locate performance problems by profiling first; you may discover that there are alternatives to object pooling.

Object Unrolling
Object unrolling makes use of the fact that objects consume more memory than do primitive values. (Remember the overhead we discovered with our hypothetical sizeof() operator?) If your application uses lots of objects from a particular class that can be represented instead by the primitive values of which the object is composed, you can save space by replacing occurrences of the object with the primitive values. For example, suppose you have an application that involved 2D graphics. (Were going to assume here that the application employs presentation graphics. In other words, coordinates can be represented as integers rather than floating-point values.) During execution, your application creates large numbers of rectangles, polygons, etc. Youve chosen the java.awt.Point object to store (x,y) coordinates that represent the vertices of the 2D objects in your system. You can save some space by replacing occurrences of the java.awt.Point object with individual x and y integer primitives. To put this in perspective, an array of 1000 java.awt.Point objects will consume just over 20000 bytes. However, if you replace those objects with a pair of onedimensional integer arraysint x[1000] and int y[1000] the result takes just over 8000 bytes. The technique of object unrolling works best if the object being unrolled is a simple data structure. Unrolling more complex objects leads to code obfuscation. Furthermore, unrolling an object requires that you sever the connection between an objects data and its methods, unwinding one of the fundamental fabrics of object oriented programming. Consequently, you should (as mentioned above) carefully profile your application to determine where the memory is being consumed before hacking through your code, unrolling objects. In other words, you should convince yourself that the memory savings youll achieve is worth ruining your code with this (or any other) optimization.

Special Environments
Some Java environments dont look at all like what youll find on the desktop. These environments either involve limited memory, or implement peculiar memory types designed with specific applications in mind. These peculiarities affect object lifecycles, and therefore how the programmer should approach memory management.

Page 16 of 23

Memory Planning in Java

Well look at three such environments: Realtime Java as described in the RTSJ (and as distinct from the realtime Java specified by the JConsortium) KJAVA (and the KVM), which is a kind of interim CLDC environment (and implementation which exists for the PalmOS). JavaCard, the ultra-small-footprint Java environment for smartcard applications.

The Realtime Specification for Java (RTSJ)

Realtime systems are almost always employed to interact with and control external hardware (e.g., valves, wheels, sensors, etc.). Realtime systems differ from desktop systems in that a realtime system has timeliness requirements that desktop systems do not. If a realtime system fails to carry out a function, or respond to an event, within a specific amount of time, the realtime system is not just late, its wrong (in the same way that a calculator application in a desktop system is wrong if it adds two and two and gets six). Realtime systems are usually grouped into two categories: hard realtime and soft realtime. A hard realtime system (which the RTSJ seeks to address) involves timeliness requirements that are rigid. Within a hard realtime systems are deadlines i.e., task A must be run to completion once every X milliseconds the system must meet. The correctness of the system is tied to the timeliness of the response. A soft realtime system does have timeliness requirements, but they are not as strict as with a hard realtime system. A soft realtime system can accept a certain number of missed deadlines within a given period of time; precisely how many misses, how great the latency of the miss, and over what period of time depends on the application. In any case, we are interested in hard realtime systems, because the RTSJ has been crafted to address hard realtime applications. Obviously, for a designer to have any hope of creating a hard realtime system, the behavior of the system must be predictable; that is, the system must be deterministic. Without going into great detail, this requirement usually boils down to requirements placed on how the systems task scheduling and dispatching components work, and those components unflagging adherence to a well-defined behavior. In particular, they must work predictably, so that the designer can ascertain whether a given workload (combination of tasks) will satisfy an applications correctness requirements. Hard realtime life with a garbage collector is tough. A hard realtime system fails if determinism is thwarted; that is, if a thread thats scheduled to execute at a particular point in time is delayed past its deadline. Couple that fact with the fact that the garbage collector thread cannot guarantee when it will stop once it starts (and that might hold up one or more application threads waiting on memory requests to complete), and you have a recipe for incompatibility. Incompatibility, that is, between Java and hard realtime. Enter the RTSJ, which offers solutions to this problem; solutions that take two forms. NoHeapRealtimeThreads First, RTSJ defines a particular kind of thread, defined by the NoHeapRealtimeThread class. Objects of this thread class are otherwise like Page 17 of 23

Memory Planning in Java

ordinary threads, with the exception that they make a solemn vow (which the system forces them to keep) that they will never access an object that is on the Java heap. NoHeapRealtimeThread objects cannot break this vow because they run at a priority higher than the garbage collector. This gives them the ability to behave predictably; they cannot be blocked by the garbage collector. As a consequence, NoHeapRealtimeThread objects can access only stack-based primitives or objects stored in a special sort of memory, ScopedMemory (which we will describe below). Because a NoHeapRealtimeThread has access only to local objects, there appears, to be no way for normal threads and NoHeapRealtimeThread objects to communicate and synchronize. (Were some sort of synchronization object defined, NoHeapRealtimeThread objects and normal threads could not share access to the object. Allowing that would impose the impossible requirement that a normal thread have its priority raised above that of the garbage collectors so as to avoid priority inversion, a variant on deadlock.) The RTSJs solution to this is the creation of several non-blocking queues. The Java runtime guarantees that the access to the queue is synchronized in such a way that the queues contents are never invalid. Threads on either end of the queue can read from it, write to it, or poll it for the arrival of contents. (Note: The RTSJ also defines a RealTimeThread class, which does not preempt the garbage collector. A RealTimeThread must wait for a safe preemption point. The GarbageCollector class in the RTSJ provides a getPreemptionLatency() method, which allows a RealTimeThread to, in effect, ask the garbage collector: How long will I have to wait before you are preemptible?) Memory Types The RTSJs other solution to the incompatibility between hard realtime and Java comes in the form of new memory types whose characteristics are engineered to tackle specific problems in realtime programming. These memory types are implemented via a set of classes, and the instantiation of an object from one of the classes creates a memory area. How objects behave (and are treated) within a specific memory area varies depending on the memory type. ImmortalMemory - A realtime Java environment can have only one instance of the ImmortalMemory class. As its name implies, ImmortalMemory never dies; that is at least true from the applications point of view. Once an instance of the ImmortalMemory class is instantiated, it exists for the remainder of the life of the application. Similarly, once an object is instantiated in ImmortalMemory, it lives there for the remainder of the life of the program. The garbage collector never treads on ImmortalMemory. This means that the programmer must do up-front planning with regards to what objects will reside in ImmortalMemory (and, therefore, how much ImmortalMemory should be allocated). An object in ImmortalMemory to which all references are lost is truly leaked, the garbage collector will never reclaim it.

Page 18 of 23

Memory Planning in Java

ScopedMemory - A ScopedMemory region is a block of memory that is allocated by a thread (usually a NoHeapRealtimeThread), and exists for as long as the scope within which the memory object was created exists. In a way, it is a sort of temporary heap memory with a couple of important differences. Once a ScopedMemory region is created (entered), execution of the new operator allocates objects in that scope (rather than on the heap). When the scope is exited, its contents are garbage collected en masse. In addition, during the lifetime of the ScopedMemory object, the garbage collector never scans its contents. This places some restrictions on references involving ScopedMemory : 1. Objects in scoped memory can only reference objects in an outer (enclosing) scope. This is reasonable, because if references into an inner scope were allowed, then the moment that inner scope was exited, all the references would suddenly become dangling pointers. 2. Objects in scoped memory cannot be referenced by objects on the heap. 3. Objects in scoped memory cannot be referenced by objects in ImmortalMemory. Restrictions (2) and (3) follow from the explanation given in (1). Namely, scoped memory cannot be reached by references that may exist past the termination of the scope, or those references will become dangling pointers. In sum, the ScopedMemory class provides functionality similar to what would be available with objects allocated on the stack. A ScopedMemory region is an ideal place for a NoHeapRealtimeThread to create objects it needs for the duration of its execution, because the ScopedMemory region is unaffected by the garbage collector. PhysicalMemory - The PhysicalMemory class permits the creation of objects that have direct access to memory. This class provides accessor methods that allow a Java application to read and write Java datatypes directly to physical memory. Obviously, the PhysicalMemory class was designed with embedded applications in mind, as it provides for access to memory-mapped devices from a Java application (without that application having to resort to JNI). The RTSJ meets hard realtime needs by providing special thread and memory classes. These additional components of RTSJ seek in one way or another to skirt the undesirable influence of Javas garbage collector. A NoHeapRealtimeThread avoids the garbage collector by being restricted from interfering with the Java heap. ScopedMemory and ImmortalMemory classes provide memory regions the garbage collector promises never to visit.

KJAVA
KJAVA is the implementation of a CLDC (connected limited device configuration one of the specifications within J2ME) atop the K virtual machine (KVM). In fact, it was the KJAVA virtual machine project that gave KVM its name. (KVM is described as the minimal Java virtual machine allowed for a J2ME application.)

Page 19 of 23

Memory Planning in Java

KJAVA includes an interim graphical interface provided by Sun for the CLDC. It is interim because the CLDC doesnt define a graphical interface, thats left up to the profile, which as of this writing has yet to be defined for handheld devices. Nevertheless, there are a number of freeware and commercial products that have appeared and that implement KJAVA and its associated GUI. The first (and, as of this writing, only) handheld implementation of the KVM runs on the PalmOS. Obviously, there isnt a lot of memory on Palm devices, and what memory there is, is shared between storing the applications and executing them. That is, there is no distinction made between primary storage RAM and secondary storage hard disk and CD-ROM as on a desktop system. For example, a Palm IIIx has 4MB of useable memory. While some PalmOS devices have more memory (the Palm Vx has 8MB of memory, and the newer Palm m500-series devices include memory expansion slots) others have less (the m100 has only 2MB of memory). Dealing with limited memory, as well as limited computing power (the Motorola Dragonball processor in most Palm devices has nowhere near the power of a 700 MHz Pentium) presents special challenges to a Java programmer. And the original KVM garbage collector was non-moving, so a programmer had to be especially diligent memory fragmentation was likely. Fortunately, the most recent edition of the KVM (as of this writing) included an exact, compacting garbage collector. Nevertheless, the KVM is definitely not Hotspot; there is no JIT, the garbage collector is not generational, etc. Consequently, all the strategies that programmers had employed to minimize the effects of the garbage collector, all of those optimizations that we talked about earlier that ruined your code and that Hotspot makes mostly unnecessary object unrolling, object pooling, etc. are worth investigating in KVM. Another unfortunate characteristic of the KVM (as it stands now), is its lack of a profiling API. The KVM does support a debugger interface, but that is unusable in gathering memory statistics for a particular application. In general, the programmer must resort to the old trick of peppering code with output statements that periodically display the amount of memory still available and trying to deduce behavior from that.

JavaCard
If you thought memory was Spartan on a handheld device, even less memory is available to Java applications running in a JavaCard environment. To give you an idea of how restrictive a JavaCard can be, the specification (as of JavaCard 2.0) set a minimum of 16KB of ROM, 8KB of EEPROM, and wait for it 256 bytes of RAM. These are cramped spaces, indeed. To make matters more interesting, the JavaCard VM is not required to even have a garbage collector. Programmers are advised to write their code in anticipation of a garbage collectors absence. From the specification: Application programmers cannot assume that objects that are allocated are ever deallocated. Storage for unreachable objects will not necessarily be reclaimed. This certainly simplifies having to deal with the garbage collector; theres no sense worrying about a thing which is not there.

Page 20 of 23

Memory Planning in Java

The JavaCard environment expects long-lived applications: the JavaCards virtual machine runs forever. Even when the card is not in use, the JVM is said to be executing with an infinite clock cycle. Most JavaCard applets are expected to reside on the card for its lifetime. Consequently, most objects on a JavaCard are persistent objects. They exist in EEPROM. An application should create all the objects it needs for its lifetime at constructor time. JavaCard does have a provision for transient objects; specifically, transient arrays. JavaCards framework JCSystem class includes methods for building transient arrays of booleans, bytes, shorts, and objects. The specification is clear (and firm) that such transient arrays never be stored in a persistent memory technology. Consequently, transient arrays are not designed to provide any kind of space savingsthey are consuming precious RAM space. Transient objects are, in a sense, garbage collected. More specifically, their fields are cleared, and this is done at one of two times (specified when the transient array is created): Cleared on DESELECT - The transient array is cleared when the applet that created the array is deselected. (Authors note: This warrants a brief explanation. On a JavaCard, only one applet can be executing at a given time; this is known as the selected applet. The selected applet defines a context, and the architecture of the JavaCard environment is such thatunless explicitly accounted for otherwisean applet can only access objects within its own context. Applets can share objects, but that must be done explicitly.) Cleared on RESET - The transient arrays contents are maintained even if the applet that created it is deselected. However, the array is cleared if the card is reset.

Security is a critical concern on JavaCard systems, which accounts for the strict controls placed on transient arrays. For example, a transient array that is cleared on RESET would be useful for storing a session key (a unique identifier for the current sessions established when the card is placed in a card reader). Multiple applets might execute during that session, in which case the session key would be passed among the applets. However, once the card was removed from the reader, and RESET, the session key would be cleared, making it unavailable to applets that execute within the next session (the next time the card is placed in a reader) To sum up, JavaCard applications have minimal complexity; most perform a single function. Programmers are forced to manage memory resources using much the same techniques employed on 8-bit processor applications written in machine language. Any concerns about the performance of the garbage collector are needless hand wringing, owing to the unlikelihood that a collector even exists. Memory is so scarce and data structures so simple one wonders sometimes why a language that supports objects is used at all. We suppose this allows the Java subset implemented on JavaCards to maintain some sort of similarity with its larger cousins on handhelds and on the desktop.

Page 21 of 23

Memory Planning in Java

Conclusions
Javas automatic memory management is a garden that grows flowers side-by-side with brambles. On the one hand, the programmer need not trouble with explicit object deallocation. On the other hand, the programmer cannot simply ignore memory issues. The key to making your programs optimal is a good understanding of what occurs backstage. Here are a few nuggets of (we hope) useful information: Dont use finalize(). If you have to, use PhantomReference instead...it isnt less efficient. As weve shown, the latter technique is less likely to cause difficult-to-locate problems. There are techniques you can use to decrease memory overhead e.g., object pooling and object unrolling. However, the capabilities of the desktop VMs are such now that the benefits to modifying your code to squeeze whatever extra memory from it you can will be offset by code obfuscation. You will gain far more mileage by commonsense activities such as proper selection of collection objects, copious use of a profiler to locate potential leaked objects, and so on. Find a good profiler that youre happy with and use it a lot (see preceding point). On the other hand, environments do exist Realtime Java, KJAVA, and JavaCard where unusual measures must be taken in order to manage memory explicitly. Understanding what the particular virtual machine is doing (and why its doing it) is the key to fine-tuning your code for the particular application.

Finally, garbage collectors are getting better. As weve already mentioned, the HotSpot garbage collector provides incremental collection capabilities. Its likely we will soon see pacing controls within virtual machines, so that large applications can adjust the operation of the garbage collector as memory loads ebb and flow. New handheld Java VMs are appearing that provide advanced garbage collection and, amazingly, even JITs! One day, even allocating fewer objects might not be worth the effort. And, while you wait for that day to come, try not to ruin your code!

Appendix: Profilers
There are several profilers available free as well as commercial. Most employ the JVMPI, although at least one requires a special (proprietary) variant of the JVM. Free profilers include: hprof - This profiler is provided with the current JDK, and can be executed by running java -Xrunhprof. The output of hprof is a large, not-easily-interpreted text file. Some tools are available that parse the output of hprof, and present the information mined from it either tabular or graphical form. jprof The jprof profiler is available from http://starship.python.net/crew/garyp/jProf.htm. The output of jprof is reasonably decent. It can tell you, for example, at what point in time a given Page 22 of 23

Memory Planning in Java

object was accessed, how many instances of an objects class there were at the conclusion of the application, the peak and total instance count during an applications execution, and so on. The most recent version we were able to find is dated June, 1999. The future of jprof is, consequently, uncertain. JUM (Java Usage Monitor) This tool appears to be more of a library for resource monitoring, and not precisely a profiler. It reports a threads CPU time and memory usage. JUM is available from http://www.iro.umontreal.ca/~lolouarn/jum.html. Jinsight The Jinsight tool only runs on a custom version of the 1.1.8 JDK. It actually consists of two pieces: a trace library that you link into your code, and a visualizer with which you explore the data. Its output is quite impressive, though its peculiar JVM requirements limit its usefulness. Jinsight is available from http://www.alphaworks.ibm.com/tech/jinsight. There are at least three commercial profiling tools of note. They are: NuMega DevPartner Java Edition Available from Compuware. See http://www.numega.com/products/java.shtml. JProbe Available from Sitraka Software. See http://www.sitraka.com. OptimizeIt Available from Intuitive Systems. See http://www.optimizeit.com.

Rick Grehan is a Technical Evangelist for the NuMega product line of Compuware Corporation Paul McLachlan is a Software Engineer for the NuMega product line of Compuware Corporation

Further Reading
Java Platform Performance: Strategies and Tactics. Easily the best Java performance book available, it covers a broad range of safe memory optimization topics. Steve Wilson, Jeff Kesselman.. Published in 2000 by Addison-Wesley. ISBN 0-201-70969-4 Garbage Collection: Algorithms for Automatic Dynamic Memory Management. A comprehensive summary of the state of garbage collection research. Richard Jones, Raphael Lins. Published in 1996 by John Wiley & Sons Ltd. ISBN 0-471-94148-4

Page 23 of 23