Memory Cache Optmization

Key Performance Ideas Knowledge Base
Memory Cache Optimization
Description Application Version Submitted By Date
Memory Cache Optimization Essbase 6x+ Rob Redford 11/12/07
This document is the property of Key Performance Ideas, Inc. and is made available to personnel with the understanding that it shall not be duplicated, reprinted, or disclosed to any third party, either in whole or in part.

Essbase uses five memory caches to coordinate memory usage:
1. Index Cache is a buffer in memory that holds index pages. How many index pages are in memory at one time depends upon the amount of memory allocated to the cache. 2. Data File Cache is a buffer in memory that holds compressed data files (.PAG files). Essbase allocates memory to the data file cache during data load, calculation, and retrieval operations, as needed. The data file cache is used "only" when direct I/O is in effect. 3. Data Cache is a buffer in memory that holds uncompressed data blocks. Essbase allocates memory to the data cache during data load, calculation, and retrieval operations, as needed. 4. Calculator Cache is a buffer in memory that Essbase uses to create and track data blocks during calculation operations. 5. Dynamic Calculator Cache is a buffer in memory that Essbase uses to store all of the blocks needed for a calculation of a Dynamic Calc member in a dense dimension (for example, for a query).
Essbase provides default size settings for each cache. You can adjust the size of any of these five caches as needed for each database. Appropriate cache size is affected by many factors, including database size; block size, index size, and available memory on the server. Cache size settings can affect database and general server performance significantly.
Memory Cache Optimization Sizing the Index Cache
The index is stored in index files on disk. When a database is active, the most recently accessed index pages are held in the index cache, which is a memory area that is allocated for index pages. How much of the index can be held in memory at one time depends upon the amount of memory you allocate to the index cache.
When a data block is requested, Essbase looks at the index pages in the index cache to find its location on disk. If the block location is not found in index pages in the index cache, the index page containing the block location is pulled into the index cache from disk. If the index cache is full, the least recently used index page in the cache is dropped to make room for the index page containing the location of the data block.
The effectiveness of the index cache size depends on the nature of the calculation you are performing. For example, if you were reloading and recalculating an entire database (such as a database that is refreshed each month), a high index cache size is not helpful because Essbase is creating new blocks rather than searching the index cache for existing blocks during calculation.
Recommended Value: Combined size of all ESSn.IND files, if possible; as large as possible otherwise. Do not set this cache size higher than the total index size, as no performance improvement results.
Minimum value: Default Value:
1 MB (1048576 bytes) "Buffered I/O" "Direct I/O" 1 MB (1048576 bytes)
10 MB (10485760 bytes)
Memory Cache Optimization Sizing the Data File Cache
The data file cache holds data files (.PAG files) in memory, if you are using direct I/O. If you are not using direct I/O, the data file cache is not used. How much of the data within data files can fit into memory at one time depends on the amount of memory you allocate to the data file cache.
In general, if you have to choose whether to allocate memory to the data cache or to the data file cache, choose the data file cache if you are using direct I/O.
Recommended Value: Combined size of all ESSn.PAG files, if possible; otherwise as large as possible.
This cache setting not used if Essbase is set to use buffered I/O.
Minimum value: Default Value:
"Direct I/O"
10 MB (10485760 bytes) 32 MB (33554432 bytes)
"Direct I/O"
Memory Cache Optimization Sizing the Data Cache
Data blocks reside on physical disk and in memory. The data cache is the memory area that is allocated to hold uncompressed data blocks. The number of blocks that can be held in the data cache at one time depends on the amount of memory you allocate to the data cache.
When a block is requested, Essbase searches the data cache for the block. If Essbase finds the block in the cache, it is accessed immediately. If the block is not found in the cache, Essbase searches the index for the appropriate block number and then uses the index entry of the block to retrieve it from the appropriate data file on disk. Retrieving a requested block from the data cache is faster, and therefore improves performance.
In general, if you have to choose whether to allocate memory to the data cache or to the data file cache, choose the data file cache if you are using direct I/O.
Recommended Value: 0.125 * the value of data file cache size. Increase value if either of these conditions exists:
Many concurrent users are accessing different data blocks. Calc scripts contain functions on sparse ranges, and the functions require all members of a range to be in memory (for example, when using @RANK and @RANGE) For data load, the number of threads specified by the DLTHREADSWRITE setting is very high and the expanded block size is large.
Minimum Value:
3 MB (3145728 bytes)
Memory Cache Optimization Default Value: 3 MB (3145728 bytes)
Testing Cache Settings
You can check Essbase performance in several ways. One way is to check the Runtime page of the Database Information dialog box. A second method is to click the Statistics tab on the Database Properties window in Administration Services.
To check values in the Run-time page of the Database Information dialog box, select Database > Information from the Application Manager menu, and check the hit ratios:
The cache hit ratio indicates the percentage of time that a requested piece of information is already in the cache. A higher hit ratio indicates that the data is in the cache more often. This improves performance because the requested data does not have to be retrieved from disk for the next process. A hit ratio of 1.0 indicates that every time data is requested, it is found in the cache. This is the maximum performance possible from a cache setting. The Hit Ratio on Index Cache setting indicates the Essbase Kernel s success rate in locating index information in the index cache without having to retrieve another index page from disk. The Hit Ratio on Data File Cache setting indicates the Essbase Kernel s success rate in locating data file pages in the data file cache without having to retrieve the data file from disk. The Hit Ratio on Data Cache setting indicates Essbase s success rate in locating data blocks in the data cache without having to retrieve the block from the data file cache. Check memory allocation. Add smaller amounts of memory at a time, if needed, because a smaller increment may have the same benefit as a large one. Large, incremental allocations of memory usually result in very little gain in the hit ratio.
Memory Cache Optimization Sizing the Calculator Cache
Essbase can create a bitmap, whose size is controlled by the size of the calculator cache, to record and track data blocks during a calculation. Determining which blocks exist using the bitmap is faster than accessing the disk to obtain the information, particularly if calculating a database for the first time or calculating a database when the data is very sparse. Essbase uses the calculator cache bitmap if the database has at least two sparse dimensions, and "either" of these conditions is also met:
1. You calculate at least one, full sparse dimension 2. You specify the SET CACHE ALL command in a calc script.
The best size for the calculator cache depends on the number and density of the sparse dimensions in your outline. Use these topics to understand the calculator cache bitmap, size the calculator cache, and change the size of the calculator cache (and therefore the largest possible size for the bitmap), if required:
Understanding the Calculator Cache Bitmap
For the calculator cache, Essbase separates sparse dimensions in the database into two groups:
1. Bitmap dimensions: the sparse dimensions from the database outline that Essbase fits into the bitmap until the bitmap is full. Each member combination of the sparse dimensions placed in the bitmap occupies 1 bit of memory, and there must be enough space in the bitmap for every member combination of a sparse dimension for it to be placed in the bitmap. 2. Anchoring dimensions: the remaining one or more sparse dimensions in the database outline that do not fit into the bitmap.
Essbase starts with the first sparse dimension in the database outline and fits as many sparse dimensions as possible into the bitmap. The dimensions that fit are the bitmap dimensions. Essbase stops the process when it cannot fit another complete sparse dimension into the bitmap. Because the calculator cache controls the size of the bitmap, the number of sparse dimensions that can fit in the bitmap depends on the size of the calculator cache (and the number and size of the sparse dimensions).
The remaining sparse dimensions are the anchoring dimensions. For anchoring dimensions, Essbase cannot use the bitmap to determine whether or not blocks exist. To see which dimensions are anchoring dimensions and which are bitmap dimensions, use the SET MSG DETAIL calculation command to display bitmap information in the application log. Carefully order the sparse dimensions in your outline so that as many dimensions as possible can be placed into the bitmap. Start with the dimension that contains the fewest members, and continue until the dimension with the most members is last. This order allows more dimensions to fit into the bitmap and results in improved calculation performance.
Essbase uses a single bitmap if there is more than one anchoring dimension or if the calculator cache is not large enough to support multiple bitmaps, and uses two or more bitmaps if there is a single anchoring dimension.
A single bitmap has these properties:
A single bitmap is used to track child blocks. A single bitmap uses the least memory but is less efficient than multiple bitmaps.
Multiple bitmaps have these properties:

Two or more bitmaps are used, one to track child blocks and one to track parent blocks. Multiple bitmaps use more memory but are faster than using a single bitmap. The performance improvement is particularly high when you are calculating the database for the first time. The number of bitmaps used is determined by the maximum number of dependent parents for any of the members in the anchoring dimension. A member has one dependent parent, unless it has a shared member. For example, consider the Product dimension of the Sample Basic database. The member Cola (100-10) has one parent, which is Colas (100). However, Diet Cola (100-20) has two parents, which are Diet Drinks (Diet) and Colas (100). No members of Product have more than two dependent parents. Therefore, if Product is the anchoring dimension, the maximum number of dependent parents is 2. 48 Essbase chooses one of three options for the calculation (listed in order of optimal performance):
1. A single anchoring dimension with multiple bitmaps (optimal performance). 2. A single anchoring dimension with a single bitmap. 3. Multiple anchoring dimensions with a single bitmap (worst performance).
Essbase chooses the optimal performance method for a database calculation, based on the size of the calculator cache. If the calculator cache size is too small for any of the above options, Essbase does not use the calculator cache. If the calculator cache is not used calculation performance may be significantly impaired.
Calculating the Calculator Cache Size
10
Memory Cache Optimization The optimum size of the calculator cache depends on the amount of memory the system has available. It also depends on the nature and configuration of the database. You can calculate the calculator cache size that is required for each of the three options using the database example below.
Consider an example database with five sparse dimensions (S1 to S5):
Sparse Dimension
# of Members
Dependent Parents
S1 S2 S3 S4 S5
20 20 50 50 200
Not applicable Not applicable Not applicable Not applicable 3
Option 1: Single Anchoring Dimension, Multiple Bitmaps - "Optimal"
In order for Essbase to use multiple bitmaps for this database with a single anchoring dimension, the calculator cache needs to be 625,000 bytes.
Bitmap dimensions: Anchoring dimension:
S1, S2, S3, S4 S5 3
Dependent parents in anchoring dimension:
11
Bitmap size in bytes = = =
(S1 * S2 * S3 * S4) / 8 (20 * 20 * 50 * 50) / 8 125,000 bytes
Number of bitmaps = the anchoring bitmaps = =
Maximum number of dependent parents in dimension +2 constant
3+2 5
Calculator cache
= = =
Bitmap size * Number of bitmaps 125,000 * 5 625,000 bytes
12
Memory Cache Optimization Option 2: Single Anchoring Dimension, Single Bitmap
In order for Essbase to use a single bitmap for this database with a single anchoring dimension, the calculator cache needs to be 125,000 bytes.
Bitmap dimensions: Anchoring dimension:
S1, S2, S3, S4 S5 Not applicable
Dependent parents in anchoring dimension:
(S1 * S2 * S3 * S4) / 8 (20 * 20 * 50 * 50) / 8 125,000 bytes
Number of bitmaps = =
Single bitmap 1
Calculator cache
= = =
Option 3: Multiple Anchoring Dimensions, Single Bitmap
In order for Essbase to use a single bitmap for this database with a single anchoring dimension, the calculator cache needs to be 2,500 bytes.
13
Bitmap dimensions: S1, S2, S3 Anchoring dimensions: S4, S5 Not applicable
Dependent parents in anchoring dimensions:
(S1 * S2 * S3) / 8 (20 * 20 * 50) / 8 2,500 bytes
Number of bitmaps = =
Single bitmap 1
Calculator cache
= = =
14
Memory Cache Optimization Choosing a Calculator Cache Size for a Database
The following table shows which calculator cache option Essbase uses, depending on the calculator cache size specified:
Minimum Size
Specified Option Selected
625,000 bytes 125,000 bytes 2,500 bytes
Option 1 (provides optimal performance) Option 2 Option 3
If you specify a calculator cache size of less than 2,500 bytes, Essbase does not use a calculator cache during the calculation. Calculation performance may be significantly impaired. You can check which calculator cache option Essbase is able to use on a database by using the SET MSG SUMMARY command in a calc script. Run the following calc script on the empty database:
SET MSG SUMMARY; CALC ALL;
Essbase displays the calculator cache setting in the ESSCMD window or in the application log.
15
Memory Cache Optimization The maximum calculator cache size that you can specify is 200,000,000 bytes. The default is 200,000 bytes. The calculator cache size that you choose depends on how much memory is available and the configuration of the database.
Calculating the Database for the First Time
If you are calculating the database for the first time, the size of the calculator cache is particularly significant. If possible, ensure that the calculator cache is large enough for Essbase to use the optimal calculator cache option.
Changing the Calculator Cache with Calc Scripts
You can use the default calculator cache size, or you can set the size of the calculator cache within a calc script "Set Cache" (see below). If you set the size from a calc script, the setting is used only for the duration of the calc script.
SET CACHE
The SET CACHE command specifies the size of the calculator cache. Syntax SET CACHE HIGH | DEFAULT | LOW | OFF | ALL;
HIGH, DEFAULT, and LOW
Levels defining the size of the calculator cache. You set the values of HIGH, DEFAULT and LOW in the essbase.cfg file. If you do not set the value of DEFAULT in the essbase.cfg file, Hyperion Essbase uses a default value of 200,000 bytes. The maximum calculator cache size that you can specify is 200,000,000 bytes.
16
Memory Cache Optimization OFF Hyperion Essbase does not use a calculator cache.
ALL
Hyperion Essbase uses a calculator cache, even when you do not calculate at least one full sparse dimension.
17
Memory Cache Optimization Description

SET CACHE specifies the size of the calculator cache. Hyperion Essbase uses the calculator cache to create and track data blocks during calculation. Using the calculator cache significantly improves your calculation performance. The size of the performance improvement depends on the configuration of your database. You can choose one of three levels. The size of the calculator cache at each level is defined using the CALCCACHE{HIGH | DEFAULT | LOW} settings in the essbase.cfg file. The level you choose depends on the amount of memory your system has available and the configuration of your database. For detailed information on setting the size of your calculator cache, see the Database Administrator's Guide. You can specify whether, by default, Hyperion Essbase uses a calculator cache using the CALCCACHE TRUE | FALSE setting in the essbase.cfg file. By default CALCCACHE is set to TRUE. Hyperion Essbase uses the calculator cache providing that:
Your database has at least two sparse dimensions. You calculate at least one, full sparse dimension (unless you specify the CALCCACHE ALL option).
You can use this command more than once within a calculation script. You can display the calculator cache setting using the SET MSG command.
Notes For detailed information on setting the size of your calculator cache, see the Database Administrator's Guide. Example
If the essbase.cfg file contains the following settings: CALCCACHEHIGH 1000000 CALCCACHEDEFAULT 300000 CALCCACHELOW 200000
SET CACHE HIGH;
18

sets a calculator cache of up to 1,000,000 bytes for the duration of the calculation script.
SET CACHE DEFAULT; sets a calculator cache of up to 300,000 bytes for the duration of the calculation script.
SET CACHE LOW; sets a calculator cache of up to 200,000 bytes for the duration of the calculation script.
SET CACHE ALL; SET CACHE LOW; sets a calculator cache of 200,000 bytes to be used even when you do not calculate at least one, full sparse dimension.
SET CACHE OFF; means that Hyperion Essbase does not use a calculator cache.
19
Memory Cache Optimization Changing the Calculator Cache for Large, Flat Database Outlines
You can use the SET CALCHASHTBL command to optimize how Essbase uses the calculator cache when it calculates large, flat database outlines (for example, where one member has more than 5000 children). To use SET CALCHASHTBL, you must enable the calculator cache. When you enable SET CALCHASHTBL, Essbase uses a hash table to optimize use of the calculator cache for large, flat databases. Using this feature may significantly improve the performance of a CALC ALL of the database or a CALC DIM of the dimension containing the member with over 5000 children.
SET CALCHASHTBL
The SET CALCHASHTBL command optimizes the calculation of large, flat database outlines.
Syntax
SET CALCHASHTBL ON|OFF;
ON Turns on optimization of large, flat database outlines. OFF Turns off optimization of large, flat database outlines. The default setting is OFF. You can change this setting using CALCOPTCALCHASHTBL=TRUE in the essbase.cfg file. Description
When you enable this feature, Hyperion Essbase uses a hash table to optimize use of the calculator cache for large, flat databases. A large flat database is, for example, a database in which one or members has over 5000 children. Using this feature may significantly improve the performance of a CALC ALL of the database or CALC DIM of the dimension containing the member with over 5000 children. You can set the limit of the hash table using the CALCHASHTBLMEMORY setting in the essbase.cfg file. For more information on the calculator cache, see the Database Administrator's Guide. This feature has no effect if you are using parallel calculation. For more information about parallel calculation, see the Database Administrator's Guide.
20
Memory Cache Optimization Example

SET CALCHASHTBL ON;
CALCHASHTBLMEMORY
CALCHASHTBLMEMORY sets the calculator hash table memory limit.
Syntax
CALCHASHTBLMEMORY MemSize
MemSize Maximum amount of memory (in bytes) that you want the calculator hash table to use. The default is 20 megabytes. Description
When hash table optimization is in effect, Essbase uses the calculator hash table to optimize the use of the calculator cache when calculating large, flat databases. To turn on hash table optimization, use CALCOPTCALCHASHTBL or SET CALCHASHTBL. If Essbase reaches the hash table memory limit during calculation, Essbase cannot use the hash table method.
Example
CALCHASHTBLMEMORY 30000000
21
Memory Cache Optimization Sizing Dynamic Calculator Caches
Hyperion Essbase uses a separate dynamic calculator cache for each open database. The DYNCALCCACHEMAXSIZE setting in the configuration file, ESSBASE.CFG, specifies the maximum size of each dynamic calculator cache on the server. By default, the maximum size is 20 MB. Hyperion Essbase allocates area in a dynamic calculator cache for data blocks until the maximum memory area specified by the DYNCALCACHEMAXSIZE setting is allocated.
Changing the Dynamic Calculator Cache Size
Four configuration file settings are relevant to dynamic calculator caches. The optimum values for these four dynamic calculator cache settings depend on the amount of memory on the server machine, the configuration of all databases on the server machine, and the nature of user queries.
DYNCALCCACHEMAXSIZE
This setting specifies the maximum size Hyperion Essbase can allocate to each dynamic calculator cache on the server.
Recommended setting value = C * S * U.
C is the value of the appropriate CALCLOCKBLOCK setting in the ESSBASE.CFG file. (The SET LOCKBLOCK command specifies which CALCLOCKBLOCK setting to use.)
22

S is the size of the largest expanded block across all databases on the machine. To calculate the expanded block size, multiply the number of members (including Dynamic Calc members) in each dense dimension together, and then multiply the result by the size of each member cell, 8 bytes.
For example, for the dense dimensions in Sample Basic, 12 (Year) * 8 (Measures) * 3 (Scenario) * 8 bytes = 2304 bytes.
U is the maximum number of expected concurrent users on the database that has the largest number of concurrent users. Assigning the value 0 (zero) to DYNCALCACHEMAXSIZE tells Hyperion Essbase not to use dynamic calculator caches.
The default size is 20 megabytes (20,971,520 bytes).
DYNCALCCACHEWAITFORBLK
If Essbase uses all of the area allocated for a dynamic calculator cache, this setting tells Essbase whether to wait until space becomes available in the cache or to immediately write and calculate the blocks in memory outside the dynamic calculator cache. If the dynamic calculator cache is too small, it is possible for more than one thread to be in queue, each thread waiting to calculate its data blocks.
Recommended setting value = FALSE (default value).
Before setting to TRUE, try these alternatives:
Add physical memory to the server machine
23

Increase the value of DYNCALCCACHEMAXSIZE, test, and repeat until you verify that you cannot use any more memory for the dynamic calculator cache.
24
Memory Cache Optimization DYNCALCCACHEBLKTIMEOUT
If Essbase is to wait for available space in the dynamic calculator cache, this setting defines how long it waits.
Recommended setting value = WT / B.
WT is the maximum tolerable wait time for a query; for example, 5 seconds.
B is the total number of logical blocks accessed in the largest query.
To determine the value of B, check the messages in the application log for the largest number of Dyn.Calc.Cache Big Block Allocs for a query.
DYNCALCCACHEBLKRELEASE
If Essbase has waited the specified time and space is still not available in the dynamic calculator cache, this setting tells Essbase whether to write and calculate the blocks immediately outside the dynamic calculator cache or to create space in the dynamic calculator cache by swapping out blocks and temporarily compressing the swapped blocks in a dynamic calculator cache compressed-block buffer.
Recommended setting value = FALSE (default value).
Set to TRUE only if you are experiencing severe memory shortage problems.
DYNCALCCACHECOMPRBLKBUFSIZE
25
If Essbase has waited the specified wait time and the DYNCALCCACHEBLKRELEASE setting is TRUE, this setting is the size of the dynamic calculator cache compressed-block buffer.
Recommended setting value = (C * S) / 2.
C is the value of the current CALCLOCKBLOCK setting in the ESSBASE.CFG file. The SET LOCKBLOCK command specifies which CALCLOCKBLOCK configuration setting is current.
S is the size of the largest expanded block across all databases on the machine. Calculate S as described for the DYNCALCCACHEMAXSIZE setting.
26

Memory Cache Optmization

Diunggah oleh

Informasi Dokumen

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Memory Cache Optmization

Diunggah oleh

Hak Cipta:

Format Tersedia

Key Performance Ideas Knowledge Base

Memory Cache Optimization

Description Application Version Submitted By Date

Memory Cache Optimization Essbase 6x+ Rob Redford 11/12/07

Key Performance Ideas Knowledge Base

Memory Cache Optimization

Memory Cache Optimization

Key Performance Ideas Knowledge Base

Memory Cache Optimization Sizing the Index Cache

Minimum value: Default Value:

1 MB (1048576 bytes) "Buffered I/O" "Direct I/O" 1 MB (1048576 bytes)

Key Performance Ideas Knowledge Base

Memory Cache Optimization Sizing the Data File Cache

Minimum value: Default Value:

10 MB (10485760 bytes) 32 MB (33554432 bytes)

Key Performance Ideas Knowledge Base

Memory Cache Optimization Sizing the Data Cache

Key Performance Ideas Knowledge Base

Memory Cache Optimization Default Value: 3 MB (3145728 bytes)

Testing Cache Settings

Key Performance Ideas Knowledge Base

Memory Cache Optimization

Key Performance Ideas Knowledge Base

Memory Cache Optimization Sizing the Calculator Cache

Understanding the Calculator Cache Bitmap

Key Performance Ideas Knowledge Base

Memory Cache Optimization

A single bitmap has these properties:

Multiple bitmaps have these properties:

Key Performance Ideas Knowledge Base

Memory Cache Optimization

Calculating the Calculator Cache Size

Key Performance Ideas Knowledge Base

Consider an example database with five sparse dimensions (S1 to S5):

Not applicable Not applicable Not applicable Not applicable 3

Option 1: Single Anchoring Dimension, Multiple Bitmaps - "Optimal"

Bitmap dimensions: Anchoring dimension:

S1, S2, S3, S4 S5 3

Dependent parents in anchoring dimension:

Key Performance Ideas Knowledge Base

Memory Cache Optimization

Bitmap size in bytes = = =

(S1 * S2 * S3 * S4) / 8 (20 * 20 * 50 * 50) / 8 125,000 bytes

Number of bitmaps = the anchoring bitmaps = =

Maximum number of dependent parents in dimension +2 constant

Bitmap size * Number of bitmaps 125,000 * 5 625,000 bytes

Key Performance Ideas Knowledge Base

Memory Cache Optimization Option 2: Single Anchoring Dimension, Single Bitmap

Bitmap dimensions: Anchoring dimension:

S1, S2, S3, S4 S5 Not applicable

Dependent parents in anchoring dimension:

Bitmap size in bytes = = =

(S1 * S2 * S3 * S4) / 8 (20 * 20 * 50 * 50) / 8 125,000 bytes

Bitmap size * Number of bitmaps 125,000 * 1 125,000 bytes

Option 3: Multiple Anchoring Dimensions, Single Bitmap

Key Performance Ideas Knowledge Base

Memory Cache Optimization

Bitmap dimensions: S1, S2, S3 Anchoring dimensions: S4, S5 Not applicable

Dependent parents in anchoring dimensions:

Bitmap size in bytes = = =

(S1 * S2 * S3) / 8 (20 * 20 * 50) / 8 2,500 bytes

Bitmap size * Number of bitmaps 2,500 * 1 2,500 bytes

Key Performance Ideas Knowledge Base

Memory Cache Optimization Choosing a Calculator Cache Size for a Database