Francie Tanner
panagenda
2011 Wellesley Information Services. All rights reserved.
1
In This Session Were Going To
Learn how to configure your hardware, operating system, and
Domino servers and applications so they can perform their best
Gett the
G th tools
t l tto h
help
l you id
identify
tif whats
h t slowing
l i your servers
down
Find out how to keep your data healthy and corruption free
And if that didnt work how to resolve corruption
Versions 4-8
Bunch o certifications
3
The Cost of NOT Maintaining Your Servers and Data
Bad server performance
Agents dont finish running in time
M i t
Maintenance tasks
t k dont
d t finish
fi i h over night
i ht
Users are unable to open their mail in the morning
5
Optimizing Your Hardware
Hardware isnt everything
But you can avoid performance bottlenecks here
f Put everything on the same slow disks and this wont help
Drive cache
Optimize
O ti i this
thi especially
i ll if youre
going
i to
t let
l t your OS
manage this
U a 64-bit
Use 64 bit OS,
OS if possible,
ibl as it has
h more capabilities
biliti
Even with 32-bit Domino code
Disk probe
Memory probe
Network probe
7
Disk Input/Output Bottlenecks
Domino is extremely read and write intensive
Disk access rates and configuration affects Domino profusely
9
Domino Software Tuning Version 8
You do not need to adjust the following as the UMB/NSF Buffer
Pool is automatically set to 512MB on 32-bit Domino and 1024MB
on 64-bit
PercentAvailSysResources
ConstrainedSHM, ConstrainedSHMSizeMB
MEMAddressableMem, MEMAddressableMemSizeMB
MEM_EnablePreAlloc, MEM_EnableSubAlloc
NSF_Buffer_Pool_Size_MB
f All of the above get set by default but some variables will still
get honored
10
NSF_DbCache_Maxentries=(number)
Can improve server performance by increasing the number of
databases that Domino can cache in memory at one time
Only
y modifyy if yyour server has sufficient memoryy
ConstrainedSHMSizeMB=(number in MB)
Decreases shared memory so private memory can increase
12
13
Transaction Logging Tuning
Transaction logging
Records transactions in a log and then writes them to disks
14
If youre
running
i Domino
D i 8.5.1
8 5 1 Create_R85_Log=1
C t R85 L 1
Optimizes transaction log format for Domino 8.5 servers
Enables logs to use block size higher than the default size of
512 bytes
f This requires that you bring down your server, back up, and
delete existing
g transaction logs
g before recreating
g them
DANGER WILL ROBINSON!
15
Indexer Tuning
Your directory is the brain of your Domino server
If its not properly optimized, ACL and view look-ups, client
sessions, and everything else will be slow
Mi i i customization
Minimize t i ti
Particularly to the number of custom views that have to be
maintained
Set the view rebuild directory to something other than your data
directory
View_Rebuild_Dir=<directory
_ _ yppath>
Update_Access_Frequency=(number in days, default 7)
Reduces the frequency of updates to rarely accessed views
allowing faster updates to more frequently used views instead
f DO NOT set this too low, or below 4
16
17
Indexer Tuning (cont.)
DEBUG_DISABLE_CHRONOS=1 (default is 0)
Stops the Chronos task from performing hourly updates of FTIs
set to immediate
f Only
O l dod this
thi after
ft understanding
d t di the th impact
i t
Update_Fulltext_Thread=1 (default is 0)
Allocates a separate thread to do full text indexing, so that long
full text indexes dont delay view updates
f View updating will still work in updall
18
f Tells the server to use the old index while the new one
catches
t h up
DEBUG_ENABLE_UPDATE_FIX=8191
19
What Well Cover
Introduction
Tuning your hardware, OS, disks, and network
Domino transaction logging and indexer tuning
Working with your ODS, clusters, tasks, and ports
Preventing corruption and maintaining your data
Working with DTC, DDM, statistics, and activity trends
Wrap-up
20
ODS Tuning
Domino 8.5 uses ODS 48
Use Create_R8_Databases=1
21
ODS Tuning (cont.)
New applications and those who have the following command run
on them, get automatically upgraded to the new ODS
Load compact c
ENABLE_ON_DEMAND_COLLATIONS=1
Reduces the unnecessary server load of creating indexes for
columns that are not being used
This is done by deferring the creation of indexes until the user
first chooses to sort the view by a specific column
f Available
A ailable to applications with
ith ODS 48 or higher
DAOS
Saves up to 50% of space by consolidating attachments
Enabled via database properties Use Domino Attachment and
Object Service for transaction logged ODS 51 applications
22
23
Cluster Tuning
Depending on what style of clustering you use (active-passive vs.
active-active) Spread all databases evenly across your cluster
servers
Putting all mail files on one server leaves the others un-utilized
un utilized
Monitor Server.AvailabilityThreshold
The threshold at which the server will enter the
busy state and fail over
f Set config Server_Availability_Threshold = n
24
25
Cluster Tuning (cont.)
As mentioned earlier, use multiple NICs and private LAN
segments for cluster traffic
Server_Clueter_Default_Port
Replica.Cluster.SecondsOnQueue.xx
Number of seconds replication
p events are waiting
g to replicate
p
with other cluster members
f High numbers but especially high averages indicate a cluster
replication problem
Check disk, memory, and CPU bottlenecks
27
Cluster Performance Tuning Cluster Analysis Tool
Will help you find issues with your cluster environment
Decommission Server Tool does something similar if youre
just looking to verify if you have a matching set of applications
28
29
Working with Server Tasks (cont.)
Dont schedule archiving, domain indexing, or maintenance
during production hours
More on maintenance next
D t schedule
Dont h d l domain
d i indexing
i d i during
d i peakk hours
h
Reduce logging when possible
For example, with Converter_Log_level=10
30
Tuning Routing
Minimize the number of mail rules used by the server
Use journaling only when necessary
31
Turn Off and Disable All Unused Ports
Again, seems obvious but
32
Avoid
A id encrypting
ti TCP/IP traffic
t ffi unless
l necessary
It adds up to 20% to the existing response time
33
What Well Cover
Introduction
Tuning your hardware, OS, disks, and network
Domino transaction logging and indexer tuning
Working with your ODS, clusters, tasks, and ports
Preventing corruption and maintaining your data
Working with DTC, DDM, statistics, and activity trends
Wrap-up
34
35
How Do I Find Corruption?
Ad hoc Your phone rings/you get a ticket
Corruption can cause lack of access to applications
36
Updall
Evoked on an as needed or scheduled basis
37
The 3 1/2 Commands That Fix Most Issues (cont.)
Compact
Upgrades the On-Disk Structure (ODS) of a database
f In-place
In place with space recovery
f In-place with space recovery and reduction in file size
f Copy-style compacting
38
39
The 3 1/2 Commands Compact Styles (cont.)
Copy-style compacting
Meaning compact c
40
Updall database.nsf
database nsf T
T $Servers
Updates a specific view
42
Compact Switches
Compact S 15
Compacts DBs with 15% or more unused space
Compact R
Compacts without conversion to current Domino release
Uses copy-style
Compact D
Discards built view indexes and runs a copy-style compact
Compact a
Archives
A hi and
d deletes
d l t documents,
d t then
th compacts t DB
f For more compact switches, see Administrator Help
43
Fixup Switches
Fixup L
Logs every database that Fixup opens and checks
Fixup C
It only checks the data and reports back, but does not purge
corrupted documents
f If you need to show yourself that Fixup isnt needed, maybe
tryy using
g this
Fixup F
Checks all documents in the database
44
Can
C beb run against
i t entire
ti server, specific
ifi databases,
d t b allll
databases within a folder
Automated Using
g Domino Domain Monitoring
g ((DDM))
45
Best Practice Maintenance
Transaction logging
Compact b
Updall R X
No transaction logging
Fixup F L
Updall R X
Compact B
46
47
How to Execute Database Maintenance
From the Domino server console in your admin client
Load compact mail\ftanner.nsf c
48
49
Automating Maintenance
Note that nearly all of this maintenance and monitoring can be
automated easily in Domino 7 and later
Use Program documents to schedule compact task
f Use
U indirect
i di t files
fil tot speedd it up
Use OS scheduling to automate tasks that require the server to
be down
Use DDM to automate the Review items
50
Command
C d liline: c < C
C:\lotus\domino\data\maint.txt
\l t \d i \d t \ i t t t
f Be careful, they are all submitted at the same time
51
Using Indirect Files to Automate Maintenance
If compact is taking too long, try breaking the list up into smaller
pieces
IND files or indirect files were introduced in ND6
An
A iindirect
di t file
fil is
i a ttextt fil
file th
thatt lists
li t ddatabase
t b file
fil names or
directory names
f You can use them with the compact, updall, or Fixup task on
the Domino console command line
The indirect file allows you to specify more than one
database on the command line
52
Updall RR system.ind
system ind
53
Run Indirect Files Using a Program Doc
Combine the methods to run your maintenance routines
Create an IND file that contains the path and file names you
want to run maintenance on
Create
C t a TXT fil file containing
t i i ththe commandsd you wishi h to
t run
Create a Program document to tie it all together
54
f Files.ind
Fil i d contains
t i theth files
fil youll
ll maintain
i t i
f Offlinemaint.cmd is the command file you schedule
It shuts down the server, then runs all the maintenance offline
55
Best Practice Maintenance
Avoid conflicting/overlapping maintenance commands
It will cause corruption
Remove S
R ServerTasksAt2
T k At2 ffrom your Notes.ini
N t i i
Most admins dont know its there and schedule compact or
other conflicting server tasks at 2 a.m., causing corruption
Avoid editing the Notes.ini directly via the operating system
56
57
Domino Configuration Tuner
Runs on your client and analyzes server documents, tasks,
Notes.ini, database properties of one or many servers
Download it here:
www 01 ibm com/support/docview wss?uid=swg24019358
www-01.ibm.com/support/docview.wss?uid=swg24019358
After initializing the server list for the first time, the scan will run
in the background
Watch your status bar for completion
58
59
Domino Configuration Tuner (cont.)
Be sure to check out the Links and References section
Check for updates from time to time
Turn off unwanted rules under Preferences
60
Server Statistics
Weve talked about several statistics to monitor
Admin client Server Statistics shows them in a
categorized way to make them easy to access
61
Disk Statistics to Watch
Disk stats are the number one problem on slow performing
servers
Diskk stats
Di t t are collected
ll t d in
i different
diff t ways, but
b t allll off them
th have
h
PctUtil as part of the stat
It could be Logical Disk or Physical Disk
Like Platform.LogicalDisk.1.PctUtil.Avg
62
63
Putting DDM to Work for You
Use any applicable DDM probe to help you understand server
short comings
64
65
Activity Logging and Activity Trends
Enabled via the Configuration Settings document Activity
Logging tab
Records user activity by person, database, and access protocol
Can
C be b summarized,
i d averaged,d or trended
t d d over time
ti to
t measure
and compare workloads across servers
Can be used in conjunction with the Domino Change Manager
to automate the creation and execution of workload
redistribution plans to load a new server, decommission an old
one, or balance workloads across unevenly burdened servers
Activity logging is categorized under Monitoring/Managing
Servers
f Activity Trends is categorized under Performance
66
67
Resources
Lotus Domino Domain Monitoring Redpaper
www.redbooks.ibm.com/abstracts/redp4089.html
68
Resources (cont.)
Notes.ini analyzer and reference by entry
www.lntoolbox.com/en/notesini-reference.html
69
7 Key Points to Take Home
When dealing with performance problems, dont just address the
data OR the hardware OR Domino
Only adjust settings after identifying a problem, testing, and
measuring their impact
Use automated tools to maintain your data and prevent corruption
Use DTC to quickly understand what can be improved
Implement transaction logging to minimize disk writes and
corruption due to impartially written transactions
Keepp current with application
pp ODS and server versions as theyy
provide many performance improvements
Use DDM, statistics, cluster analyzer, and activity logging to help
you understand whats happening on your server
70
Your Turn!