High Availability Architectures With Mysql and DRBD: Jimmy Guerrero

High Availability Architectures with
MySQL and DRBD
Jimmy Guerrero
Senior Product Manager
jguerrero@mysql.com
Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 1
Agenda
• Introduction to MySQL
• Comparing MySQL HA Architectures
– MySQL Replication
– MySQL Replication plus Linux Heartbeat
– MySQL Cluster
• MySQL, Heartbeat plus DRBD Architectures
• Setup and Configuration
• Managing Failover
• High Availability Services from MySQL
• Q&A
Overview of MySQL AB
• Founded in 1995
• Operations in 24 countries
• World’s Most Popular Open Source Database
– Over 10 million installations
– 50,000 downloads each day
• Reduces TCO of database management
– Reliable, high performance, easy to use
• Bundled by over 100 software & hardware companies
• Partners
HP, Dell, Novell, Red Hat, Unisys, Accenture, LogicaCMG, T-Systems and IPL
Established & Emerging Companies
High Volume Websites
– Web 2.0
– Dynamic content
– eCommerce
– Look to Book
– Session Management
– Gaming & Entertainment craigslist
– Scale Out
Enterprise
– Data Warehousing
– High-Volume OLTP
– Scale Out
Embedded
– Software & Hardware
MySQL, Heartbeat and DRBD Cluster
• High availability solution built on open source software
– MySQL: Database
– Linux Heartbeat: Virtual IP management
– DRBD: Synchronous block level replication and fast data
resynchronization after node failure
• Low cost
– Many popular Linux distributions supported
– Can be built on COTS hardware components
– No migration hassles
• Supports MySQL “scale out” for reads
• Supports transactional storage engines
– InnoDB
– Falcon
What is DRBD?
• Block device which leverages synchronous replication
– Open source Linux kernel module
– Bundled by several popular Linux distributions
– Project maintained by LinBit
• Core functionality
– The entire block device is mirrored via a dedicated network
– Similar to RAID-1 over Ethernet
– DRBD operates at disk I/O layer
– Transparent to MySQL and the underlying file system
– MySQL fails over to passive DRBD server via "virtual IP
management"
• Typically accomplished with the Linux Heartbeat package.
– MySQL transactional storage engine independent
• InnoDB, Falcon
http://www.drbd.org/ http://www.linux-ha.org/
MySQL HA Solutions
Clustering &
Geographical
Redundancy
MySQL
Clustering Cluster
Technologies
MySQL + DRBD
Replication
MySQL + SAN
Technologies
Well-Managed MySQL
Replication
Unmanaged
Small ISPs & Telco

Data Banking Military
Business Mainstream Centers Medical
Business Defense
9
35 days
9.
4 days
9
8 hours
9
50 mins
9
5 mins
%
What is Replication?
Asynchronous Replication
update
customer
Synchronous Replication
update
customer
MySQL Replication
• Asynchronous Requirements Support
– Statement Availability
– Row-based (5.1) Automated IP Fail Over No
– Mixed (5.1)
Automated DB Fail Over No
• Read vs. Write
Typical Fail Over Time Varies
– Response times
Resynchronization of Data No
– Load balancing
– Scale Out Geographic Redundancy Yes
• Administration Scalability
– Backups Built-in Load Balancing Scale-Out
– Upgrades Read Intensive Yes

– Testing Write Intensive No
• Geographic Redundancy Dozens for
# of Nodes
Reads
MySQL Replication Topologies
Master > Slave Master > Slaves
Masters > Slave (Multi-Source) Master > Slave > Slaves
Circular (Multi-Master)
Master < > Master (Multi-Master)
MySQL Replication - Scale Out
• Write to one master
• Read from many slaves
• Perfect for read intensive apps
Load Balancer
Writes & Reads Reads Reads
MySQL Replication
MySQL Replication & Linux Heartbeat
IP Management
Active Passive
MySQL MySQL
Server Server
Asynchronous
MySQL Replication
Heartbeat implements serial, UDP, and PPP/UDP heartbeats together with IP

address takeover, this solves MySQL Replication’s “fail over” problem, but, data
resynchronization is not automatic and can be complex.
MySQL Cluster
Requirements Support
• In-Memory Database
Availability
– Disk Data (5.1)
• Shared-nothing storage Automated IP Fail Over No
• Synchronous replication Automated DB Fail Over Yes
• Fast-automatic data fail over Typical Fail Over Time < 3 Secs
• High performance Resynchronization of Data Yes

– NDB API Yes
Geographic Redundancy
(w/ Replication)
• No special hardware
Scalability
• Best for primary key lookups
Load Balancing Scale-Out
• Transactional
• Online backup Read Intensive Yes (Look ups)
• No automated IP failover Write Intensive Yes
5.0: 64
# of Nodes
5.1: 255
MySQL Cluster: Architecture
Clients
NDB API Application Nodes

(NDB API and/or MySQL Server)
Management
Nodes
(Typically 2)
Data Nodes
NDB Storage Engine
MySQL Cluster: Data Distribution
PK col1 col2 col3 Partitions

- Automatic or User-Defined
1 data data data
- Horizontal
2 data data data - MD5 hash of Primary Key
3 data data data
4 data data data
Two Data Nodes
* Complete copy of database on each node
Two Replicas
* Two copies of each partition
Node Group
One Node Group

P1 - Primary P1 - Replica * One set of identical Data Nodes
P2 - Replica P2 - Primary
MySQL, Heartbeat & DRBD Cluster
• Distributed storage Requirements Support
• Similar to network RAID 1 Availability
• Synchronous replication Automated IP Fail Over Yes
• No special networking Automated DB Fail Over Yes

components
Typical Fail Over Time < 30 secs
• Excellent performance
Resynchronization of Data Yes
– Blocks vs. rows of data
MySQL
• Manages inconsistencies of Geographic Redundancy
Replication
data during a failure Scalability
• Hides the complexity of many Built-in Load Balancing No
recovery actions
Read Intensive Yes
• Heartbeat manages fail over
and virtual IPs Write Intensive Possible
# of Nodes 1 Active
MySQL, Heartbeat & DRBD Cluster
IP Management
Active Passive
DRBD DRBD
Server Server
Synchronous
Block Replication
Scale Out
MySQL Replication w/ DRBD Cluster
IP Management
Active Passive
Server Server
MySQL Replication Slaves – Read Scalability - Asynchronous

Application Partitioning
• Data is separated into distinct, autonomous data silos

or “shards” within DRBD Clusters
• Applications access “shards” directly or through load
balancing
• Used in conjunction with MySQL Replication, can
increase read & write scalability
• Potential to also increase availability depending on
partitioning scheme
• Partitioning on different levels
– Application
– MySQL data partitioning in 5.1
– Reads vs Writes
Application Partitioning
Load Balancer/SQL Relay
Shard 1 Shard 2
IP Management IP Management
Block Devices
• A device node is a type of special file
– Used on many Unix-like operating systems
• Facilitates transparent communication
– Between user space applications and hardware
• Correspond to resources allocated by the operating
system kernel
• Accessed using standard system calls and treated like
regular files
• Used to correspond to devices
– Data is transmitted in the form of blocks.
• Make use of buffered input and output routines
DRBD Basics
• Each device has one of two states
– Primary
– Secondary
• Application runs and accesses the device on the
primary node
• Writes are sent to the local lower level block device
and to the node with the device in secondary state
• Secondary device writes the data to its lower level
block device
• Reads are always carried out locally
– Key advantage over SAN/NAS
• If the primary node fails
– Heartbeat switches the secondary device into a primary state
and starts the application (MySQL)
DRBD Basics
• When the failed node comes up
– Becomes a secondary node
– Must synchronize its data to the primary
– Synching happens in the background without service
interruption
– Only the parts of the device which have changed since the
failure will be resynchronized.
• Total resynchronization in just several minutes
– Regardless of device size
– Even after the hard crash of an active node
– Resynch rates may be configured per device
• Device sizes currently limited to 4 TB
Setup & Configuration
System Requirement Support
Hosts Minimum 2 (More for scale-out or partitioning)
COTS or SMP - 32 & 64-bit

Hardware
Multiple NICs for redundancy
Varies (More is generally better)

Memory
(32 KB per 1 GB block)
Storage SCSI, RAID 0+1 for I/O Performance
100 Mbps or better

Network
(Gigabit recommended, others supported)
Operating System Linux (Redhat, Suse, CentOS, Fedora, Debian)
Setup & Configuration
• Multiple NICs will increase redundancy
• MySQL traffic switched over public LAN
• DRBD & Heartbeat over private LAN
IP Management
Active Passive
DRBD DRBD
Server Server
Setup & Configuration - DRBD
• Additional prerequisite packages may be needed
– Depending on kernel version
– When building from source
– Or using RPMs
• Pre-built x86 and x86_64 packages for specific kernel
versions available with a support subscription
– SuSE 9 & 10
– Redhat AS 4
• If kernel is upgraded, DRBD must be as well
• Current DRBD version is 8 (Jan 24, 2007)
– Better, tuneable recovery
– Ability to run DRBD on top of LVM
– Performance optimization (Jumbo frames)
Setup & Configuration - DRBD
• Install DRBD package
• Edit and copy configuration file (drbd.conf) to nodes
• Choose primary node
• Synchronize the underlying devices
• Device is now ready
– Create a file system if one does not exist
• Mount DRBD on primary
• Heartbeat handles
– Changing of DRBD primary or secondary status
– Mounting and unmounting volumes
Setup & Configuration - MySQL
• Ensure all MySQL files are installed on the DRBD
volume
• Create ‘mysql’ group and user
• Create MySQL directory
• Install MySQL
• Shutdown MySQL
• Unmount the DRBD volume
Setup & Configuration - Heartbeat
• Install Heartbeat V2 package
– http://www.linux-ha.org/
– Ships with many Linux distributions
• Configuration
– ha.cf: describes the participating machines and their
configuration
– haresources: specifies virtual IP and managed services
– ipfiles: detects when one of their network links has become
unusable, and compensate accordingly
– authkeys: three types - crc, md5, and sha1
– resource.d/mysql: start and stop script for MySQL
Initialization & Next Steps
• Start DRBD
• Start Heartbeat
– Node becomes DRBD primary
– DRBD volume is mounted
– MySQL starts
– IP address taken over
• Next steps
– Fail over testing
– Benchmarking
– Stress testing
– Create monitoring framework
– Attach MySQL Replication slaves for read scalability
Managing Failures
IP Management
Passive X Active
DRBD DRBD
Server Server
X
X
X
Data Resynchronization after Host Failure
IP Management
Passive Active
DRBD DRBD
Server Server
Resynchronization
Active Server is still able to process transactions during a resynchronization operation

Comparing MySQL HA Solutions
MySQL MySQL,
MySQL
Requirements Replication + Heartbeat + MySQL Cluster
Replication
Heartbeat DRBD
Availability
Automated IP Fail Over No Yes Yes No
Automated DB Fail Over No No Yes Yes
Typical Fail Over Time Varies Varies < 30 secs < 3 secs
Auto Resynch of Data No No Yes Yes
Geographic Redundancy Yes Yes MySQL Replication MySQL Replication
Scalability
MySQL
Built-in Load Balancing MySQL Replication MySQL Replication Yes
Replication
Read Intensive Yes Yes MySQL Replication Yes
Write Intensive No No Yes Yes
# of Nodes per Cluster Master/Slave(s) Master/Slave(s) Active/Passive 255
Dozens for
# of Slaves Dozens for Reads Dozens for Reads Dozens for Reads
Reads
Services for DRBD from MySQL
• MySQL High Availability Jumpstart

– Analyze your current database infrastructure
– Analyze your Scale-out and High-Availability database
requirements
– Identify potential issues and limitations
– Install, Configure and Fine-Tune MySQL
– Configure MySQL or MySQL Cluster to work with HA
Technologies (e.g. DRBD, SAN, Shared SCSI, Heartbeat,
etc), as required
– Secure your database environment against failures and
downtime
– Benchmark and load test against sample queries
http://www.mysql.com/consulting/packaged/scaleout.html
Case Study: footnote.com
• Online repository of historical documents & images
• Founded in 1994, based in Utah, 80 employees
Component Detail
Network Foundry Networks FastIron
Load Balancing ServerIron
Storage NetApp FAS 3020HA w/ SATA disks
Application 35-40 Dell 1955 Blades
Operating
CentOS 4.4
System
Database MySQL 5.0.36
Replication 1 read-only slave

Schema Details
Woodstone Servers Alive, Ganglia &
Monitoring 63 Tables
Splunk
Backup Automysqlbackup Largest Table: 36,000,000+ rows
DB Size: 15 GB
Summary
• High availability solution built on open source software
– MySQL: Database
– Linux Heartbeat: Virtual IP management
– DRBD: Synchronous block level replication and fast data
resynchronization after node failure
• Low cost
– Many popular Linux distributions supported
– Can be built on COTS hardware components
– No migration hassles
• Supports MySQL “scale out” for reads
• Supports transactional storage engines
– InnoDB
– Falcon
MySQL Conference & Expo 2007
April 23-27, Santa Clara
MySQL 5.1 – Falcon – Partitioning
Cluster – Replication – Management
Performance Tuning – Security
Administration - DRBD
Over 100 Sessions and Tutorials!

Q&A
Thank You!
Jimmy Guerrero
Senior Product Manager
jimmy@mysql.com
White Papers - Case Studies - Press Releases - News - Events - Live Webinars - Forums

High Availability Architectures With Mysql and DRBD: Jimmy Guerrero

Diunggah oleh

Informasi Dokumen

Deskripsi Asli:

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

High Availability Architectures With Mysql and DRBD: Jimmy Guerrero

Diunggah oleh

Hak Cipta:

Format Tersedia

High Availability Architectures with

MySQL and DRBD

Small ISPs & Telco

– Backups Built-in Load Balancing Scale-Out

– Upgrades Read Intensive Yes

Masters > Slave (Multi-Source) Master > Slave > Slaves

Writes & Reads Reads Reads

Heartbeat implements serial, UDP, and PPP/UDP heartbeats together with IP

• Synchronous replication Automated DB Fail Over Yes

• High performance Resynchronization of Data Yes

• No automated IP failover Write Intensive Yes

NDB API Application Nodes

NDB Storage Engine

PK col1 col2 col3 Partitions

One Node Group

• Synchronous replication Automated IP Fail Over Yes

• No special networking Automated DB Fail Over Yes

MySQL Replication Slaves – Read Scalability - Asynchronous

• Data is separated into distinct, autonomous data silos

Load Balancer/SQL Relay

System Requirement Support

Hosts Minimum 2 (More for scale-out or partitioning)

COTS or SMP - 32 & 64-bit

Varies (More is generally better)

Storage SCSI, RAID 0+1 for I/O Performance

100 Mbps or better

Operating System Linux (Redhat, Suse, CentOS, Fedora, Debian)

Active Server is still able to process transactions during a resynchronization operation

Automated IP Fail Over No Yes Yes No

Automated DB Fail Over No No Yes Yes

Auto Resynch of Data No No Yes Yes

Geographic Redundancy Yes Yes MySQL Replication MySQL Replication

Read Intensive Yes Yes MySQL Replication Yes

Write Intensive No No Yes Yes

# of Nodes per Cluster Master/Slave(s) Master/Slave(s) Active/Passive 255

• MySQL High Availability Jumpstart

Network Foundry Networks FastIron

Load Balancing ServerIron

Storage NetApp FAS 3020HA w/ SATA disks

Application 35-40 Dell 1955 Blades

Database MySQL 5.0.36

Replication 1 read-only slave

Backup Automysqlbackup Largest Table: 36,000,000+ rows

Cluster – Replication – Management

Performance Tuning – Security

Over 100 Sessions and Tutorials!

Anda mungkin juga menyukai