MultimediaDatabases

Database Architectures
Courtesy of Lecture Notes from Silberschatzh, Korth and Sudarshan, and other sources

Dr. Vidhya Balasubramanian Amrita School of Engineering

Overview
1.MultimediaArchitectureRequirements 2.MultimediaServerRequirements 3.DistributedMultimediaSystems 4.ClientServerSystems 5.PeertopeerSystems 6.HeterogeneousDBMSs 7.PerformanceandStorageIssues

Dr. Vidhya Balasubramanian Amrita School of Engineering

Material

Chapter8:LynneDunckley,"Multimedia Databases:AnObjectRelationalApproach", PearsonEducation,2003 Silberschatz,KorthandSudarshan,Database Concepts,4thEdition,TataMcGrawHill. Notes

Dr. Vidhya Balasubramanian Amrita School of Engineering

MultimediaArchitectureRequirements

Databasesupportstransactionswhichmustpossess 4properties(ACID)

Atomicity:Allornothingproperty.Atransactionisan indivisibleunitthatiseitherperformedornot Consistency:Atransactionmusttransformthedatabase fromoneconsistentstatetoanotherconsistentstate Independence:Transactionsmustexecute independentlyofoneanother Durability:Theeffectsofasuccessfullycommitted transactionshouldbepermanentlyrecordedinthe database

Dr. Vidhya Balasubramanian Amrita School of Engineering

SpecialChallenges:MultimediaData

Atomicityisachallenge

Transactiontakeslongertimeduetothesizeofdata Lockshavetobemaintainedforalongertime Mustavoidtransmissionofmediadataacrossnetworks duringatransaction Play,fastforward,reverse,zoom Onedimensional(text,audio),twodimensional( image),3dimensional(video)

Dr. Vidhya Balasubramanian Amrita School of Engineering

Needforoperationslike

Accessmethodsdifferentfordifferentdata

GeneralDatabaseArchitectures

CentralizedSystems ClientServerArchitectures DistributedSystems

Dr. Vidhya Balasubramanian Amrita School of Engineering

CentralizedSystems

Runonsinglesystemanddonotinteractwithothersystems Generalpurposecomputersystem:onetoafewCPUsanda numberofdevicecontrollersthatareconnectedthrougha commonbusthatprovidesaccesstosharedmemory. Singleusersystem(e.g.,personalcomputerorworkstation):

desktopunit,singleuser,usuallyhasonlyoneCPUand oneortwoharddisks;theOSmaysupportonlyoneuser. moredisks,morememory,multipleCPUs,andamultiuser OS. Servealargenumberofuserswhoareconnectedtothe systemvieterminals.Oftencalledserversystems.

Dr. Vidhya Balasubramanian Amrita School of Engineering

Multiusersystem:

CentralizedSystem

Dr. Vidhya Balasubramanian Amrita School of Engineering

ClientServerSystems

Serversystemssatisfyrequestsgeneratedatm clientsystems

http://support.quadstone .com/info/requirements/

Networkofworkstationscanconnecttoservers replacingneedforexpensivemainframes

Providesflexibilityinlocatingresourceandexpanding facilities Bettermaintainanceandbetterinterfaces

Dr. Vidhya Balasubramanian Amrita School of Engineering

ClientServerSystems

Databasefunctionalitycanbedividedinto

Backend:managesaccessstructures,queryevaluation andoptimization,concurrencycontrolandrecovery Frontend:consistsoftoolslikeforms,reportwrites andGUIfacilities

Interfacebetweenfrontendandbackendthrough SQLoranapplicationprograminterface.

Dr. Vidhya Balasubramanian Amrita School of Engineering

ServerSystemArchitectures

TransactionServers:usedmostlyinrelationalDB DataServers:usedinOODBs

Microsoft Transaction Server

Dr. Vidhya Balasubramanian Amrita School of Engineering

http://www.treehugger.com/f iles/2008/11/green_grid_mp g.php

TransactionServers

CalledQueryServerSystemsorSQLServerSystem

Clientssendrequesttoserver,transactionsperformed atserverandresultssentback TransactionalRPCallowsmanyRPCcallstoforma transaction CprograminterfacefromMicrosoftforconnectingtoa server,sendingSQLrequests,gettingresults JDBCisforJava

Dr. Vidhya Balasubramanian Amrita School of Engineering

RequestsspecifiedinSQL,communicationviaRPC

OpenDatabaseConnectivity(ODBC)

DataServers

UsedinhighspeedLANs,incaseswhere

Theclientsarecomparableinprocessingpowertothe server Thetaskstobeexecutedarecomputeintensive.

Dataareshippedtoclientswhereprocessingis performed,andthenresultsshippedbacktothe server. Thisarchitecturerequiresfullbackend functionalityattheclients. Usedinmanyobjectorienteddatabasesystems, speciallymultimediasystems

Dr. Vidhya Balasubramanian Amrita School of Engineering

CaseStudy:VideoServers

VoDServersdeliversvideobasedonuserrequest

Videostreamdedicatedtoeachrequest Periodicdeliveryofaudiosamplesandvideoframes Steadydataratetoavoidstarvation Minimumbuffersizepreferred Needsupportforcueing

Playbackrequires

Topauseandcontinueplaying

Affectedduetolimitedbufferspace Scheduledecidedbysystem Dr. Vidhya Balasubramanian

Amrita School of Engineering

Videoonschedulesystemstoovercomethislimitation

MultimediaServerRequirements

Oftenlargeapplications Designofamediaservermustconsider

Useraccessbehavior Heterogeneityofdata Highvolumesofinformationandrealtime requirements

Largediskspace Highbandwidthforrealtimesupport Additionalstorageloadnecessitatingcompression

Temporalrequirementsofdata

Compressionwithoutcompromisingquality
Dr. Vidhya Balasubramanian Amrita School of Engineering

RequirementsofMultimediaServer

Minimalresponsetime Fastprocessingcapabilityandlowdatarates ReliabilityandAvailability

Datamaybeonthediskwhichmaymaketheaccess slower Minimizeneedforphysicaldataaccess Affectsnumberofclientsaservicecanhandle

Abilitytosustainguaranteednumberofstreams

Dr. Vidhya Balasubramanian Amrita School of Engineering

Realtimedelivery

Requirements

Timeguaranteesforbothindividualstreamsandall streamstogether BetterresourceschedulingatOSlevel

RealtimeOSnecessary

HighStorageCapacity

Compressionmaybeneeded,mustbefastandefficient Endenddelay,throughput,reliabilityetc MustguaranteeQoSbasedonclientdevice,network connection,datatypesetc

Dr. Vidhya Balasubramanian Amrita School of Engineering

QualityofServiceRequirements

Requirements

ExploitUserAccessPatterns

Monitoruserbehaviordynamicallytominimizesystem loadandnetworktraffic e.gpopulardatacanbedistributedclosetousers Handlestreamingmedia,billingandaccount transactions,andinterprocesscommunication Servermustbeaffordable

Handledifferenttypesoftraffic

CostEffectiveness

Dr. Vidhya Balasubramanian Amrita School of Engineering

MMDBDesignStrategies

3waystorealizeamultimediadatabase

Usingtraditionaldatabaseswherecertainitemsreferto filesofmediadatasothattheseactaspointers Useobjectorientedorobjectrelationaldatabaseswith multimediadatatypes

Datastoredondifferentservers Tobefiguredthisoutbytheendoftheclass!!

Usefullyintegratedmultimediadatabase

Dr. Vidhya Balasubramanian Amrita School of Engineering

DistributedSystems

Dataspreadovermultiplemachines(alsoreferred toassitesornodes). Networkinterconnectsthemachines Datasharedbyusersonmultiplemachines

Dr. Vidhya Balasubramanian Amrita School of Engineering

DistributedDatabases

Homogeneousdistributeddatabases

Samesoftware/schemaonallsites,datamaybe partitionedamongsites Goal:provideaviewofasingledatabase,hidingdetails ofdistribution Differentsoftware/schemaondifferentsites Goal:integrateexistingdatabasestoprovideuseful functionality Alocaltransactionaccessesdatainthesinglesiteat whichthetransactionwasinitiated. Aglobaltransactioneitheraccessesdatainasite Dr. Vidhya Balasubramanian differentfromtheoneatwhichthetransactionwas Amrita School of Engineering

Heterogeneousdistributeddatabases

Differentiatebetweenlocalandglobaltransactions

HomogeneousDistributedDatabases

Allsiteshaveidenticalsoftware Areawareofeachotherandagreetocooperatein processinguserrequests. Eachsitesurrenderspartofitsautonomyinterms ofrighttochangeschemasorsoftware Appearstouserasasinglesystem

Dr. Vidhya Balasubramanian Amrita School of Engineering

HeterogenousDistributedDatabases

Differentsitesmayusedifferentschemasand software

Differenceinschemaisamajorproblemforqueryprocessing Differenceinsoftwareisamajorproblemfortransaction processing

Sitesmaynotbeawareofeachotherandmay provideonlylimitedfacilitiesforcooperationin transactionprocessing

Dr. Vidhya Balasubramanian Amrita School of Engineering

ADistributedMultimediaSystem
Video camera and mike

Local network

Local network

Wide area gateway

Video server

Digital TV/radio server

Distributed Systems: Concepts and Design: Instructors Guide

Dr. Vidhya Balasubramanian Amrita School of Engineering

ExampleofDMS

http://www.ercim.eu/publication/Ercim_News/enw62/boeszoermenyi.gif

Dr. Vidhya Balasubramanian Amrita School of Engineering

Example:DistributedVoDServer

http://hpc.ee.ntu.edu.tw/~murphy/homepage_2000/Reports/MMnet/final/Image15.jpg
Dr. Vidhya Balasubramanian Amrita School of Engineering

DistributedMultimediaDatabase Systems
DBMS Presentation Device DBMS Network A Network B DBMS Audio Audio Audio DBMS Image

DBMS

Video

DBMS

Text

Dr. Vidhya Balasubramanian Amrita School of Engineering

Src: notes from Arif Ghafoor

27

ArchitectureofaDistributed MultimediaDatabaseManagement
Multimedia Database Client
Visual Tool for Multimedia Document Generation Multimedia Database Interface Multimedia Presentation Subsystem

API for SBS Network

Integrated Multimedia Information Server Multimedia Meta Data Management

API for SBS Network Multimedia Database Server

Database Management System

Distributed Query Processor

Directory Management

Meta Data

Media Server Subsystem

API for SBS Network

Database Connectivity

...

Text

Image

Video

Audio

Multimedia Database Server

Multimedia Database Server

Dr. Vidhya from Arif Ghafoor Src: notes Balasubramanian Amrita School of Engineering

28

AdvantagesofDDB

Modularity FaultTolerance HighPerformance DataSharing LowCost

Improvedavailability Easierandcheaperexplansion

Reliabilityandavailability Transparentmanagementofdistributeddata
Dr. Vidhya Balasubramanian Amrita School of Engineering

Disadvantages

Complexity

Databasedesignandmanagementmorecomplex Fragmentationofmultimediadataacrossdifferent serversmorecomplex

Cost Security Integrity

Difficulttomaintain

Dr. Vidhya Balasubramanian Amrita School of Engineering

Transparency

Transparencyistheseparationofthehigherlevel semanticsofasystemfromthelowerlevel implementationissues. Providedataindependence

Networktransparency Replicationtransparency Fragmentationtransparency Thiscanbespreadacrossmultipledatabases,butuser doesnotknowthis

Distributed DBMS 1998 M. Tamer zsu & Patrick Valduriez Dr. Vidhya Balasubramanian
Amrita School of Engineering

Example:AccessemployeeinformationofIBM

DistributedDatabaseDesign

GeneralIssues

DistributionofData

replicated&nonreplicateddatabasedistribution

Directorymanagement synchronizationofconcurrentaccesses consistencyandisolationoftransactions'effects deadlockmanagement atomicityanddurability

Dr. Vidhya Balasubramanian Amrita School of Engineering

ConcurrencyControl

FaultToleranceandRecovery

Heterogeneity

ScalabilityIssues

Increasingnumberofissues

Moreusersleadstonetworktrafficload Betternetworkprotocolsmustbeused StrainsnetworkandI/Ocapacitylimitingnumberof requestssupported

Sizeofdataobjects

Amountofaccessibledataincreasing Nonuniformrequestdistribution

Systemloadvariesduringthedayorbasedonlocations
Dr. Vidhya Balasubramanian Amrita School of Engineering

DistributedDataStorage

Assumerelationaldatamodel Replication

Systemmaintainsmultiplecopiesofdata,storedin differentsites,forfasterretrievalandfaulttolerance. Relationispartitionedintoseveralfragmentsstoredin distinctsites Relationispartitionedintoseveralfragments:system maintainsseveralidenticalreplicasofeachsuch fragment.

Dr. Vidhya Balasubramanian Amrita School of Engineering

Fragmentation

Replicationandfragmentationcanbecombined

DataReplication

Arelationorfragmentofarelationisreplicatedif itisstoredredundantlyintwoormoresites. Fullreplicationofarelationisthecasewherethe relationisstoredatallsites. Fullyredundantdatabasesarethoseinwhichevery sitecontainsacopyoftheentiredatabase.

Dr. Vidhya Balasubramanian Amrita School of Engineering

AdvantagesofReplication

DataReplication(ProsandCons)

Availability:failureofsitecontainingrelationrdoesnotresultin unavailabilityofrifreplicasexist. Parallelism:queriesonrmaybeprocessedbyseveralnodesin parallel. Reduceddatatransfer:relationrisavailablelocallyateachsite containingareplicaofr. Increasedcostofupdates:eachreplicamustbeupdated. Increasedcomplexityofconcurrencycontrol

DisadvantagesofReplication

concurrentupdatestodistinctreplicasmayleadtoinconsistentdata Onesolution:chooseonecopyasprimarycopyandapplyconcurrency controloperationsonprimarycopy

Dr. Vidhya Balasubramanian Amrita School of Engineering

Divisionofrelationrintofragmentsr1,r2,,rnwhich containsufficientinformationtoreconstructrelation r. Horizontalfragmentation

DataFragmentation

eachtupleofrisassignedtooneormorefragments theschemaissplitintoseveralsmallerschemas Allschemasmustcontainacommoncandidatekey(or superkey)toensurelosslessjoinproperty. Aspecialattribute,thetupleidattributemaybeaddedto eachschematoserveasacandidatekey.

Dr. Vidhya Balasubramanian Amrita School of Engineering

Verticalfragmentation

HorizontalFragmentation
branch_name Hillside Hillside Hillside account_number A-305 A-226 A-155 balance 500 336 62

account1 = branch_name=Hillside (account ) branch_name Valleyview Valleyview Valleyview Valleyview account_number A-177 A-402 A-408 A-639 balance 205 10000 1123 750

account2 = branch_name=Valleyview (account )

Dr. Vidhya Balasubramanian Amrita School of Engineering

VerticalFragmentation
branch_name customer_name tuple_id Lowman 1 Hillside Camp 2 Hillside Camp 3 Valleyview Kahn 4 Valleyview Kahn 5 Hillside Kahn 6 Valleyview Green 7 Valleyview deposit1 = branch_name, customer_name, tuple_id (employee_info ) account_number balance tuple_id

500 A-305 1 336 A-226 2 205 A-177 3 10000 A-402 4 62 A-155 5 1123 A-408 6 750 A-639 7 deposit2 = account_number, balance, tuple_id (employee_info )
Dr. Vidhya Balasubramanian Amrita School of Engineering

Horizontal:

Advantages

allowsparallelprocessingonfragmentsofarelation allowsarelationtobesplitsothattuplesarelocatedwhere theyaremostfrequentlyaccessed

Vertical:

allowstuplestobesplitsothateachpartofthetupleisstored whereitismostfrequentlyaccessed tupleidattributeallowsefficientjoiningofverticalfragments allowsparallelprocessingonarelation Fragmentsmaybesuccessivelyfragmentedtoanarbitrary depth. Dr. Vidhya Balasubramanian

Amrita School of Engineering

Verticalandhorizontalfragmentationcanbemixed.

DegreeofFragmentation

Fragmentationcanoccuratdifferentlevels Findsuitablelevelofpartitioning

tuples or attribute s

relation s

Dr. Vidhya Balasubramanian Amrita School of Engineering

Completeness

CorrectnessofFragments

DecompositionofrelationRintofragmentsR1,R2,...,Rn

iscompleteifandonlyifeachdataiteminRcanalsobe foundinsomeRi

Reconstruction

IfrelationRisdecomposedintofragmentsR1,R2,...,Rn, thenthereshouldexistsomerelationaloperatorsuch thatR=1inRi

Disjointness

IfrelationRisdecomposedintofragmentsR1,R2,...,Rn, anddataitemdiisinRj,thendishouldnotbeinany otherfragmentRk(kj).

Dr. Vidhya Balasubramanian Amrita School of Engineering

AllocationAlternatives

Nonreplicated

partitioned:eachfragmentresidesatonlyonesite fullyreplicated:eachfragmentateachsite partiallyreplicated:eachfragmentatsomeofthe sites

read only queries 1 replicationisadvantageous If update queries

Replicated

Ruleofthumb:

Elsereplicationcancauseproblems
Dr. Vidhya Balasubramanian Amrita School of Engineering

ComparisonofReplicationAlternatives
FullReplication PartialReplication Partitioning QueryProcessing Easy SameDifficulty DirectoryManagement EasyorNonexistent SameDifficulty ConcurrencyControl Moderate Difficult Easy Reliability VeryHigh High Low Reality PossibleApplication Realistic PossibleApplication

Dr. Vidhya Balasubramanian Amrita School of Engineering

DistributioninDMS

Optimizedataallocationsothat

Datasituatedwheretheyaremostneeded Minimizenetworktraffic Improvereliabilityalsowhenonenodefails InVoDsystemssincedataisntupdatedoftenreplication ispreferrable Improvesperformance

Replicationhelpsminimizenetworktraffic

User'srequestforvideocanbedirectedtoserverwithlightest load
Dr. Vidhya Balasubramanian Amrita School of Engineering

StorageManagement

Storagehierarchiescanbeused

Tosupportdistributedandheterogeneousdatabases

Supportuseraccesstovareityofinformation: layeredaccess

Quickerretrieval

Staging Clustering Caching

Dr. Vidhya Balasubramanian Amrita School of Engineering

LoadBalancinginDistributedMultimediaServers

SelfStudy

Statetheneedforloadbalancing. Whataretheissuesinloadbalancing. Givetwotechniquesusedforloadbalancing.

DistributedvsClientServerArchitecture PeertoPeernetworks

Whatisap2pnetwork? Howaretheyusedformultimediaapplications? Challengesinp2pnetworks

Submitreport:3perteam,eachpersonspecify role Dr. Vidhya Balasubramanian

Amrita School of Engineering

ArchitecturesforContentOrganization

PrincipleofAutonomy

Groupallimages,video,documentsandindexeachof them

Theindexingmustbemaximallyefficientfortheaccessrequirements Ensureseachmediatypesisorganizedinamediaspecificmannersuitablefor thatmediatype

Issues:

Requirescreationofalgorithmsanddatastructuresforeachmediatype Joinsneededacrossdifferentdatastructureswhichisexpensive

Advantages:

FastqueryprocessingtimesusingspecializedDSthatindexeachdata efficiently SuitableforlegacydatawherealgorithmsandDSalreadyexist

Dr. Vidhya Balasubramanian Amrita School of Engineering

PrincipleofAutonomy

Dr. Vidhya Balasubramanian Amrita School of Engineering

PrincipleofUniformity
FindasingleabstractstructureAthatcanindexallthe mediatypes

Formsanunifiedindextoaccessdifferentmedia Usesingledatastructure

Issues

Requiresabstractionofcommonpartofallmediatypes

Usedwidelyinindustry Donethroughannotationandmetadata,wheremetadataindexed

Creationofannotationmanuallyorautomaticallydifficult Informationmaybelostofannotationnotexpresiveenough

Dr. Vidhya Balasubramanian Amrita School of Engineering

PrincipleofUniformity

Dr. Vidhya Balasubramanian Amrita School of Engineering

PrincipleofHybridOrganization

Takesadvantagesofgoodfeaturesofbothtechniques

Mediatypesthatarelegacysourcesandhaveexisting index,weuseprincipleofautonomy Formediatypesthatarenotlegacysourcesandhaveno existingindex,uniformrepresentationused

Unlessoflossofdetailedinformationliketextureofpixelsetcis nottolerable

Codetoperformjoinsacrossmultipledatasources

Leveragesexistingcodeandadditionalcodeeffortminimum Domainspecificindexescanbeused Multiindexjoins

Dr. Vidhya Balasubramanian Amrita School of Engineering

PrincipleofHybridOrganization

Dr. Vidhya Balasubramanian Amrita School of Engineering

SoftwareArchitectureofaMultimedia DatabaseManagementSystem
User
To Presentation Device
Multimedia Structuring Module Document Generator Tool Library

HyTime/SGML/XML Translator Multimedia Meta-Data HyTime/SGMLXML Parser --> Multimedia Query Language

Src: notes from Arif Ghafoor

OODBMS and MQL Distributed Query Processor

Temporal Synchronization Manager

Text Database

Video Database

Image Database

Audio Database 54

Dr. Vidhya Balasubramanian Amrita School of Engineering