9/29/2016

EverythingYouAlwaysWantedtoKnowAboutSDRAM(Memory):ButWereAfraidtoAskPrintView

OriginalLink:http://www.anandtech.com/show/3851/everythingyoualwayswantedtoknowaboutsdram
memorybutwereafraidtoask

Everything You Always Wanted to Know About

SDRAM (Memory): But Were Afraid to Ask

47

Comments

by Rajinder Gill on August 15, 2010 10:59 PM EST

Postedin Memory

Guides

Intel

Itscominguponayearsincewepublishedourlastmemoryreview;possiblythelongesthiatusthissection
ofthesitehaseverseen.Tobehonest,thereasonweverefrainedfrompostingmuchofanythingisbecause
thingshaventchangedallthatmuchoverthelastyearbarringanecessaryshifttowardslowvoltage
orientedICs(~1.30Vto~1.50V)fromthelikesofElpidaandPSC.Partsofthesetypeswilleventuallybecome
thenormasmemorycontrollersbasedonsmallerandsmallerprocesstechnology,likeIntels32nmGulftown,
gaintractioninthemarket.
Whilevoltagerequirementshavechangedforthebetter,factorsrelatingtoimportantmemorytimingslikeCL
andtRCDhaventseenanimprovement;werealmostatthesamepointwewereayearago.Backthen
ElpidaprovidedaglimpseofpromisewiththeirHyperseriesofICs.TheHyperpartwascapableofhigh
speed,lowlatencyoperationintandem.Unfortunately,duetoproblemswithlongtermreliability,Hyperisnow
defunct.CorsairandperhapsMushkinstillhaveenoughstocktosellforawhile,butonceit'sgone,thatsit.

CorsairDominatorGTsbasedonElpidaHyperthey'rebeingphasedoutforsomethingslower...
ThesupersedingElpidaBBSEvariantICsandaspreadofchipsfromPSCnowdominatethememoryscene,
rangingfrommainstreamDDR31333speedsallthewaytoinsanelyratedpremiumDDR32500kits.Some
ofthesepartsarecapableofkeepingupwithHyperwhenitcomestoCL,butdosobyaddingafew
nanosecondsofrandomaccesslatencyduetoaloosertRCD.Giventhatreadandwriteaccessoperations
makeupasignificantportionofmemorypowerconsumption,thisstepbackwardsinperformancemaybea
requisitefactorforreliabilityperhapssomethingwasfoundbyElpidaduringtheproductionlifetimeofHyper
ICsthatpromptedareexamination,leadingtoamoreconservativerecipefordatatransfer/retrieval.

http://www.anandtech.com/print/3851/everythingyoualwayswantedtoknowaboutsdrammemorybutwereafraidtoask

1/17

9/29/2016

EverythingYouAlwaysWantedtoKnowAboutSDRAM(Memory):ButWereAfraidtoAskPrintView

Afewofthenewermodulestograceourdoorstep
Todaysmemorysectioncomebackwasfuelledbythearrivalofanumberofmainstreammemorykitsatour
testlabsmanyofthekitswewereusingformotherboardreviewsarenolongerforsalesoweneededto
updateourinventoryofmodulesanyway.Corsair,CrucialandGSkillkindlysentmemoryfromtheir
mainstreamlineups.Theoriginalintentwastolookatafewofthosekits.
However,duringthecourseoftestingthesekits,ourfocusshiftedfromwritingamemoryreview(showingthe
sameoldboringgraphs)tocompilingsomethingfarmoremeaningful:aguidetomemoryoptimizationand
addressing,includingadetailedlookatimportantmemorytimings,andanaccountingofsomeofIntels
lesserknownmemorycontrollerfeatures.Assuch,thisarticleshouldmakeaverycompellingreadforthose
ofyouinterestedinlearningmoreaboutsomeofthedesignandengineeringthatgoesintomakingmemory
work,andhowalittleunderstandingcangoalongwaywhenlookingforcreativewaystoimprovememory
performance

Synchronousdynamicrandomaccessmemory(SDRAM)ismadeupofmultiplearraysofsinglebitstorage
sitesarrangedinatwodimensionallatticestructureformedbytheintersectionofindividualrows(WordLines)
andcolumns(BitLines).Thesegridlikestructures,calledbanks,provideanexpandablememoryspace
allowingthehostcontrolprocessandothersystemcomponentswithdirectaccesstomainsystemmemoryto
temporarilywriteandreaddatatoandfromacentralizedstoragelocation.
Whenassociatedingroupsoftwo(DDR),four(DDR2)oreight(DDR3),thesebanksformthenexthigher
logicalunit,knownasarank.2GBDDR3DualInlineMemoryModules(DIMM)areundoubtedlythemost
populardensitychoiceamongtoday'senthusiastusers.Mostnewpartsofthistypeareconfiguredastwo
identicalranksofeightbankseach;onesideoftheDIMMhousingthoseICsthatmakeupRank1,withRank
2populatingtheoppositefaceofthemodule.Forthisreason,singlesidedDIMMstypicallycompriseonlya
singlerankofaddressablememoryspace.

http://www.anandtech.com/print/3851/everythingyoualwayswantedtoknowaboutsdrammemorybutwereafraidtoask

2/17

9/29/2016

EverythingYouAlwaysWantedtoKnowAboutSDRAM(Memory):ButWereAfraidtoAskPrintView

Figure1.TypicalfunctionalarrangementofSDRAMmemoryspace.OneBankonlyisshownforclarity
Figure1showsthetypicalfunctionalarrangementofSDRAMmemoryspace.Inthecaseofourexample
dualsideddualrankunbuffered2GBSDRAMDIMM,thefullypopulatedmodulecontainsatotalof16ICs,
eightperside.EachICcontainseightbanksofaddressablememoryspacecomprising16Kpagesand1K
columnaddressstartingpointswitheachcolumnstoringasingle8bitword.Thisbringsthetotalmemory
spaceto128MB(16,384rows/bankx1,024columnsaddresses/rowx1byte/columnaddressx8stacked
banks)perIC.AndsincethereareeightICsperrank,Rank1is1GB(128MBx8contiguousbanks)insize,
withthesameforRank2,foragrandtotalof2GBpermodule.
Ifeachrowcontains1K(1,024)columnaddressstaringpointsandeachcolumnstores8bits(1byte),this
wouldmeaneachrow(page)is8,192bits(1,024x8bits)or1Kbytesperbank.It'simportanttounderstand
thateachpageofmemoryissegmentedevenlyacrossBanknofeachICfortheassociatedrank.Forthis
reason,eachpageisinactuality8KB(1KBx8contiguousbanks)insize.SowhenwetalkaboutICdensity
wearereferringtoeightdistinctstackedbanksandthetotalmemoryspacetherein,whereaswhenwetalk
aboutpagespace,wearereallyworkingwithBanknspreadacrossthetotalnumberofICsperrank.Inthe
endthemathcomesoutthesame(8ICsversus8banks),butconceptuallyit'sacriticaldistinctionworth
acknowledgingifwearetoreallygrasptheinsandoutsofmemoryaddressing.
WecannowseewhytheDDR3corehasa8nprefetch(wherenreferstothenumberofbanksperrank)as
everyreadaccesstothememoryrequiresaminimumof64bits(8bytes)ofdatatobetransferred.Thisis
becauseeachbank,ofwhichthereareeightforDDR3,fetchesnolessthan8bits(1byte)ofdataperread
requesttheequivalentofonecolumn'sworthofdata.Whetherornotthesystemactuallymakesuseofall8
bytesoftransferreddataisirrelevant.Anydelivereddatanotactuallyrequestedcanbesafelydisregardedas
it'sjustacopyofwhatisstillretainedinmemory.

http://www.anandtech.com/print/3851/everythingyoualwayswantedtoknowaboutsdrammemorybutwereafraidtoask

3/17

9/29/2016

EverythingYouAlwaysWantedtoKnowAboutSDRAM(Memory):ButWereAfraidtoAskPrintView

SDRAMcan,inmanyways,bebestdescribedasasimplestatemachine(Figure2)whichiseitheridle,
active,orprechargingoneormoreopenbanks.Aswithanymachine,transitionfromonestatetoanother
requiresaminimumwaittimebeforethesystemisreadytorespondtosubsequentrequeststodoadditional
work.ThesedelayshaveamajorimpactonSDRAMreadandwriteperformance,andmoreimportantly,
performanceofthesystemasawhole.
SinceSDRAMmemorycellsarereallyjustminiaturecapacitors,thechargetheycontainwilldissipateaway
naturallyovertimeduetomanyfactorsthatcaninfluencetheleakagerate,includingtemperature.Amarked
reductioninstoredchargecanresultineitherdatalossordatacorruption.Inordertopreventthisfrom
happeningSDRAMmustbeperiodicallyrefreshedbytoppingoffthechargecontainedineachindividual
memorycell.Thefrequencywithwhichthisrefreshneedoccurdependsonthesilicontechnologyusedto
manufacturethecorememorydieandthedesignofthememorycellitself.
ReadingorwritingtoamemorycellhasthesameeffectasrefreshingtheselectedcellbyissuingaRefresh
(REF)command.Unfortunately,notallcellsarereadfromorwrittentoduringthenormalcourseofoperation
andsoeachcellinthearraymustbeaccessedandwrittenback(restored)beforetheexpirationofthe
refreshinterval.Inmostcases,refreshcyclesinvolverestoringthechargealonganentirepage.Overthe
courseoftheentireinterval,everypageisaccessedandsubsequentlyrestored.Attheendoftheinterval,the
processbeginsagain.AtypicalRefreshPeriod(tREF)ishundredstopossiblyathousandormoreclocks.
AllbanksmustbeprechargedandidleforaminimumoftheRASPrecharge(tRP)delaybeforetheRefresh
(REF)commandcanbeapplied.Anaddresscounter,internaltothedevice,suppliesthebankaddressused
duringthecourseoftherefreshcycle.Whentherefreshcyclehascompleted,allbanksareleftinthe
precharged(idle)state.AdelaybetweentheREFcommandandthenextActivate(ACT)commandor
subsequentREFcommandmustbegreaterthanorequaltotheRowRefreshCycleTime(tRFC).Inother
words,aminimumwaitoftRFCcyclesisrequiredfollowingarefreshtoanidlebankbeforeitcanbeagain
activatedforaccess.

http://www.anandtech.com/print/3851/everythingyoualwayswantedtoknowaboutsdrammemorybutwereafraidtoask

4/17

9/29/2016

EverythingYouAlwaysWantedtoKnowAboutSDRAM(Memory):ButWereAfraidtoAskPrintView

Figure2.AsimplifiedSDRAMstatetransitiondiagram.Don'tstaretoolong
BeforetheSDRAMisreadytorespondtoreadandwritecommands,abankmustfirstbeopened(activated).
Thememorycontrolleraccomplishesthisbysendingtheappropriatecommand(ACT),specifyingtherank,
bank,andpage(row)tobeaccessed.ThetimetoactivateabankiscalledtheRowColumn(orCommand)
DelayandisdenotedbythesymboltRCD.Thisvariablerepresentstheminimumtimeneededtolatchthe
commandatthecommandinterface,programthecontrollogic,andreadthedatafromthememoryarrayinto
theSenseAmplifiersinpreparationforcolumnlevelaccess.
Followingactivation,theopenbankcontainswithinthearrayofSenseAmpsacompletepageofmemoryonly
8KBinlength.Atthistime,multipleRead(READ)andWrite(WRI)commandscanbeissued,specifyingthe
startingcolumnaddresstobeaccessed.Thetimetoreadabyteofdatafromtheopenpageiscalledthe
ColumnAddressStrobe(CAS)LatencyandisdenotedbythesymbolCLortCAS.Thisvariablerepresents
theminimumtimeneededtolatchthecommandatthecommandinterface,programthecontrollogic,gate
therequesteddatafromtheSenseAmpsintotheInput/Output(I/O)Buffers,throughaprocessknownaspre
fetching,andplacethefirstwordofdataontheMemoryBus.
Onlyonepageperbankmaybeopenatatime.Accesstootherpagesinthesamebankdemandstheopen
pagefirstbeclosed.Aslongasthepageremainsopenthememorycontrollercanissueanycombinationof

http://www.anandtech.com/print/3851/everythingyoualwayswantedtoknowaboutsdrammemorybutwereafraidtoask

5/17

9/29/2016

EverythingYouAlwaysWantedtoKnowAboutSDRAM(Memory):ButWereAfraidtoAskPrintView

READorWRIcommands,sometimesswitchingbackandforthbetweenthetwo,untilsuchtimeastheopen
pageisnolongerneededorapendingrequesttoread/writedatafromanalternatepageinthesamebank
requiresthecurrentpagebeclosedsothatanothermaybeaccessed.Thisisdonebyeitherissuinga
Precharge(PR)commandtoclosethespecifiedbankonlyoraPrechargeAll(PRA)commandtocloseall
openbanksintherank.
Alternatively,thePrechargecommandcanbeeffectivelycombinedwiththelastreadorwriteoperationtothe
openbankbysendingaReadwithAutoPrecharge(RDA)orWritewithAutoPrecharge(WRA)commandin
placeofthefinalREADorWRIcommand.ThisallowstheSDRAMcontrollogictoautomaticallyclosethe
openpageassoonascertainconditionshavebeenmet:(1)AminimumofRASActivationTime(tRAS)has
elapsedsincetheACTcommandwasissued,and(2)aminimumofReadtoPrechargeDelay(tRTP)has
elapsesincethemostrecentREADcommandwasissued.
PrechargingpreparesthedatalinesandsensecircuitrytotransmitthestoredchargeintheSenseAmpsback
intotheopenpageofindividualmemorycells,undoingthepreviousdestructiveread,makingtheDRAMcore
readytosamplethenextpageofmemorytobeaccessed.ThetimetoPrechargeanopenbankiscalledthe
RowAccessStrobe(RAS)PrechargeDelayandisdenotedbythesymboltRP.Theminimumtimeinterval
betweensuccessiveACTcommandstothesamebankisdeterminedbytheRowCycleTimeofthedevice,
tRC,foundbysimplysummingtRASandtRP(tobedefined).TheminimumtimeintervalbetweenACT
commandstodifferentbanksistheReadtoReadDelay(tRRD).

TheprocessofmovingdatainandoutoftheMemoryArrayandovertheMemoryBusisnotoverly
complicated,althoughthemassiveparallelizationoftheactualeffortcanmakeitsomewhatdifficulttofully
envisionwhat'sreallyhappeningwithoutsomeprettyconcisevisualaids.We'lltryourbesttohelpyououtin
thisregard.
BothreadandwriteaccesstoDDR[3]SDRAMisburstoriented;accessstartsataselectedlocationand
continuesinapreprogrammedsequenceforaBurstLength(BL)of8bits,or1byte,perbank.Thisbegins
withtheregistrationofanACTcommandandisfollowedbyoneormoreREADorWRIcommands.
ChipSelect(S0#,S1#),oneforeachrank,eitherenables(LOW)ordisables(HIGH)thecommanddecoder
whichworkslikeamasktoensurecommandsareacteduponbythedesiredrankonly.
ThelengthoftheeachReadBurst(tBurst)isalways4clocks(4T)asDDRmemorytransmitsdataattwice
thehostclockrate(4clocksx2transactions/clock=8transactionsor8bitsperbank).
TheaddressbitsregisteredcoincidentwiththeACTcommandareusedtoselectthebankandpage(row)to
beaccessed.Forourhypothetical2GBDIMMdescribedonPage2ofthisarticle,BankSelectsBA0BA2
indicatethebankandAddressInputSelectsA0A13indicatethepage.Threebitsareneededtouniquely
addressalleightbanks;likewise14bitsareneededtoaddressall16,384(214)pages.
TheaddressbitsregisteredcoincidentwiththeREADorWRIcommandareusedtoselectthetargeted
startingcolumnfortheburst.A0A09selectthecolumnstartingaddress(210=1,024).A12isalsosampled
duringthisoperationtodetermineifaBurstChop(BC)of4bitshasbeencommanded(A12HIGH).Even
thoughaBurstChopdeliversonlyhalfthatdataofaregularReadBurst,thetimeperiodtocompletethe
transferisstillthesame:4T.TheSDRAMcoresimplymaskstheoutgoingdataclockstrobeforthesecond
halfofthefullreadcycle.

http://www.anandtech.com/print/3851/everythingyoualwayswantedtoknowaboutsdrammemorybutwereafraidtoask

6/17

9/29/2016

EverythingYouAlwaysWantedtoKnowAboutSDRAM(Memory):ButWereAfraidtoAskPrintView

Figure3.Memoryreadandwriteoperationscanbebrokendownintoaseriesofwelldefinedevents
DuringaPrechargecommandA10issampledtodeterminewhethertheprechargeisintendedforonebank
(A10LOW;BAselects)orallbanks(A10HIGH).
DataInput/OutputpinsDQ0DQ63providethe64bitwidedatainterfacebetweenthememorycontroller
embeddedintheCPUandeachDIMM.ThosewithatriplechannelcapableCPU,liketheIntelCorei7series
processor,willcometounderstandwhythememorybuswidthisreportedas192bitthreeindependently
operatedchannelseachwitha64bitinterfacemakes192.ThoseofyourunningaCore2oraCorei3/i5will
havetomakedowithjusttwochannelsforatotalbuswidthof128bits.
EachchannelcanbepopulatedwithuptotwoDIMMs.Thismeanstherecouldbeamaximumoffourranks
perchannel,assumingweinstallamatchedpairofdualrankmodules.InstallingmorethanoneDIMMper
channeldoesnotdoubletheMemoryBusbandwidth,asmodulescolocatedinthesamechannelmust
competeforaccesstoashared64bitsubbus;however,addingmoremodulesdoeshavetheaddedbenefit
ofdoublingthenumberofpagesthatmaybeopenconcurrently(twicetheranksfortwicethefun!).
Figure3attemptstoprovideatopdownlookattheminimumcycleneededtofirstopenapageinmemory,
andthenreaddatafromtheactivatedpage;Figure4showsthesame,onlyfromamuchmorefundamental
perspective;andFigure5providesadetailedaccountingofthetiminginvolved.

http://www.anandtech.com/print/3851/everythingyoualwayswantedtoknowaboutsdrammemorybutwereafraidtoask

7/17

9/29/2016

EverythingYouAlwaysWantedtoKnowAboutSDRAM(Memory):ButWereAfraidtoAskPrintView

Figure4.Nowitallmakessense!(punintended)
Inthisexampleweassumethebankhasnoopenpage,thusisalreadyintheproperprechargedstateto
supportanewpageaccesscommand.Step1selectsthebank;Step2selectsthecolumn;andStep3bursts
thedataoutovertheMemoryBus.A1bitrowaddressanda2bitcolumnaddressareallweneedtoread
anydatastoredinour2x4bitx1(bank)MemoryArray.
AnActivatecommandpromptstheroutingofthespecifiedpageaddresstobeaccessedtotheRowDecoder
whereittriggersthechosenWordLinetoappearattheinputoftheSenseAmps.Aspreviouslystated,this
takesafinitetimeRowColumn(orCommand)Delay(tRCD)isusedtoprogramtheminimumwaittimethe
memorycontrollerallowsforthistooccurbeforeitissuesthenextcommandinthesequence.Attemptingto
settoolowatimingcanleadtoinconclusiveoperation,oftenresultingindatacorruptionandotherdata
accessissuesthatultimatelyleadtosystemcrashesandotherapplicationerrors.
Next,thecolumnaddressprovidedwiththeReadcommandselectstherightBitLine,beginningtheprocess
ofdisregardingthosebitsthatwerenotaddressed.ThewaitassociatedwiththeseeventsistheCASLatency
(CLortCAS).
TheSenseAmpsworkbysensingthedirectionofthevoltageswinginducedonthesenselinewhentheWord
Lineisactivated.Activatingthepagegatesontheswitchingelementholdingbacktheaccumulatedchargein
atrenchfilledwithdielectricmaterialusedtocreatethecapacitivestorageelementofthememorycell.When
thishappensthesenseline,startingfromVRefDQ(VDDQ),eitherswingspositiveornegative,depending
onthepotentialofthesampledmemorycell.Anincreaseinvoltageencodesa1,whileadecreasemeans0.

http://www.anandtech.com/print/3851/everythingyoualwayswantedtoknowaboutsdrammemorybutwereafraidtoask

8/17

9/29/2016

EverythingYouAlwaysWantedtoKnowAboutSDRAM(Memory):ButWereAfraidtoAskPrintView

Clicktoenlarge
Figure5.Shownherearepairof"backtoback"reads.OurexampleRowCycleTime(tRC)letsustransferup
to16bytesofdatawithaminimumPageopentimeof24TusingCLtRCDtRPtRAStimingsof66618
TheSenseAmpsarenotcomparators.Rather,eachSenseAmpinterfaceswithapairofmemorycells,
reducingthetotalnumberofamplifiersneededtootherwisesensetheentirearraybyafactoroftwo.
Followingtheread,anychargestoredinthememorycellsisobliterated.Thisiswhatismeantbya
destructiveread:notonlydotheSenseAmpscachethepageforaccess,theynowholdtheonlyknowncopy
ofthatpageofmemory!PrechargingthebankwillforcetheSenseAmpsto"write"thepagebacktothearray
andwillpreparethesenselinesforthenextpageaccessby"precharging"themtoVDDQ.This
accomplishestwothing:(1)Itreturnsallsenserailstoaknown,consistentpotential,and(2)itsetsthepre
senselinevoltageatexactlyhalfthefullscalevalueofVDDQ,ensuringwhateverthepotentialstoredinthe
cell,therewillbeaswinginvoltagewhentheproperWordLineisactivated.

Everyread/writememorytransactioncanbesegmentedbytypeintooneofthreeperformancebins
dependingonthestatusofthebank/pagetobeaccessed.Thesebins,inorderofbesttoworst,arepagehit,
pageempty,andpagemiss.Forthemostpart,anythingwecandotoincreasethenumberofpagehit
transactionsorreducethenumberofpagemisstransactionsisagoodthing.
Apagehitaccessisdefinedasanyreadorwriteoperationtoanopenpage.Thatis,thebankcontainingthe
openpageisalreadyactiveandisimmediatelyreadytoservicerequests.Becausethetargetpageisalready
open,thenominalaccesslatencyforanymemorytransactionfallingintothiscategoryisapproximatelytCAS
(theCASLatencyofthedevice).

Clicktoenlarge
Figure6.Pagehittiming(withprechargeandsubsequentbankaccess)
Figure6showstheminimumreadlatencyassociatedwithabestcasepagehitscenario.Forapartwitha
CASLatencyof6T,thememorycontrollerwaitsonlysixshortclocksbeforethestartofdatareturn.Duringa
ReadwithAutoPrecharge,theReadcommandwillexecuteasnormalexcepttheactivebankwillbegin
prechargingCASlatency(CL)clockcyclesbeforetheendoftheburst.Thisfeatureallowstheprecharge
operationtobepartiallyorcompletelyhiddenduringperiodsofburstreadcycles,dependentonCL.When
tuningoursystemswealwaysseektosettRTPsuchthattRTP+tRPequalsCL+tBurstforexactlythis
reason.Putanotherway,ifCLandtRParethesameset4TforDDR3(2TforDD2).
Sequentialreadstothesamepagemakethesetypesoftransactionsevenmoreprofitableaseach
successiveaccesscanbescheduledataminimumoftBurst(4T)clocksfromthelast.Thetimingiscaptured
astheCAStoCASDelay(tCCD)andiscommonlyreferredtoas'BacktoBackCASDelay'(B2B),asshown
perFigure7.Thisfeaturemakespossibleextremelyhighdatatransferratesfortotalburstlengthsofone
pageorlessinourcase,8KB.

http://www.anandtech.com/print/3851/everythingyoualwayswantedtoknowaboutsdrammemorybutwereafraidtoask

9/17

9/29/2016

EverythingYouAlwaysWantedtoKnowAboutSDRAM(Memory):ButWereAfraidtoAskPrintView

Clicktoenlarge
Figure7.TripleBurstChopreadwithprechargeandsubsequentbanksaccess
Althoughnotideal,apageemptyaccessisstillpreferredtoamiss.Inthiscasethebanktobeaccessedis
Idlewithnopageopen.Commonsensetellsusanyattempttoreadorwritedatatoapageinthisbankfirst
requiresweActivatethebank.Inotherwords,nominalaccesslatencynowincludesthetimetoopenthe
pageRowColumn(orCommand)Delay(tRCD).Thisisadoublingoftheminimumaccesslatencywhen
comparedtothatofthepagehitcase!Twelvecycles(tRCD+CL)nowelapsebeforethefirstwordis
returned.Figure8showsthisthedetail.

Clicktoenlarge
Figure8.Pageemptytiming.Pageremainsopen
Finally,asiftherelativepenaltyofpageemptyaccesswasn'tbadenough,herecomespagemiss.Amiss
occursanytimeamemorytransactionmustfirstcloseanopenpageinordertoopenanalternatepageinthe
samebank.Onlythencanthespecifieddataaccesstakeplace.Firstclosinganopenpagerequiresa
Precharge,addingtheRASPrecharge(tRP)delaytoanyalreadylengthyoperation.Asyoucanseeby
Figure9,thenominallatencyofanaccessofthistypeisthreetimesthatofonepagehitoperation!

Clicktoenlarge
Figure9.Pagemisstiming.Pageremainsopen
Therelativegain/lossratioforeachaccesstypecanbequicklyassessedsimplythroughacursoryreviewof
themostbasicdevicetimings.ImagineamemorykitratedforoperationatDDR31600,66618(CLtRCD
tRPtRAS):Withnothingmorewecanestimatesixcyclesforapagehitaccess,12cyclesforapageempty
access,and18cyclesforapagemissaccess.
Normalizedtothepagehitaccesslatency,pageemptyaccessistwiceaslong,andpagemissaccessisa
wholethreetimesaslong.IfwecombinethiswithwhatweknowabouttheinnerfunctionsoftheSDRAM
statemachineweseepagehitandpagemissarereallyjustsubsetsofthesamebankstate(active).Of
course,pageemptyaccessnecessarilyimpliesanidlebank.Thefollowingproofrewardsuswithsome
powerfulinsight.

http://www.anandtech.com/print/3851/everythingyoualwayswantedtoknowaboutsdrammemorybutwereafraidtoask

10/17

9/29/2016

EverythingYouAlwaysWantedtoKnowAboutSDRAM(Memory):ButWereAfraidtoAskPrintView

Thevariablenalsorepresentsthepercentageofaccessestobankswithopenpagesthatmustresultina
pagehitaccessifwearetosimplypacethenominalaccesslatencythatwouldbeachievedifeveryread
accesswastoanidlebank.AndtheonlythingthisdependsonistheRASPrechargedelayandtheRow
Column(orCommand)Delayofthedeviceinquestion.
Youwouldthinkbyworkingtomaximizen,performancewouldbemaximizedaswell.Andyou'dberight.Let's
takewhatwe'velearnedthusfarandstepitupanotch.Wepromise,afterthisyou'llneverseememory
timingsinthesamelighteveragain.
Beforeproceeding,we'vepreparedavideoforthoseofyouthatwouldliketoviewafewsimpleanimations
meanttohelpvisualizeeachtransactiontype:

Whatdoyoumeanyou'veneverheardofAdaptivePageManagement(APM)Technology?Well,thatmustbe
becauseIntelMarketingdoesn'tseemtofeeltheneedtobringitup.
Simplyput,Intel'sAPMdetermines,basedonthepotentialimplicationsofpendingmemorytransactions,
whetherclosingopenpages,orallowingthemtoremainopenlongermaybebeneficialtooverallmemory
performance.Inresponse,thememorycontrollermay(ormaynot)electtoissuecommandstoclosepages,
dependingontheprogrammedoperation.
Figure10providesthegeneralflowofeventsrequiredtomanagesuchaprocess.Inourexplanationwe
intendtointroduceyoutoallknownregistersettingsneededtoadjustthefunctionalcontrolpolicy,butfirstwe
needtodetailthenecessaryactions,andpurpose,ofthedesignelementsthatmakeupsuchamechanism.
Abetterunderstandingoftheunderlyinglogicwillpaydividendsasyouattempttodialinmeasurable
performanceimprovementsthroughexperimentation.
PerFigure11,theTransactionQueuestoresmemorytransactionsgeneratedbytheprocessor.Unlikea
typicalFirstInFirstOut(FIFO)queuewithatail,intowhichmemorytransactionmaybepushed,andahead,
fromwhichmemorytransactionsmaybepopped,thistransactionqueueisapluralityofstorageelements

http://www.anandtech.com/print/3851/everythingyoualwayswantedtoknowaboutsdrammemorybutwereafraidtoask

11/17

9/29/2016

EverythingYouAlwaysWantedtoKnowAboutSDRAM(Memory):ButWereAfraidtoAskPrintView

allowingsinglememorytransactionstoberemovedfromthelistanddispatchedtowardthememoryina
differentorderingthanwhenoriginallyaddedtothequeue.

Figure10.Genericmethodusedbythememorycontrollertoadaptivelygeneratepageclosemessages.
Differentsystemusagepatternswillmostlikelynecessitatechangestothebasedecisionlogic
Commandreorderingcanimproveperceivedmemoryperformancebygroupingtogetherreads/writestoa
commonphysicalpageinmemory,savingthetimethatwouldotherwisebeneededtolaterreopenthesame
page,shouldaconcurrentaccesstothesamebankforceittocloseearly.Afterall,theminimumdelay
betweensequentialaccessestothesameopenpageisequaltotheCASLatency(CLortCAS)ofthedevice.
Accessingabank(openingapage)increasesthelatencyofthepostinterleavedoperationbytheRow
Column(orCommand)Delay(tRCD),approximatelydoublingtheeffectivedataaccesstime.
Oneshouldalsoappreciatethattherearevaryingdegreesoffreedomwhenshufflingtransactionsintime.
Likeinthecaseofareadandwritetothesamememorylocation:thememorycontrollerwouldbedisallowed
frommovingthedependentreadeitheraheadoforbehindtheassociatedwriteastheorderingmustbe
implicitlymaintainedorcoherencywillbelost.
TheAddressDecoderpartiallydecodesthememorytransactionsstoredintheTransactionQueueasneeded
todeterminethebankandpageselectedbyeachqueuedrequest.Fromthere,theBankSelectmessages
controlthemultiplexersusedtoinputthecontentsofaBankRegistertoacomparatorusedtocheckifthe
selectedpagewasalsothemostrecentlyopenedpageforthatbank(assuch,eachBankRegisterislarge
enoughtostorenbitswhereeachbankcomprises2npages).AmatchresultsinthecreationofaPageHit
Resultmessage.

http://www.anandtech.com/print/3851/everythingyoualwayswantedtoknowaboutsdrammemorybutwereafraidtoask

12/17

9/29/2016

EverythingYouAlwaysWantedtoKnowAboutSDRAM(Memory):ButWereAfraidtoAskPrintView

Figure11.OurASUSRampageIIIExtremebetaBIOSincludessettingsusedtoestablishtheboundary
regionsthatdefinewheneachpreprogrammedAlgorithmisactive,theoperatingfrequencyofthepolicy
adaptionfeedbackloop,andthemaximumsingleinstancelifetimeforeachdecisiontoallowapagetoidle
openjustalittlelonger
TriggeredbythePageStateLogic,theSchedulerfetchespreidentifiedqueuedmemorytransactionsforre
orderingbasedontheMemorySelects(bothbankandpage)andtheassociatedPageHitResults.Anarray
ofBankStateRegisterstrackactionsperformeduponeachbankbystoringastatewordindicating,among
otherthings,whethertheAdaptivePageCloseLogicdecidedtoclosethebankinresponsetoaprevious
memorytransactiontothesamebank.
Finally,basedonthepolicyinstantiatedbytheAlgorithmSelector,aPageCloseMessageeitherisorisnot
generatedbasedonthesamePageHitResults,BankStateRegisters,andBank/PageSelectsinaneffortto
increasethenumberofsubsequentpagehitaccessesand/ordecreasethenumberofpagemissaccesses.
Animmediateandtangiblegainisachievedforeverysuccessfullyreorderedtransactionasapagehit
accessismoreefficientthanapageempty,oratworst,apagemiss.ThisisalwaysthecasewithCorei7and
isoneofthisarchitecture'swellknownshinningpoints.SwitchoffAdaptivePageManagement(disable
AdaptivePageClosinginBIOS)andthisiswheretheprocessends.Thepagemaystayopenforsomefinite
timeoritmaybeclosedrightaway;we'renotsureasthere'sreallynowaytoknowwithoutsomeinsidehelp.
TheAdaptivePageCloseLogicmustnowdecidewhethertocollectallwinnings,andclosethepage,orletit
ride,andleaveitopenjustawhilelonger.Whileanotherpagehitaccessmayyieldfurthergains,"guessing"
wrongwillcauseacostlypagemissaccessinplacewhatwouldhavebeenjustapageemptyaccess.Ifonly
thereweresomewaythesystemcouldmeasuretheeffectivenessofpreviousclosedecisionsandthenadjust
policytofit...
Surprise!ThePageManagermadeupofthePageStateLogic,AdaptivePageCloseLogic,andScheduler
doesexactlythis.Howthiseffectivenessismeasured,andhowtheresultofthatevaluationisusedtoadapt
thedecisionmakingprocessisournexttopicofdiscussion.

http://www.anandtech.com/print/3851/everythingyoualwayswantedtoknowaboutsdrammemorybutwereafraidtoask

13/17

9/29/2016

EverythingYouAlwaysWantedtoKnowAboutSDRAM(Memory):ButWereAfraidtoAskPrintView

First,considera4bitcounterwithtwoadjustablethresholds(Figure12).WhenCountisgreaterthanHigh
Threshold,AlgorithmAisdeemedappropriate;thesameistrueconcerningAlgorithmBandCountlessthan
LowThreshold.
FortherangebetweenHighThresholdandLowThreshold,eitheralgorithmmaybeineffect.Thisisbecause
aswitchfromAlgorithmBtoAlgorithmAwilloccuronlywithCountgreaterthanHighThresholdand
increasingandaswitchfromAlgorithmAtoAlgorithmBwilloccuronlywithCountlessthanLowThreshold
anddecreasing.
TheoverlaprangeisalsotheTargetRangeasthesystemwillnaturallyattempttomaintainCounterbetween
thesetwopoints.ThisistruesinceAlgorithmAtendstolowerCountwhileAlgorithmBtendstoraiseCount.
Thissystemactstoreduceoreliminaterapidthrashingbetweenalgorithms.

Figure12.Anotherwayoflookingatthis:iftheMSBofCountisa1,thenthepageclosepolicyistooloose
Next,defineatruthtable(Figure13)defininghowCountwillvary.Bydoingsowecanencodeafeedback
mechanismintooursystem.SuccessfulpredictionsbytheAdaptivePageCloseLogicapreventedpage
missaccess(good)inresponsetoadecisiontocloseapageorafacilitatedpagehitaccess(good)in
responsetoadecisiontoleaveapageopensuggestnochangetopolicyisrequiredandsonevermodify
Count.
Forafacilitatedpagemissaccess(bad)duetoapoordecisiontoleaveapageopen,incrementCount.If
Countweretotrendupwardwecouldconceivablyconcludethatthecurrentpolicywasmostoftenwrongand
notonlythat,tendedtoleavepagesopenfartoolongwhile"fishing"forpagehitoperations.Thecurrent
algorithmmustnotbeclosingpagesaggressivelyenough.
Forapreventedpagehitaccess(bad)duetoapoordecisiontocloseapageearly,decrementCount.If
Countweretotrenddownwardwewouldsuspecttheopposite:thealgorithmistooaggressivelyclosing
pagesandleavingpotentialpagehitsonthecuttingroomfloor.

http://www.anandtech.com/print/3851/everythingyoualwayswantedtoknowaboutsdrammemorybutwereafraidtoask

14/17

9/29/2016

EverythingYouAlwaysWantedtoKnowAboutSDRAM(Memory):ButWereAfraidtoAskPrintView

Figure13.Thepolicyiscontrollingjustrightwheneverwereducethenumberofpagemissoperationsand
increasethenumberofpagehitoperations
Asbestwecantell,thisconstructrepresentrealityforAPMTechnology.Althoughwewouldliketobelievethe
systemhasmorethantwogears(algorithms),ourmodelperfectlyexplainstheexistingcontrolregisterbothin
typeandnumber.
LookingaheadyouwillseeMaxPageCloseLimitandMinPageCloseLimitarethespecifiedHighandLow
Thresholdvalues,respectively.Settingalargerdifferenceincreasesthesizeofthefeedbackdeadband,
slowingtherateatwhichsystemrespondstoitsownevaluativeefforts.MistakeCounterisrepresentedbythe
startingCountandshouldbesetsomewherenearthemiddleofthedeadband.
AdaptiveTimeoutCountersetstheassertiontimeofanydecisiontokeepapageopen(i.e.howlongbefore
thedecisiontokeepapageopenstandsbeforewegiveuphopeofapagehitaccess).Repeatedaccessto
thesamepagewillresetthiscountereachtimeaslongastheremaininglifetimeisnonzero.Lowervalues
resultinamoreaggressivepageclosepolicyandviceversaforhighervalues.
RequestRate,webelieve,controlshowoftenCount(MistakeCounter)isupdated,andthereforehow
smoothlythesystemadaptstoquicklychangingworkloads.Theremustbeagoodreasonnottoflippantlyset
thisinterruptrateaslowaspossible.Perhapsthisdepleteshardwareresourcesneededforotheroperations
ormaybehigherdutycyclesdisproportionallyraisespowerconsumption.Whateverthereason,there'smore
thanafairchanceyoucanhurtperformanceifyou'rejustspitballingwiththissetting.

HereatAnandTechwedecidedtogotheextramileforyou,ourloyalreader.Afewweeksbackwe
approachedASUSUSATechSupportwitharequesttosetupatechnicalconsultationwiththeirFirmware
EngineeringDepartment.Afterpassingalongourrequest,whatcameoutofthemeetingwasaspecialbeta
BIOSthataddedanumberofpreviouslyunavailablememorytuningregistersonceexcludedfromdirectuser
control.
Intheinterestoffulldisclosure,wedidrequestthesamehelpfromEVGAandalthoughtheywerewillingto
backourplay,technicaldifficultiespreventedthemfromdeliveringeverythingwehadoriginallyhopedfor.
Seenbelow,thesenewregistersare:AdaptivePageClosing,AdaptiveTimeoutCounter,RequestCounter,
MaxPageCloseLimit,MinPageCloseLimit,andMistakeCounter.Assuspected,thefirstsettingisusedto
enabledordisablethefeatureentirely.Interestingenough,Intelchosenottoenablethisfeaturebydefault;so
weleaveituptoyou.

Clicktoenlarge
Youwon'thavefullresolutionwhenworkingwiththesesettings,butthenagain,youwon'tneedthemanyway
Ashortdescriptionofeachregisterisshownbelow(takenfromIntelCorei7900DesktopProcessorExtreme
EditionSeriesandIntelCorei7900DesktopProcessorSeriesDatasheet,Volume2,page79,datedOctober
2009).Beawarethesourcemostlikelycontainsatleastoneknownerror.Inparticular,Intelhasprovided
exactlythesamedescriptionforAdaptiveTimeoutCounterandMistakeCounter.Aswell,thebitcountfor
MistakeCounterinthetabledoesnotmatchthevalueinthetext,furthersuggestingsomeonegoofed.

http://www.anandtech.com/print/3851/everythingyoualwayswantedtoknowaboutsdrammemorybutwereafraidtoask

15/17

9/29/2016

EverythingYouAlwaysWantedtoKnowAboutSDRAM(Memory):ButWereAfraidtoAskPrintView

Yep,IntelowesusacorrectiontoMistakeCounter
Onceyou'vehadtimetofullydigesttheinformationaboveandponderhowawesomewearewewouldlike
tocordiallyinviteyoutodosomeofyourowntestingandreportyourresultsatourforums.AnandTech
readerswithavalidlogincandownloadASUSRampageIIIExtremeBIOSrelease0878now.Wehaven't
reallyhadachancetodoanysignificantexperimentingwithwhatlittlesparetimewehaveandweneedyour
helpexploringunchartedterritory...

Wehopeyouveenjoyedreadingthisarticleasmuchasweveenjoyedputtingittogether.Ifyoutookthetime
tothoroughlyperuseanddigesttheinformationwithintheintricaciesofbasicmemoryoperationshouldno
longerbesuchabafflingsubject.Withthegroundworkoutoftheway,wenowhaveasolidplatformfrom
whichtobuildaswemorecloselybeginexploringotheravenuesforincreasingmemoryperformance.Weve
alreadyidentifiedadditionaltopicsworthdiscussing,andprovidedthetimeshowsuponthebooks,planto
bringyoumore.
Assumedly,theonebigquestionthatmayremain:Whataretherealworldbenefitsofmemorytuning?
Technically,wecoveredthesubjectindepthlastyearinapreviousarticle.Wesuggestyoureadthroughit
onceagainforarefresherbeforeyouembarkonanyoverclockingjourneys(orbeforeyourushouttoover
spendonmemorykits).Everythingwritteninthatarticlethenisjustasvalidtoday.Weveruntestshereon
ourGulftownsamplesandfoundexactlythesamebehavior.Undoubtably,Intelhavetakenstepstoensure
theirarchitecturesaren'tprematurelybottleneckedbygivingthememorycontrollerabig,fatbusfor
communicatingwiththeDIMMs.

http://www.anandtech.com/print/3851/everythingyoualwayswantedtoknowaboutsdrammemorybutwereafraidtoask

16/17

9/29/2016

EverythingYouAlwaysWantedtoKnowAboutSDRAM(Memory):ButWereAfraidtoAskPrintView

ASUSRampageIIIExtrememarriedto12GBofsweet,sweetDDR3goodness
Fromwhatwecantell,thenextgenerationofperformanceprocessorsfromIntelaregoingtomoveovertoa
256bitwide(quadchannel)memorycontroller,leavinglittleneedforultrahighfrequencymemorykits.Thus
wereiteratesomethingmanyhavesaidbefore:atopprioritywhenitcomestoimprovingmemoryICsand
theirrespectivearchitecturesshouldbetofocusdevelopmentonreducingabsoluteminimumlatency
requirementsfortimingssuchasCASandtRCD,ratherthanchasingrawsyntheticbandwidthfiguresor
settingoutrightfrequencyrecordsattheexpenseofundulyhighrandomaccesstimes.
Steppingawayfromtheperformancesegmentforamoment,somethingelsethat'salsocometolightis
rumorednewsthatIntel'sSandyBridgearchitecture(dueQ12011)will,bydesign,limitreferenceclockdriven
overclockingonmainstreampartsto5%paststockoperatingfrequency.Ifthisisindeedthecasethe
consequencewillbeaveryrestrictedabilitytocontrolmemorybusfrequencywithlimitedgranularitytotune
thefirst50~70MHzpasteachstep,followedbymandatoryminimumjumpof200MHztothenextoperating
level.Accessinghiddenpotentialwillbeevenmoredifficult,especiallyforusersofmainstreammemorykits.
Whilethereisnodownsidetothisfromaprocessingperspective(hey,morespeedisalwaysbetter),this
couldbeanotherseriousnailinthecoffinofanalreadywaningoverclockingmemoryindustry.

DESKTOPANDPORTABLE
STORAGESOLUTIONS.
HighPerformance.HighReliability.

LEARNMORE

Advertisement

http://www.anandtech.com/print/3851/everythingyoualwayswantedtoknowaboutsdrammemorybutwereafraidtoask

17/17