5/20/2017 HowwemigratedourJavascriptstacktoWebpackSkroutzEngineering

Menu

JAVASCRIPT / WEBPACK / YARN / FRONTEND

HowwemigratedourJavascriptstacktoWebpack
BYMARIAKOUSTA APRIL26,2017 COMMENT TWEET LIKE +1

WehaverecentlymovedourmainproductsJavascriptstackfromRailsassetpipelineand
RequireJStoWebpack.ThereisalotofstuffonlineaboutthebenefitsofWebpack,soIwont
becoveringitinthispostbutIwilltrytogiveyouthereasoningfromourownexperiencewith
assetpipelineandRequireJShereatSkroutz.

WhatwehadOurbaggage

WeusedtohaveamixtureofRailsassetpipelineandRequireJSinourmainproduct.Asyou
knowassetpipelinedoesnottakeintoaccountJavascriptmodulesystemsandvarious
frontendfeaturessuchaslazyloadingondemandJavascriptfiles.Thiswasthereasonwealso
hadincorporatedthepopularRequireJSlibraryinourstack.Weuseditformanagingour
modulesanddependenciesaswellastolazyloadsomenoncriticalJavascriptfiles.Inorderto
makeitworktogetherwithRailsweusedourownforkoftherequirejsrailsgem.

Thissetupstayedwithusforalongtimebutunfortunatelyhadalsosomeseriousissues.First
andforemost,wewereboundtoAMDmodules.ThismadeitdifficulttousecurrentJavascript
librariesandtherichNodeJSmodulesecosystem.Wehadtoshimeverythingthatwasnon
AMD.

Also,therequirejsgemaddedalotofcomplexityinoursetup.Wehadtodoublecompileour
assets.SprocketshadtoproducebothdigestedandnondigestedfilestobeusedinRequireJS
bundling,wehadtomaintain2manifestfiles,andwehadtodecidemanuallywhichfileis
neededinwhichbundle.Thisresultedinerrorpronebuildsthatveryfewofourdevs
understoodhowtofix.

Wealsohadtofindamanualway(commithashbased)forcachebusting.Railsassetcache
bustingmechanismwasnotanoptionbecausewewereusingrequirejsgeneratedbundlesthat
welazyloaded.Lastbutnotleast,wehadnoeasywaytoincorporategraduallyinourstack
ES2015(wewereusingCoffeescriptatthetime).

ForaddingandmanagingexternallibrariestouseinourproductweusedBower.Although
Bowerwasagoodsolutionandhasmadeituntiltodaywithus,wehadalsosomeissueswith
it.Forexamplesomepackagesweneededwerenotintheregistry.Themainproblemthough

https://engineering.skroutz.gr/blog/howwemigratedtowebpack/ 1/9
5/20/2017 HowwemigratedourJavascriptstacktoWebpackSkroutzEngineering

wasthatwedidnottrusttheavailabilityofpackagessowewerecheckinginourpackagesto
versioncontrol.Thismadetheworkflowofpackagemanagementseemmorecomplicatedand
lessintuitivetoourdevs.WewantedtotrythepromisingYarnpackagemanagerandaneasy
waytoincorporateitinourstackanduseourexternaldependencies.

LatelytheJavascriptecosystemhasbeenadvancingalotandRailshasnotbeenabletocatch
upwiththepaceofthesechanges(unmaintainedgems,assetpipelinenotkeepingupwith
modulesystemsetc).Thetimehadcometomovetosomethingbettersothejourneytoa
betterJavascriptstackbegun.

HerecomesWebpackDefinethedestination
BeforesettlingtoWebpackwehadtodefineourneedsandresearchtheavailabletoolsthat
couldfillourneeds.WeexaminedvariousoptionsforbundlingandloadingsuchasBrowserify,
Webpack,JSPM,anothertakeonRequireJSetcetc.After23minimumsetupstryingoutthe
varioustools,thewinnerforusbecameclear.ItwasWebpack.Thisdoesnotmeanthatthe
othertoolsarenotgood,justthatWebpackwasexactlywhatmetourneeds.Ihavetoadmit
herethatwedidntthinkWebpackwouldbethewinnerrightfromthestart.Thatswhythis
stepwasreallyimportant.YoucanfindvariouscomparisonsonlineonthesetoolsbutIsuggest
thatyouskipthehypeonvariousJavascripttoolsandreallydiveintoyourapplicationsspecific
needsbecauseyoumightenduptryingtokillamosquitowithashotgun!

Forustheminimumrequirementswere:

1)PlaynicewithRails

OurstackisbasedonRubyonRails,soaddingatoolthatneedsalotoftweakingtoworkwith
Railswasnotanoption.Weactuallyalreadyhadabitterexperiencewithsuchastack.Inthis
case,itseemsthatRailsandWebpackplaynicetogether,sincetheyareagnostictoeachother.
Moreover,Rails5.1willbeincorporatingWebpackforJavascriptmanagementsoitfeltlike
makingtheinevitablestepahead.

2)Performance

Performanceisquiteimportantforus(evenmorethanitisforeveryone).Sincetheappis
gettingbiggerandbiggereveryday,weneedfeaturessuchaslazyloading,bundlingand
servingonlywhatisnecessaryoneverypage(orafterdifferentuseractions)withoutmuch
manualconfiguration.Wetrytokeepourbundlesminimalandsafefromdevelopersbloating
thecriticalonesbymistake.

3)Flexible

https://engineering.skroutz.gr/blog/howwemigratedtowebpack/ 2/9
5/20/2017 HowwemigratedourJavascriptstacktoWebpackSkroutzEngineering

Sinceourproductisbeingactivelyenhancedeveryday(variousdeploysdaily),wedonothave
theluxurytofreezeourfeaturedevelopmentandmakethoroughchangesonourcodebase.
Thereforeweneededtopickatoolthatwouldenableustomakechangesonourstack
graduallywithoutholdingbackourdevelopmentatall.Inordertomaketheinitialminimum
changesandthenapplyanychangesgradually,weneededatoolthatwouldsupportvarious
thingsfromourcurrentstack,likeAMDmodulesforexample.

4)Activelysupported

Itwasalsoimportanttopickatoolthatwouldstaywithusforalongtime,sobeingactively
supportedandseeingafutureinourtoolofpreferencewasaprioritytoo.Whiletryingout
Webpack1,Webpack2wasalreadyonitswaytobereleasedandwehadtomigrate.The
activecommunitybehindthetoolanditsecosystem,aswellastheseamlessmigrationtov2
madeusfeelsafeforitssupportandconfidentforthegrowingcommunitybehindit.

Webpackmettheaboverequirementsandalotmoreaswefoundoutontheway.

Letsdoit!Thejourneybegins
ThejourneytoWebpackwasnotquiteeasybutthatwouldbetrueforuswithanyother
bundlingorloadingtoolbecauseourJavascriptcodebase,pipeliningandworkflowwastightly
coupledwithRails.Thiswassomethingwewantedtochangeandwefoundoutthatthiswas
ourchancetodoit!

IntherestofthispostIlltrytoexplaintheproblemswecameacrossandhowwetriedto
solvethemonestepatatimewhilemigratingtoWebpack.

Thejourney

1ststp
InordertoaddWebpackandYarntoourstackweneededtoupgradeourdevelopmentand
deployermachinesfromNode0.10toversion4.7.2sinceWebpackrequiresatleastnode
v4.3.ItmightcomeasasurprisethatwehadsuchanoldNodeversionbutitsnotreallygiven
thatourmachinesrunonDebianJessiestable(seehere).Atthattimewehadtobackport
NodeJS4.7andaddedYarntoourinternalDebianrepository.

2ndstp

https://engineering.skroutz.gr/blog/howwemigratedtowebpack/ 3/9
5/20/2017 HowwemigratedourJavascriptstacktoWebpackSkroutzEngineering

WewerenowabletoaddWebpacktoourcodebasethroughYarn.Wealsoaddedthe
webpackrailsgemtoourRailsprojectmainlytogetsomeassetpathhelpersandsetupoutof
thebox.Wealsoaddedwebpackdevserverforthedevelopmentenvironment.norderto
simplifytheworkflowforourdevelopersweaddedForemantoruntheWebpackdevserver
concurrentlywiththeRailsserver.

WithaminimumWebpackconfigurationwewereabletoloadabasicentryfileforour
Javascriptapp.Fromthispoint,everythingwasamatterofconfiguration.Forexampleacoffee
loaderforourCoffeescriptfiles,ahandlebarstemplateloaderforour.hbstemplates,exports
loaderforlibrariesdefiningglobalvariablesetcetc.

Hereisa(simplified)partofourWebpackconfig:

1 varconfig={
2 resolve:{
3 modules:[
4 path.join(__dirname,'..','app','assets','javascripts'),
5 'node_modules',
6 'templates'
7 ],
8 extensions:['.js','.coffee','.js.coffee','.hbs','.js.es6']
9 },
10
11 entry:{
12 'skr_load':['./app/assets/javascripts/skr_load']
13 },
14
15 output:{
16 path:path.join(__dirname,'..','public','assets','webpack'),
17 publicPath:'/assets/webpack/',
18
19 filename:production?'[name][chunkhash].js':'[name].js',
20 chunkFilename:production?'[name][chunkhash].js':'[name].js',
21 hashDigestLength:32
22 },
23
24 module:{
25 rules:[
26 {test:/highcharts/,use:['exportsloader?Highcharts']},
27 {test:/modernizr/,use:['exportsloader?Modernizr']},
28 {test:/\.coffee$/,use:['coffeeloader']},
29 {test:/\.hbs$/,use:['handlebarstemplateloader']}
30 ]
https://engineering.skroutz.gr/blog/howwemigratedtowebpack/ 4/9
5/20/2017 HowwemigratedourJavascriptstacktoWebpackSkroutzEngineering

31 }
32 }

Lines19:HerewetellWebpackwheretolookformodulesandwhichfileextensionstolook
for.WehavesimplyaddedRails app/assets/javascripts alongwithnodemodulesfolder
andourtemplatesfolderhere.

Lines1113:Hereweaddtheentryfileofourapplication.Allofitsdependenciesandthe
dependenciesdependenciesareresolvedandaddedtoabundleautomatically!

Lines1522:HerewedefinehowtheoutputofWebpackshouldbe.Namely,thefilesshould
bedigestedandputin public/assets/webpack .

Lines2432:Herewedefinesomerulesforourmodules.Forexamplehighchartsand
modernizrlibrariesshouldreturnaglobalvariableastheirmoduleexport.Also,.hbsand.coffee
filesshouldbepreprocessedbytheircorrespondingloaders.

Withwebpacksasynccodesplittingfeaturewewerealsoabletoeasilydefinesplitpointsto
ourapplicationwhereweloadextrafilesasynchronouslywhentheyareneeded.

3rdstp
Alongwithstep2wehadablockingissuewithourcurrentJavascriptcodebase.Itwascoupled
withRails.WewereusingsomeSprocketsdirectiveshereandthereandtherewerefileswhere
weusedembeddedRuby.

Sprocketsdirectiveswereeasilyreplaced.Forexample,wewereusingsprocketsdirectivesto
loadourHandlebarstemplates.UsingahandlebarstemplateloaderandsinceWebpacktreats
everyfileasamodule,wewereabletoreplacetheseSprocketsdirectiveswithJavascript
modulerequirecalls.

AsforJavascriptmoduleswithembeddedRubywetriedusingarailsloaderforWebpack.We
noticedthoughthatloadingtheRailsenvironmentwasaddingabigdelaytothecompilation
stepsowedecidedthatitwasnotworthit.WehadalreadydiscussedthatembeddingRuby
seemedlikeabadpracticetous.Besides,themainreasonweusedembeddedRubywastouse
inourJavascriptfilessomevariablesorsettingsfromourbackend.Forexampleaproductidor
ourgoogleAPIkey.Thereforewedecidedtodiscardthispractice.Wegatheredallthese
settingsandvariablesinasmallblockinginlinescriptinourRailslayoutsothateveryJavascript
modulecouldusethem.

https://engineering.skroutz.gr/blog/howwemigratedtowebpack/ 5/9
5/20/2017 HowwemigratedourJavascriptstacktoWebpackSkroutzEngineering

4thstop
Sinceourproductistranslatedinto3languages(skroutz.gr,alve.com,scrooge.co.uk)weare
usingtheFastGettextgemasthebackendofI18n.Wealsousethegettext_i18n_railsplugin
forintegratingFastGettextintoRails.Inordertobeabletousegettextmethodsinour.jsand
.hbsfileswewereusingonceagainembeddedRuby,whichwehadtodrop.

Thissetupwasalsoproblematicforanotherreason.Wewerenotabletousesomegettext
methodsin.erbfiles,forexamplepluralssincethemethodshadtobeevaluatedatcompile
time.ThereforewedecidedtoswitchtoaJavascripttranslationssetup.

Weaddedthegettext_i18n_rails_jsgeminordertoparseour.jsand.hbsfilesforstrings
markedwithdoubleunderscore.Then,atcompiletime,withthehelpofaWebpack.pofile
loaderwereplacemarkedstringswiththeirtranslations.Forexample,inordertobeabletouse
plurals,wereplacethemwithJavascriptmethodsthatevaluatetheappropriatetranslationat
runtime.ForallthesetoworkinbothJavascriptand.hbsfileswehadtowriteacustom
Webpackplugin(weareconsideringmakingthisopensource).Soforexamplethetranslation
for __("Hello") isbeingevaluatedatcompiletimetoHellobutthetranslationof
n__("shop","shops",shop_count) isbeingreplacedwithamethodthatdecidesatruntime
ifitshouldbeshoporshopsaccordingto shop_count .

5thstop
DecouplingofourJavascriptappwithRailsenabledustoimproveourtestingmechanism.We
wereusingtheTeaspoonframeworkforrunningourmochatests.WithTeaspoonwerunour
suitebothheadlesslyandinabrowserwhilewealsohadconfiguredittorunwithourCI
(Jenkins).Wehadaverygoodexperiencewiththisframeworkbuttherewasnoreasonin
keepingitsincewedidnotneedtheintegrationwithRailsanymore.Thereforeweswitchedto
Karmarunnerandkarmawebpack.Thisresultedinamuchfasterrunforourspecsuite
(reducedtoabouthalfthetime).

Aftermath

After+1804/1613linesofcodeand650fileschanged,weshippedtheminimumpossible
changesforswitchingtoWebpack.

Positivemigrationsideeffects
JavascriptTranslationsSetup

https://engineering.skroutz.gr/blog/howwemigratedtowebpack/ 6/9
5/20/2017 HowwemigratedourJavascriptstacktoWebpackSkroutzEngineering

BetterJavascriptPackageManagement

Contentbasedcachebusting

Livereloadandsourcemapsindevelopmentoutofthebox

Assetscompilationtime20%down

JStestsrunningmuchfaster

ReadyforRailsfutureupdates

NextSteps
WearecurrentlygraduallyswitchingourexternalJavascriptdependenciestobeaddedwith
YarninsteadofBower.Yarnhasenhancedourworkflowandexternallibrariesmanagement.
Moreover,weaddedBabelandwecannowgraduallymovefromwritingCoffeescriptto
ES2015.WejustaddedthepackagesandonelineofconfigurationtoWebpacksothata
differentWebpackloadertakescareofourES2015code:

module:{
rules:[
{test:/\.coffee$/,use:['coffeeloader']},
{
test:/\.js.es6$/,
use:[{
loader:'babelloader',
options:{presets:['es2015']}
}],
}
]
}

Conclusion
SwitchingtoWebpackwasnotapieceofcakeforusmainlybecauseitunlockedvariousissues
inourcodebase,likethecouplingwithRails,thelackofasanetranslationmechanismforour
Javascriptapplication,etc.Afterdeployingandhavinglivedwithitforsometime,Icantellyou
thatitwassurelyworthitbecauseitnotonlyforcedustotackletheseissues,italsomadeit
possibletosimplifyourworkflowforcompiling,bundlingandloadingJavascript.Byadding
Yarntoourworkflow,externallibrarymanagementfeelsmuchmoreintuitivetouswhileits
timesareverygood.Thesameistrueforassetcompilationandtestsuiterunningtimes.This

https://engineering.skroutz.gr/blog/howwemigratedtowebpack/ 7/9
5/20/2017 HowwemigratedourJavascriptstacktoWebpackSkroutzEngineering

setuphasbeenworkingforaboutamonthnowwithnoissuesandhasalsoopenedtheway
forustograduallymovetoES2015.Itmightnotbeperfectbutitsurelymakesmuchmore
sensetousandfeelslikeitisasolidbasetobuilduponinthefuture.Besides,Lifeisajourney,
notadestination.

0Comments SkroutzEngineering
1 Login

Recommend Share SortbyBest

Startthediscussion

Bethefirsttocomment.

ALSOONSKROUTZENGINEERING

OptimizingContinuousIntegrationat
Skroutz
2commentsayearago
DimitrisBliabliasHelloNick.We've
thoughtaboutthatoptiontoo,butforour
needsthelongtermcostofrunningourCI
serviceatthe
Rewritingourwebanalyticstracking
infrastructureinGo
2comments4daysago
AgisThatbitwasindeednotveryclear,
I'veupdateditnow.Weusejournald(a
systemdcomponent)forcapturingthe
outputand
Subscribe d AddDisqustoyoursiteAddDisqusAdd
Skroutzinfrastructureataglance
6comments2yearsago
MarthaWhitmoreThisisanamazingpost
forme.thanksforthepostandkeepon
sharing.
AbouttheSkroutzEngineeringBlog
1comment2yearsago
MihailMalostanidisSeeinghowyoulive
onyourowninfrastructure,youreallyought
toimplementbrotliandwebp.It'sstupidly
easy
Privacy

Previous Next

About Developertools Skroutz

SkroutzBlog SkroutzGithub Alve

2017SkroutzEngineeringTeam.PoweredbyJekyll.
https://engineering.skroutz.gr/blog/howwemigratedtowebpack/ 8/9
5/20/2017 HowwemigratedourJavascriptstacktoWebpackSkroutzEngineering

https://engineering.skroutz.gr/blog/howwemigratedtowebpack/ 9/9