CI for iOS:

do more while you sleep

Kevin Munc – @muncman
Method Up LLC
[MU]
CI?
For iOS?
Pain
Pain & Suffering
 It can be done

• Start with the basics

• Xcode and the command line
• Version control and CI server
• Script it, run it, schedule it
Tip #1
 Jenkins

• http://jenkins-ci.org/
• Mac installer or via homebrew
• Many alternatives
• Many plugins
Tip #1.1
How many Simulators does
it take to break a build?
A CI server gives you:
A CI server gives you:

Automation
A CI server gives you:

Automation

Feedback
 A CI server gives you:

!
Automation
f u l
de r
on
W
Feedback
 A CI server gives you:

!
Automation i n d
f u l f m p s)
de r c e o n a
n a e r
Wo Pe bett
Feedback (fo r
Requirement: OS X
Requirement: OS X
• macminicolo.net

• hosted-ci.com

• cisimple.com

• macincloud.com

• hostmyapple.com

• macminivault.com

• xcloud.me
Requirement: OS X
• macminicolo.net

• hosted-ci.com
h t ,
n i g
• cisimple.com
C I t o r u n a
ac
t
,
u l e M
• macincloud.com
O r sch e d
o u
on y you slee
r o w n
p . . .
• hostmyapple.com w h i l e

• macminivault.com

• xcloud.me
 Security
sudo /usr/sbin/DevToolsSecurity --enable
 Security

sudo xcodebuild -license

 Credentials
• The CI server needs access to your
repository
• SSH key
• The CI server needs Keychain credentials if
you want to sign your builds
• Developer certificate and profiles
 Keychain
• Export and Import using Keychain Access app
• Keys and Certificates
• Or use the command line:
• sudo security import /path/to/distribution.cer -k /Library/Keychains/
System.keychain

• sudo security import /path/to/distribution.p12 -k /Library/Keychains/

System.keychain

• Don’t forget about provisioning profiles

• Troubleshooting: http://tinyurl.com/iOSDevToolTroubles
Tip #2
 Avoid this error
from Xcode
RunPlatformUnitTests:
warning: Skipping tests;
the iPhoneSimulator platform does
not currently support application-
hosted tests (TEST_HOST set).
 The culprit

/Applications/Xcode.app/
Contents/Developer/
Platforms/
iPhoneSimulator.platform/
Developer/Tools/
RunPlatformUnitTests
The workaround(s)
 Xcode Plugin

• https://wiki.jenkins-ci.org/display/JENKINS/Xcode+Plugin

• Specify provisioning profile

• Keychain access
• Signed IPA
 Xcode Plugin

• Recommended xcodebuild arguments:

• GCC_SYMBOLS_PRIVATE_EXTERN=NO
• COPY_PHASE_STRIP=NO
• Allows test bundle to link with Release
build symbols
Signing with the Plugin
Signing with the Plugin
Signing with the Plugin
Tip #3
 Prefer Scripts
over CI Plugins
• More flexibility
• More resilient to Apple’s changes
• More power to adapt and expand (specificity)
• Less coupling to the specific CI server type
• You can version control the settings easier
xcodebuild

xcodebuild \
clean build
Tip #4
 Be specific with
xcodebuild options
 xcodebuild
xcodebuild \
-scheme SOLID \
-target SOLID \
-configuration Debug \
-arch i386 \
-sdk iphonesimulator \
clean build
 xcodebuild
xcodebuild \
-scheme SOLID \
-target SOLID \
-configuration Debug \
-arch i386 \
-sdk iphonesimulator \
clean build
 xcodebuild
xcodebuild \
-scheme SOLID \
-target SOLID \
-configuration Debug \
-arch i386 \
-sdk iphonesimulator \
clean build
 xcodebuild
xcodebuild \
-scheme SOLID \
-target SOLID \
-configuration Debug \
-arch i386 \
-sdk iphonesimulator \
clean build
 xcodebuild
xcodebuild \
-scheme SOLID \
-target SOLID \
-configuration Debug \
-arch i386 \
-sdk iphonesimulator \
clean build
 xcodebuild
xcodebuild \
-scheme SOLID \
-target SOLID \
-configuration Debug \
-arch i386 \
-sdk iphonesimulator \
clean build
 xcodebuild
xcodebuild \
-scheme SOLID \
-target SOLID \
-configuration Debug \
-arch i386 \
-sdk iphonesimulator \
clean build
 xcodebuild
xcodebuild \
-scheme SOLID \
-target SOLID \
-configuration Debug \
-arch i386 \
-sdk iphonesimulator \
clean build
Tip #5
 Specify Alternate
Output Locations

• Don’t rely on Xcode’s cryptic locations

• This will make your scripts easier to
manage
# Build location for Instruments test runs.
CI_BUILD_DIR=/tmp/SOLID_AUTOMATION
mkdir -p $CI_BUILD_DIR

xcodebuild \

... \

CONFIGURATION_BUILD_DIR=$CI_BUILD_DIR \

clean build
# Build location for Instruments test runs.
CI_BUILD_DIR=/tmp/SOLID_AUTOMATION
mkdir -p $CI_BUILD_DIR

xcodebuild \

... \

CONFIGURATION_BUILD_DIR=$CI_BUILD_DIR \

clean build
# Build location for CI test runs.
CI_BUILD_DIR=/tmp/SOLID_AUTOMATION
mkdir -p $CI_BUILD_DIR

xcodebuild \

... \

CONFIGURATION_BUILD_DIR=$CI_BUILD_DIR \

clean build
Demo
Tip #6
Ensure Return Codes

• If the tool you are invoking in your script

doesn’t return an exit code indicating
pass/fail, it’s up to you.
• Unix-friendly 0 (success) or 1 (failure)
Tip #7
Test against multiple
SDKs

-sdk iphonesimulator5.0
-sdk iphonesimulator5.1
-sdk iphonesimulator6.0
 Other Tools
to Know About
• xcode-select (manage xcode path)
• Overridden by DEVELOPER_DIR

• xcrun (find and run dev tools)

• ibtool (for Interface Builder files)
• /Applications/Xcode.app/Contents/Developer/usr/bin/

• Variables in Xcode
• “Build Settings Reference”
OCUnit
Other Related Tools
 Other Related Tools
• OCMock • XcodeTest
• OCMockito • AutoBuild
• OCHamcrest • xcodeproj (CocoaPods)
• Expecta • Circle?
• TinyMock • Simon?
• LRMocky • ios-maven-plugin
• OCUnit2JUnit • Ceedling
• OCUnitReport • XcodeCoverage
• Kicker • JMRTestTools
• xcodebuild-rb • Nocilla
• xcodearchive • OHHTTPStubs
 OCUnit Alternatives
• Kiwi
• Cedar
• OCDSpec
• C++ Automated Test Cases in Headers
(CATCH)
• Objective-Shoulda
• Specta
Tip #8
 OCUnit2JUnit for
Test Results

• ocunit2junit.rb
• https://github.com/ciryon/OCUnit2JUnit
• Converts output to JUnit format for easier
transformation to HTML
UIAutomation
 Other UIAutomation
Tools
• tunup_js • ios-sim

• jasmine-iphone • ios-sim-locale

• uiautomation-jasmine- • iphonesim
iphone
• ui-auto-monkey
• Bwoken
• ui-screen-shooter
• Zucchini Framework
• uiautomation-rb?
• WaxSim
 UIAutomation
Alternatives
• Frank • Objective C Slim
(ocslim)
• (iCuke)
• AutomationKit
• KIF
• Sikuli
• Calabash
• MonkeyTalk (formerly
• UISpec FoneMonkey)

• Bromine • Plus other commercial

tools, such as Telerik Test
Studio for iOS
Tip #9
 PListBuddy
is Your Buddy

/usr/libexec/PlistBuddy
Tip #10
 Always quit the
simulator
• killall
• killall -m -KILL "iPhone Simulator"
• AppleScript
• osascript -e 'tell app "iPhone Simulator"
to quit'
Tip #11
 Use Jonathan Penn’s
AutomationExample

https://github.com/jonathanpenn/AutomationExample
Demo
More?
Code Coverage

“Risk coverage”
 Coverage Tools
• gcov
• GCC’s coverage tool
• GUIs for gcov
• LCOV (HTML)
• CoverStory (Mac app)
• ocov?
Tip #12
 gcovr for
Coverage Automation

Produces Cobertura-formatted output

from gcov data
Tip #12.1
Enable Test Coverage
 Static Analysis

• Clang
• http://clang-analyzer.llvm.org/
• scan-build
Tip #13
 Symbolic links
can be your ally
 Avoid scan-build Pain
sudo ln -s \
/Applications/Xcode.app/Contents/Developer/Toolchains/
XcodeDefault.xctoolchain/usr/lib/arc/
libarclite_iphonesimulator.a \
/usr/lib/arc/libarclite_iphonesimulator.a
 What about
test build deployment?
Tip #14
 Use TestFlight

• testflightapp.com
• OTA deployment
• SDK for more features
• Free!!
 TestFlight

• Web UI
• Desktop App
• REST API
• Jenkins Plugin
TestFlight Alternatives
• HockeyApp
• http://hockeyapp.net

• Also has a Jenkins plugin (forked)

• BetaBuilder for iOS

• http://www.hanchorllc.com/betabuilder-for-ios/

• More players in this space, esp. for enterprises...

 TestFlight Config

• API Token and Team Token

• Need a signed IPA
• They get an email with a link for OTA
installation!
Tip #15
 Protect Your Tokens
• API Token & Team Token
• Keep them out of scripts
• Instead, define them in Jenkins (and leverage
Jenkins security)

• 'Configure System'
• 'Global properties' section

• Or use the plugin (same parameters)

TestFlight Plugin
TestFlight Script
Deployment
Deployment
 Q: Once you have a nightly job to build,
unit test, sign an IPA and deploy it to QA,
what’s next?
 Q: Once you have a nightly job to build,
unit test, sign an IPA and deploy it to QA,
what’s next?

A: Sleeping easy.
 Documentation

• Appledoc
• Doxygen
• HeaderDoc
DocSets
Appledoc
Doxygen
Tip #16
Don’t stop there!
Other Goodies
 Other Goodies
• Ensure your site or API is up
• Scan your code for TODOs and FIXMEs
• Get trend reports for lines of code
(SLOCCount w/ sloc2html)
• Scan for duplicated blocks of code (CPD,
Simian, etc.)
• Use agvtool (Apple-Generic Versioning
Tool) to increment your build number
 Still Missing
• Cyclomatic Complexity

• Coding convention/Style checker

• Code Formatter

• Uncrustify, UniversalIndentGUI

• UML Generation

• via CLI, as opposed to OmniGraffle (AppleScript?)

• ER Diagram from Core Data schema

Tip #n
 Sources
• Slides: http://www.slideshare.net/muncman

• http://blog.carbonfive.com/category/mobile/

• http://www.stewgleadow.com/

• http://lifeandcode.net/

• http://longweekendmobile.com/blog/
• http://longweekendmobile.com/2011/04/17/xcode4-running-application-
tests-from-the-command-line-in-ios/

• http://qualitycoding.org

• et cetera
 Thank you!
• CodeMash organizers
• CodeMash sponsors
• and YOU!

[MU]
 Thank you!
• CodeMash organizers
• CodeMash sponsors
• and YOU!
Questions?
[MU]