iOS Continuous Integration with Ansible

In the previous post, we described how we used Fastlane to continuously deploy our iOS app from our continuous integration server. One of the most tricky obstacles to Continuous Delivery is  the “works-on-my-machine” phenomenon. Almost every engineer has encountered this problem at least once!

 

Challenges

We came across these challenges while maintaining our continuous integration servers:

  • Manually setting up CI server machines with all iOS related softwares takes days
  • The results produced on CI servers are not reproducible on local developer machines
  • Our build configuration steps are tightly coupled with GUI configuration in the TeamCity continuous integration server
  • The versions of software on developer machine and CI server are different
  • We were losing trust in our CI server, and ended up spending lots of time finding the root cause of the problems

At this stage, we really needed a configuration management tool which would allow us to setup all the machines with the same configuration and provision them from scratch. We needed our iOS infrastructure to be driven by code a.k.a Infrastructure as code. We were looking for something like  Docker for Darwin. There were various options like Chef, Puppet etc but we opted for Ansible.  Ansible is a simple but powerful tool for infrastructure automation, and it doesn’t require knowledge of another language like Ruby.

Provisioning iOS CI with Ansible

Previously, we would manually setup a CI or continuous integration server with all the required software for the delivery of an iOS app. This requires downloading and installing Xcode with additional components, installing required homebrew packages, setting up RVM and rubygems. It also involves setting up a build server such as TeamCity Build Agent. It would typically take a day or two to get the build machine in a working state. As well as being time consuming, it was very difficult to manage versions of the software installed on the build server. Using Ansible we managed to automate virtually all those manual steps.

It’s important to be able to reset and re-build an iOS Continuous Integration Server environment; Apple continuously release new or beta versions of Xcode and other developer tools, so you need the ability to setup and use those features quickly. The programmable infrastructure or infrastructure as code, is key to Continuous Delivery, so we needed to have a script to setup these things up automatically. Ansible allows us to write code to manage configurations and and automated provisioning of the infrastructure in additional to deployment. Ansible tasks and playbooks are simple YAML files so there is no need to learn another programming language to script an iOS infrastructure. The provisoning of iOS Continuous Integration server involves the following tasks:

  • Provisioning installation of Xcode and Xcode Command Line tools
  • Provisioning homebrew packages required for iOS development
  • Provisioning Ruby environment using RVM
  • Provisioning MacOS defaults
  • Provisioning TeamCity Agent for TeamCity Server

We have an Ansible task for each of above steps.

 

Ansible Xcode Task

Most Apple softwares including Xcode are proprietary; you need an Apple developer account to download and install that software. It was a major challenge to provision an installation of Xcode. We decided to download the Xcode .xip  file for specific Xcode version and host it on the company hosted SAMBA server. We could then mount  Xcode .xip  to any build server machine to install Xcode. We could then provision other Xcode related task like accept agreement, installing command line tools, installing additional component using Ansible tasks. The sample Ansible task looks like this:

---
- name: Mount Xcode XIP from the Samba Server
  command: bash -c "mount_smbfs //{{ ansible_env.SAMBA_USER}}:{{ ansible_env.SAMBA_PASS }}@our_server/Applications/xcode/ ~/samba/public/"
  when: xcode_dir.stat.exists == False

- name: Install Xcode from XIP file Location
  command: bash -c 'open -FWga "Archive Utility" --args ~/xcode_xip/{{ xcode_src }}'

- name: Move Xcode To Application
  command: bash -c 'mv ~/xcode_xip/Xcode*.app /Applications/'

- name: accept license agreement
  command: /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -license accept
  become: yes
  become_method: sudo

- name: install additional xcode components
  command: installer -pkg /Applications/Xcode.app/Contents/Resources/Packages/XcodeSystemResources.pkg -target /
  become: yes

We can then pass the xcode_src variable from the playbook file which is the Xcode version, we need to install. In this way, we can easily switch to another Xcode version.

Ansible Homebrew Task

As part of provisioning the iOS CI server, we needed to install some MacOS packages. Homebrew is an awesome package manager and most importantly Ansible has in-built homebrew module which allows us to tap any Homebrew formulae or install home-brew packages. Our sample home-brew task look like this:

---
- name: Ensure configured taps are tapped.
  homebrew_tap: "tap={{ item }} state=present"
  with_items: "{{ homebrew_taps }}"

- name: Ensure configured homebrew packages are installed.
  homebrew: "name={{ item }} state=present"
  with_items: "{{ homebrew_installed_packages }}"

- name: Install configured cask applications.
  homebrew_cask:
    name: "{{ item }}"
    state: present
    install_options: "appdir=/Applications"
  with_items: "{{ homebrew_cask_apps }}"
  when: configure_cask

We can then pass homebrew_taps , homebrew_installed_packages  and homebrew_cask_app  from our playbook. This means we have ability to manage all the MacOS softwares packages from code.

Ansible RVM Task

Ruby plays a very important role in the iOS application development as tools like CocoaPods, Fastlane come as Ruby libraries. We needed a  higher version of Ruby than pre-installed on MacOS which is Ruby 2-0  to benefit from the latest features from those tools. The default macOS Ruby isn’t great  to manage Rubygems  using bundler. Instead we use Ruby version management tools RVM  to manage Ruby versions with bundler. As a provisioning task, we need to install Ruby and Bundler. Our sample RVM Ansible task looks like this :

---
- name: Install RVM for the user
  command: bash -c "\curl -sSL https://get.rvm.io | bash -s -- --ignore-dotfiles"

- name: Install ruby version
  command: bash -c "~/.rvm/bin/rvm install {{ ruby_version }} --with-zlib-dir={{ zlib_directory }}"

- name: Install Ruby Gems required for iOS app developement
  command: bash -c "~/.rvm/rubies/{{ ruby_version }}/bin/gem install {{item}}"
  with_items: "{{ rubygems_packages_to_install }}"

We can then pass zlib_directory  and rubygems_packages_to_install  from the playbook.

Ansible Task for macOS Defaults

On the build server machine, we need to disable the updates and set the machine in the ‘never sleep’ mode. We can set that from Ansible task and pass the commands from the playbook.

---
- name: Setup macOS Sleep Mode
  shell: "{{ item }}"
  with_items: "{{ macos_sleep_options }}"
  changed_when: false

- name: Software Updates
  shell: "{{ item }}"
  with_items: "{{ macos_software_autoupdates }}"
  changed_when: false

Ansible TeamCity Agent Task

We use TeamCity as our Continuous Integration server. As a final task we have add the machine as TeamCity build agent to TeamCity Server and start the build agent. It involves following tasks

  • Download the TeamCity Agent .zip  package from TeamCity server
  • Add TeamCity Agent configuration inside buildAgent.properties  file
  • Start TeamCity Agent

We need to have Java installed on server machine, in order to setup TeamCity build agent. We also need to create Ansible ninja template for buildAgent.properties.j2 file with parameters teamcity_agent_server_url  and teamcity_agent_name  which can be replaced by default template file. Finally, start the TeamCity build agent. Our TeamCity Ansible task looks something like this:

---
- name: "Download Teamcity Agent Package"
  command: bash -c "curl {{ teamcity_agent_server_url }}/update/buildAgent.zip --output /tmp/buildAgent.zip"
  register: _teamcity_agent_package

- name: "Add TeamCity Agent configuration"
  template:
    src: "buildAgent.properties.j2"
    dest: "{{ teamcity_agent_install_dir }}/conf/buildAgent.properties"
    mode: 0644

- name: Start the Teamcity Agent
  command: bash -c "{{ teamcity_agent_install_dir }}/bin/agent.sh start"

Our Example Playbook

By using all the Ansible tasks mentioned above, we have created a private Ansible role which execute all the tasks. If you interested,we can share it with you.  Our example playbook looks like this:

---
- hosts: localhost
  connection: local
    xcode_src: Xcode_8.3.xip
    ruby_version: ruby-2.3.0
    zlib_directory: /usr/local/Cellar/zlib/1.2.11
    rubygems_packages_to_install:
      - bundler
    teamcity_agent_install_dir: ~/TeamCity/buildAgent
    teamcity_agent_server_url: https://our_teamcity_server.com
    teamcity_agent_name: Our_iOS_Agent

    macos_sleep_options:
      - systemsetup -setsleep Never
      - systemsetup -setharddisksleep Never
      - systemsetup -setcomputersleep Never

    macos_software_autoupdates:
      - softwareupdate --schedule off


    homebrew_installed_packages:
      - autoconf
      - openssl
      - wget
      - zlib
      - curl
      - imagemagick
    homebrew_taps:
      - homebrew/core
      - caskroom/cask
 tasks:
    - include: tasks/ios_ansible.yml

Now that, our playbook is ready to play on any macOS machine. By using this playbook, we managed to get brand new Mac Mini server provisioned with all required software for iOS development and got it attached to TeamCity server within 20-30 min. The most importantly it doesn’t need any manual intervention. Now, we can setup our CI server from scratch in few minutes.

We execute this playbook, on our CI server whenever we want to upgrade to new Xcode version or any other Apple developer tool.  It really helps us resetting and re-building CI server setup easily without spending lot of time.

Benefits

The benefits of using Ansible for provisioning iOS CI are

  • Quick Setup of new CI server machine and ease of upgrading/downgrading Xcode version
  • Ability to reset, upgrade version of software whenever required
  • Decoupled all the configuration from TeamCity GUI into the code.
  • Streamline the configuration of local machines and CI servers.

Conclusion

We have achieved continuous deployment of an iOS apps using  Fastlane as build automation tool and  Ansible as Continuous Integration provisioning a.k.a configuration management tool. What are your experiences of iOS continuous deployment and what do you think of our approach of iOS continuous deployment ?

Weigh in with comment below !

Thanks

Thanks for reading the two part of this posts. Special thanks to Marcus Ramsden (iOS Engineering Manager), Luke Charman (iOS Tech Lead) and Esther Lloyd (Test Manager) for contributing and reviewing this post.

 

Leave a Reply

Your email address will not be published. Required fields are marked *