InIT Build Server

Availability

The build server is reachable at https://ci.cloudlab.zhaw.ch/.

It can be used free of charge by all InIT employees, their project collaborators as well as all lecturers and students participating in computer science courses at the ZHAW School of Engineering.

Getting Started

Warning

Our build server runs the on-premise version of AppVeyor, called AppVeyor Enterprise. Do not confuse the two. The software works the same, but neither the accounts nor the servers nor the build images are shared.

Getting an Account

  1. Write an email to ci.init@zhaw.ch using your ZHAW email address.

  2. As soon as we have created your account, you’ll receive an invitation from “AppVeyor Enterprise” with a sign-up link. Click on the link and complete the creation of your account by completing the form.

Configuring Your Build

  1. Place a file called appveyor.yml in the root of your project.

  2. Configure your build by editing the appveyor.yml.
    1. Use the template below as a starting point or a file from one of our sample projects.

    2. Select the desired build image from the list of Available Build Images.

    3. See appveyor.yml Reference for the full list of options.

  3. On AppVeyor Enterprise, click on Projects > New Project.
    1. Select the SCM provider of your project. If you haven’t configured it yet, see Configuring SCM Integrations for details.

    2. Click on Add next to the project you like to build.

    3. If you want AppVeyor Enterprise to automatically build your project on every source code change, see Configuring Build Triggers for details.

    4. To grant your team members access to the project, check out Team Management.

appveyor.yml Template
image: ubuntu-18.04-amd64

install:
  - echo "Install additional dependencies"

build_script:
  - echo "Build your software"

test_script:
  - echo "Setup additional services if necessary"
  - echo "Run the test suite"

artifacts:
  - path: "build/libs/*.jar"

on_finish:
  - echo "Upload the test reports"

Replace all echo commands with the actual commands that drive your build (e.g. mvn clean package). The commands are directly fed to Bash (Linux) or PowerShell (Windows), respectively.

Sample Projects

We have prepared a set of sample projects to help you get started.

Documentation

The full documentation for the SaaS version of AppVeyor is available on https://www.appveyor.com/docs/. Keep in mind, that:

If you’re still stuck after reading the documentation, see the section Getting Help for how to get help.

Hardware, Network, Limits

The current build worker configuration is as follows:

  • Maximum build run time: 60 minutes (can be increased on an individual basis)

  • Maximum number of parallel jobs per user: 4 (can be increased on an individual basis)

The cluster operates in the IP range 160.85.254.192/26. If you need to access resources within the ZHAW network, you have to open a FWCR and apply for access from 160.85.254.192/26. All other InIT services with the exception of the OpenStack cluster Ned are accessible without additional changes.

Linux / Windows

  • 3 CPU cores Intel Xeon Skylake-SP at 3.4 GHz, 8 GB RAM, 256 GB local storage (SSD in a RAID-10)

  • Full hardware virtualization with nested virtualization enabled

macOS

  • 3 CPU cores Intel Core i7 (Coffee Lake) at 3.2 GHz, 8 GB RAM, 512 GB local storage (SSD)

  • Full hardware virtualization

Build Images

Available Build Images

Name

Description

ubuntu-18.04-amd64 (default)

Ubuntu 18.04 Bionic Beaver on x86_64

ubuntu-20.04-amd64

Ubuntu 20.04 Focal Fossa on x86_64

w2k19-vs19-amd64

Windows Server 2019 Standard with Visual Studio 2019 Community on x86_64

macos-10.15

macOS Catalina 10.15.4

Linux

Environment

Builds are run as user appveyor in its home directory /home/appveyor. The projects are cloned into the directory /home/appveyor/projects/<project-name>. The working directory at the beginning of the build is /home/appveyor/projects/<project-name>.

The build workers are immutable. This means that at the beginning of every build the build worker is in the same state. Changes made during the build are discarded after every build.

Available Software

The images are updated on a monthly schedule. Do you miss something? Feature requests are welcome at ci.init@zhaw.ch.

Available Software on Linux Images

Software

ubuntu-18.04-amd64

ubuntu-20.04-amd64

Go

1.13.7 (gvm)

Java

Android SDK

Apache Maven

Gradle

OpenJDK 8 (AdoptOpenJDK)

OpenJDK 8 (Ubuntu)

OpenJDK 8 (Zulu)

OpenJDK 11 (AdoptOpenJDK)

✔, default

✔, default

OpenJDK 11 (Ubuntu)

OpenJDK 11 (Zulu)

OpenJDK 14 (AdoptOpenJDK)

OpenJDK 14 (Zulu)

.NET Core

.NET Core SDK 2.1

.NET Core SDK 3.1

Node

NodeJS 10

NodeJS 12

✔, default

✔, default

NodeJS 13

NodeJS 14

yarn 1.x

Python

Python 2.7 (pyenv)

Python 2.7 (Ubuntu)

Python 3.5 (pyenv)

Python 3.6 (pyenv)

Python 3.6 (Ubuntu)

Python 3.7 (pyenv)

Python 3.8 (pyenv)

✔, default

✔, default

Python 3.8 (Ubuntu)

Ruby

2.7 (rvm)

Rust

stable

✔, default

✔, default

Tools

Docker CE

libvirt/qemu-kvm

git-lfs

hub

jfrog-cli

Packer

PowerShell Core 7.x

zip

7zip

Web Browsers

Chromium

Firefox

Android

The Android SDK is pre-installed and available on $PATH. For the emulator, use x86 images to benefit from hardware acceleration. The program emuwait stops the execution of the build until the emulator is ready.

The following code block configures an AVD with Android 4.2 and Google APIs:

install:
  - sudo java_select adoptopenjdk-8-hotspot-amd64
  - yes | sdkmanager --licenses >/dev/null
  - sdkmanager --update
  - sdkmanager 'system-images;android-17;google_apis;x86'
  - avdmanager create avd -k 'system-images;android-17;google_apis;x86' -d 'Nexus 4' -n 'JellyBean_GoogleAPI_x86'

before_test:
  - emulator -avd 'JellyBean_GoogleAPI_x86' -accel on -no-boot-anim -noaudio -no-window &
  - emuwait
Go

Go is made available via gvm. To activate a specific version of Go (e.g. 1.12 in this example):

install:
  - eval "$(gvm 1.12)"

Afterwards, go, godoc and gofmt are on the $PATH.

Java

Due to licensing issues, only variants of OpenJDK are available. You have the choice between the JDKs shipped with Ubuntu, AdoptOpenJDK and Azul Zulu. To use another JDK than the current default, use java_select, e.g.:

install:
  - sudo java_select adoptopenjdk-11-hotspot-amd64

To see a full list of available JDKs, invoke java_select -l.

Tip

It is recommended to run sudo java_select <your-preferred-jdk> at the beginning of your build because the default JDK may change over time.

Note

You can safely ignore errors that mention that some alternative is not available. The programs shipped with the JDKs vary from version to version. Therefore it’s possible that there isn’t an alternative for a program.

For repeatable builds, it is recommended to use Maven Wrapper and Gradle Wrapper, respectively, if you use either Maven or Gradle.

NodeJS

NodeJS and npm are made available via nvm. You have the choice between multiple versions of NodeJS (and npm). To use another version of NodeJS, use nvm, e.g.:

install:
  - nvm use 11

See the Node Version Manager homepage for detailed instructions.

Tip

It is recommended to run nvm use <your-preferred-node-version> at the beginning of your build because the default NodeJS version may change over time.

Python

Python and related tools are made available via pyenv. You have the choice between multiple versions of Python. To use another version of Python, use python_select, e.g.:

install:
  - sudo -i python_select 3.8@latest

To see a full list of available Python versions, invoke python_select -l. See the pyenv homepage for further instructions and for how to install additional versions of Python.

Tip

It is recommended to run sudo -i python_select <your-preferred-python-version> at the beginning of your build because the default Python version may change over time.

pip is managed by pyenv, too. To install dependencies globally, you have to invoke sudo -i pip install <package>. Most of the time it might be easier to install them locally using pip install --local <package>.

Ruby

Ruby is made available via rvm. To activate a specific version of Ruby (2.6.3 in this example):

install:
  - rvm use 2.6.3

ruby, gem are on the $PATH.

Rust

Rust is made available via rustup. To activate a specific version of Rust (stable in this example):

install:
  - rustup default stable

rustc, cargo are on the $PATH.

Windows

Available Software

The images are updated on a monthly schedule. Do you miss something? Feature requests are welcome at ci.init@zhaw.ch.

Available Software on Windows Images

Software

w2k19-vs19-amd64

Databases

SQL Server 2017 Developer

SQL Server 2019 Developer

Java

AdoptOpenJDK 8 (HotSpot, OpenJ9)

Azul Zulu 8 (HotSpot)

AdoptOpenJDK 11 (HotSpot, OpenJ9)

✔, default (HotSpot)

Azul Zulu 11 (HotSpot)

AdoptOpenJDK 14 (HotSpot, OpenJ9)

Azul Zulu 14 (HotSpot)

.NET

.NET Framework 4.6

.NET Framework 4.6.1

.NET Framework 4.6.2

.NET Framework 4.7

.NET Framework 4.7.1

.NET Framework 4.7.2

.NET Framework 4.8

.NET Core SDK 2.1

.NET Core SDK 3.1

Node

NodeJS 10

NodeJS 12

✔, default

NodeJS 14

Python

Python 3.7

Python 3.8

✔, default

Tools

Chocolatey

Docker CE

Git

Git LFS

hub

Hyper-V

IIS

jfrog-cli

PowerShell 5.x

Notepad++

nuget

Web Platform Installer

7zip

Web Browsers

Chrome

Firefox

Databases

All available services (see list below) are disabled by default. To enable them, you have to mention them in your build configuration by their service name:

services:
  - MSSQL$SQL2017

The following versions and editions of SQL Server are installed:

  • SQL Server 2017 Developer (service: MSSQL$SQL2017)

  • SQL Server 2019 Developer (service: MSSQL$SQL2019)

.NET and Visual Studio

Visual Studio 2017 Community is installed with the following workloads:

  • Microsoft.VisualStudio.Workload.CoreEditor

  • Microsoft.VisualStudio.Workload.Azure

  • Microsoft.VisualStudio.Workload.ManagedDesktop

  • Microsoft.VisualStudio.Workload.ManagedGame

  • Microsoft.VisualStudio.Workload.NativeDesktop

  • Microsoft.VisualStudio.Workload.NativeGame

  • Microsoft.VisualStudio.Workload.NativeMobile

  • Microsoft.VisualStudio.Workload.NetCoreTools

  • Microsoft.VisualStudio.Workload.NetCrossPlat

  • Microsoft.VisualStudio.Workload.NetWeb

  • Microsoft.VisualStudio.Workload.Universal

All tools are on %Path%.

Visual Studio 2019 Community is installed with the following workloads:

  • Microsoft.VisualStudio.Workload.CoreEditor

  • Microsoft.VisualStudio.Workload.Azure

  • Microsoft.VisualStudio.Workload.ManagedDesktop

  • Microsoft.VisualStudio.Workload.ManagedGame

  • Microsoft.VisualStudio.Workload.NativeDesktop

  • Microsoft.VisualStudio.Workload.NativeGame

  • Microsoft.VisualStudio.Workload.NativeMobile

  • Microsoft.VisualStudio.Workload.NetCoreTools

  • Microsoft.VisualStudio.Workload.NetCrossPlat

  • Microsoft.VisualStudio.Workload.NetWeb

  • Microsoft.VisualStudio.Workload.Node

  • Microsoft.VisualStudio.Workload.Universal

  • Microsoft.VisualStudio.Workload.VisualStudioExtension

All tools are on %Path%.

macOS

Currently, the macOS images are experimental and can change without notice.

Available Software

The images are updated on a monthly schedule. Do you miss something? Feature requests are welcome at ci.init@zhaw.ch.

Available Software on macOS Images

Software

macos-10.15

Go

1.14 (gvm)

Java

Apache Maven

Gradle

AdoptOpenJDK 8 (HotSpot)

✔, default

AdoptOpenJDK 11 (HotSpot)

AdoptOpenJDK 13 (HotSpot)

AdoptOpenJDK 14 (HotSpot)

Node

NodeJS 10

NodeJS 12

✔, default

NodeJS 13

yarn 1.x

Python

Python 2.7.17 (pyenv)

Python 3.5.9 (pyenv)

Python 3.6.10 (pyenv)

Python 3.7.7 (pyenv)

Python 3.8.2 (pyenv)

✔, default

Ruby

2.7 (rvm)

Rust

stable

✔, default

Tools

Git

Git LFS

Fastlane

Packer

Web Browsers

Chromium

Firefox

Safari

XCode

Xcode 11.4.1

iOS Simulator 13.3

watchOS Simulator 6.1.1

tvOS Simulator 13.3

Carthage

CocoaPods

Go

Go is made available via gvm. To activate a specific version of Go (e.g. 1.12 in this example):

install:
  - eval "$(gvm 1.12)"

Afterwards, go, godoc and gofmt are on the $PATH.

Java

Due to licensing issues, only AdoptOpenJDKs are available. To use another JDK than the current default, use java_select, e.g.:

install:
  - java_select adoptopenjdk-11

To see a full list of available JDKs, invoke java_select -l.

Tip

It is recommended to run sudo java_select <your-preferred-jdk> at the beginning of your build because the default JDK may change over time.

For repeatable builds, it is recommended to use Maven Wrapper and Gradle Wrapper, respectively, if you use either Maven or Gradle.

NodeJS

NodeJS and npm are made available via nvm. You have the choice between multiple versions of NodeJS (and npm). To use another version of NodeJS, use nvm, e.g.:

install:
  - nvm use 12

See the Node Version Manager homepage for detailed instructions.

Tip

It is recommended to run nvm use <your-preferred-node-version> at the beginning of your build because the default NodeJS version may change over time.

Python

Python and related tools are made available via pyenv. You have the choice between multiple versions of Python. To use another version of Python, use python_select, e.g.:

install:
  - sudo -i python_select 3.8@latest

To see a full list of available Python versions, invoke python_select -l. See the pyenv homepage for further instructions and for how to install additional versions of Python.

Tip

It is recommended to run sudo -i python_select <your-preferred-python-version> at the beginning of your build because the default Python version may change over time.

pip is managed by pyenv, too. To install dependencies globally, you have to invoke sudo -i pip install <package>. Most of the time it might be easier to install them locally using pip install --local <package>.

Ruby

Ruby is made available via rvm. To activate a specific version of Ruby (2.7 in this example):

install:
  - rvm use 2.7

ruby, gem are on the $PATH.

Rust

Rust is made available via rustup. To activate a specific version of Rust (stable in this example):

install:
  - rustup default stable

rustc, cargo are on the $PATH.

Guides

Configuring SCM Integrations

GitHub Enterprise (ZHAW)

In GitHub Enterprise:

  1. Go to https://github.zhaw.ch/settings/applications/new to create a new OAuth application.

  2. Enter the following values into the form:
    • Application name: AppVeyor Enterprise

    • Homepage URL: https://ci.cloudlab.zhaw.ch/

    • Authorization callback URL: https://ci.cloudlab.zhaw.ch/githubenterprise

  3. Click on Register application.

  4. Keep the Client ID and the Client secret. You need to enter both in AppVeyor Enterprise.

In AppVeyor Enterprise:

  1. Go to Settings > Authorizations > GitHub Enterprise.

  2. Enter the following values into the form:
    • GitHub Enterprise URL: https://github.zhaw.ch/

    • Authentication: OAuth

    • Client ID: enter the value displayed by GitHub Enterprise

    • Client secret: enter the value displayed by GitHub Enterprise

  3. Click on Authorize GitHub Enterprise.

GitHub

On GitHub:

  1. Go to https://github.com/settings/tokens to create a new personal access token.

  2. Click on Generate new token.

  3. Enter the following values into the form:
    • Token description: AppVeyor Enterprise ZHAW

    • Select repo scope for private repositories and read:org, admin:repo_hook, admin:public_key, repo:status for public repositories.

  4. Click on Generate token.

  5. Keep the token. You need to enter it in AppVeyor Enterprise.

In AppVeyor Enterprise:

  1. Go to Settings > Authorizations > GitHub.

  2. Enter the following values into the form:
    • Authorize access to GitHub with: Personal access token

    • Token: enter the value displayed by GitHub

  3. Click on Authorize GitHub.

Configuring Build Triggers

GitHub and GitHub Enterprise

In AppVeyor Enterprise:

  1. Click on Projects and then on the project you like to configure the trigger for.

  2. Go to Settings > General.

  3. In the middle of the page, copy the Webhook URL. It looks like https://ci.cloudlab.zhaw.ch/api/githubenterprise/webhook?id=abcd1234.

In GitHub or GitHub Enterprise, in the project you like to configure the trigger for:

  1. Go to Settings > Hooks.

  2. Click on Add webhook.

  3. Paste the Webhook URL from AppVeyor Enterprise into the field Payload URL.

  4. Leave the other settings alone and click on Add webhook.

Team Management

If you are working on a project with a team, you have to add the other team members to your account as collaborators in order to give them access to the projects.

For ZHAW employees and students, proceed as follows:

  1. If they do not have an account yet, ask them to open one (see Getting an Account).

  2. On the AppVeyor Enterprise interface, go to Settings > Team.

  3. Click on Invite User and complete the form:
    1. Enter the ZHAW email address of your team member (abcd@zhaw.ch or abcdefgh@students.zhaw.ch).

    2. Select the desired role for the new team member.

  4. Your team member receives an email invitation from “AppVeyor Enterprise” with a sign-up link.

  5. Your team member has to click on the sign-up link. If they are prompted to sign-in, they should do so with their normal account credentials. They should now be able to see your projects.

  6. Repeat these steps for all other team members.

Tip

One of the restrictions of the User role is that it does not allow to start builds manually. If your team members should be able to start builds manually, you have to create a separate role and assign them to this role. See Set up your team for detailed instructions on how to do so.

For people that are not members of ZHAW, perform the following steps:

  1. On the AppVeyor Enterprise interface, go to Settings > Team.

  2. Click on Invite User and complete the form:
    1. Enter the email address of your team member.

    2. Select the desired role for the new team member.

  3. Your team member receives an email invitation from “AppVeyor Enterprise” with a sign-up link.

  4. Your team member has to click on the sign-up link and to fill-in the sign-up form. Afterwards, they should be able to see your projects.

  5. Repeat these steps for all other team members.

Warning

If you add people to your account that are not ZHAW members, you are responsible for their actions.

Becoming root

By default, all commands are run under the appveyor user. If you need to run a command as root, you can do so by using passwordless sudo. Prepending sudo is sufficient when interacting with system tools (apt, …) and programs installed in standard locations (/bin, /usr/bin, etc.). But some commands require a proper login shell and changes to the environment. In those cases use sudo -i <command>. For example, this applies to all Python tools that are managed by pyenv (python, pip, …) or NodeJS if it’s used via nvm (node, npm).

Testing with Databases on Linux

We do not pre-install databases because every project needs another version. You have three options:

  • Install the database from Ubuntu’s package sources with apt-get.

  • Install the database from the official package repository provided by the database vendor with apt-get (if available).

  • Use a Docker image.

Tip

For Java projects, we recommend Testcontainers.

Our sample projects show how to configure databases using the different approaches outlined above.

A list of links to official package repositories and Docker images for some popular databases:

RDP to Build Worker (Windows Only)

For debugging purposes, it might be helpful to be able to connect to the build worker with RDP.

To enable RDP, add the following block to your appveyor.yml.

Enable RDP in appveyor.yml
environment:
  APPVEYOR_RDP_PASSWORD: <your password>

on_finish:
  - ps: $blockRdp = $true; iex ((new-object net.webclient).DownloadString('https://github.zhaw.ch/raw/InIT-Operations/ci-scripts/master/enable_rdp.ps1'))

RDP connection details will be displayed in the console. The build worker will remain online until the special “lock” file on the VM’s desktop is removed or the maximum runtime is reached (60 mins).

Access to RDP is only possible from within the ZHAW network or via VPN.

Posting Notifications to Slack (and others)

See the guide to Build Notifications in the AppVeyor documentation.

Getting Help

We’re generally happy to help. But as support is not our main job and we are only a small team, please follow the steps below to keep the amount of support emails low. It might take one to two workdays before you get a reply.

Warning

Do not contact the vendor’s support on your own. In most cases they cannot help you anyway because they do not know our environment. Also refrain from leaking details about our infrastructure on StackOverflow or similar sites.

For Students

  1. Double check this documentation for possible solutions.

  2. Consult the documentation of the tools you’re having trouble with and query your favorite search engine.

  3. Ask the tutor of your lab or the advisor of your thesis for help.

  4. Write an email to ci.init@zhaw.ch with a detailed problem description.

For ZHAW Employees

  1. Double check this documentation for possible solutions.

  2. Consult the documentation of the tools you’re having touble with and query your favorite search engine.

  3. Write an email to ci.init@zhaw.ch with a detailed problem description.