In software engineering, Continuous integration (CI) is the practice of merging all developers' working copies to shared mainline several times a day. Continuous delivery (CD) is a software engineering approach in which teams produce software in short cycles, ensuring that the software can be reliably released at any time. Both Continuous integration and Continuous Delivery are widely used practices and they are considered to be essential building blocks of DevOps adoption. We have earlier wrote how to set-up two-way integrations with Git feature branch workflow and Jenkins and TeamCity CI servers.
With the emergence of the cloud and Software as a Service, development teams don't necessarily want to host tools on their own premises. Similarly, as Deveo supports both on-premises installations, as well as cloud usage through a multi-tenant SaaS instance, where you still keep your code in private, there are dozens of hosted continuous integration and continuous delivery services available. These services offer the development teams the whole package from setting up, configuring, and running your builds and tests as well as deployment.
There are already exhaustive listings of hosted continuous integration services available from Yegor Bugayenko and James Brooks. Neither of the listings distinguished which repository hosting solutions and version control systems the services support. Most of the hosted continuous integration services only support GitHub and additionally BitBucket. There are other repository hosting services available, Deveo being one of them. Thus in this blog post, we wanted to cover four hosted CI services that support any code hosting platform or even your self-hosted repositories (given they are accessible from the internet). This post evaluates the services mostly from the ease of taking the service to use point of view.
Solano Labs supports Git and Mercurial repositories. At least with Deveo, setting up Solano Labs CI build is done by adding a new repository, choosing the "Commit hook" option, copy-pasting your SSH repository clone URL, adding the Solano SSH-key to Deveo bot account, and finally, adding a Webhook for the repository with the post-commit webhook URL. In addition to SSH, Solano also allows you to use username and password for repository authentication.
In Solano, most of the stuff is configured in
solano.yml file. The full reference for the configuration file can be found from their official documentation. In addition to the
solano.yml file, the web UI offers some separate integrations, such as Flowdock, New Relic and Code Climate. You can also create a status badge through the web UI that you can link to your
Defining post build actions in Solano to integrate it with existing tools is done using Post-Build Hooks. The post-build hooks can be used to send build status notifications, to Team chat tools, or in our case integrate the build status to Deveo's feature branch and code review workflow. At the time of writing, Solano didn't offer a way to setting environment variables through the web UI, but enforced to either use
solano config command-line tool or adding them explicitly to
solano.yml configuration file.
- Basic build setup is easy using
- No Subversion support
- Cannot set environment variables from the web UI
- No free plans, only 14-day free trial
Magnum CI supports Git, Subversion and Mercurial repositories. The configuration for Mercurial repository happens by the exact same steps as you would expect; copy-paste the repository URL, add the deploy key from Magnum CI and finally add the post-commit hook to the repository.
Unfortunately, I was unable to test Magnum CI further in action because of some difficulties in the configuration phase. According to their documentation, builds are configured either using
magnum.yml or through their web UI.
Magnum CI offers a way to notify external services either with a custom script that is launched in After test execution or through their generic webhook. I couldn't find a way to configure custom environment variables, which means everything needs to be stored to version control system, or if using their generic webhook, to your service of choice.
Magnum CI has tighter deployment integrations to some 3rd party services. Magnum CI supports deploying directly to Heroku, and Capistrano in addition to more typical bash script invocation. Notification integrations include Slack, HipChat, Flowdock, Campfire, and IRC.
- Supports Git, Subversion, and Mercurial
- Supports generic build status webhook
- Ability to configure the build steps through Web UI in addition to configuration file
- Problems when configuring and testing the build
- No ability to store environment variables
- Currently in beta
DeployBot is a product from Wildbit LLC. It is deployment tool rather than a full-fledged CI, but regardlessly, earns its place in the blog post. Adding a repository happens either with SSH key or username and password. The SSH key was offered as a download instead of copy-pasting, which is peculiar user experience choice, the downloaded key also contained one extra whitespace character in the end, which made Deveo complain about it when copy-pasting it from the file.
In DeployBot, you configure environments where you deploy to. You need to define branch to be built for each environment, which makes it hard to integrate it to a feature branch based workflow. The ability to deploy based on commit messages helps in deploying, though.
DeployBot has many niceties, such as support for invoking deployments with commit messages. You can write
[deploy: production] into your commit message, and DeployBot starts a production deployment for example. Another nice thing is parameterized webhooks for commencing exact deployments. In addition, DeployBot also supports badges that can be added to
README.md files or any other place of convenience.
- Commit message based deployments
- Parameterized webhooks
- Many configuration options
- Does not offer free tier or trial
- minor SSH key inconveniences
- Only Git support
Appveoyr approaches the continuous integration and delivery spectrum from a different angle, as it offers continuous delivery service for Windows. Appveyor supports all Git, Subversion and Mercurial, and authentication works either through SSH key, or credentials.
For initiating the builds, Appveyor uses webhook URL that is easy to setup. It also offers possibilities to configure, whether to ignore the
appveoyr.yml configuration, or skip building such branches that don't contain
appveoyr.yml configuration file. You may find the full documentation for Appveyor configuration file from their official documentation.
Since Appveyor is about CI on Windows, it has configuration options either for MSBuild, or custom scripts. Most of the stuff that is configurable in both the configuration file or through the Web UI. Appveyor has notification options such as email, webhook, HipChat, Slack, and CampFire. It also allows you to publish a badge you can add to your
README.md file for example. Built-in deployment providers are fairly simple, such as Web deploy, FTP or Amazon S3.
- Configuration from both Web UI and
- Support for Git, Subversion, and Mercurial
- Ability to add before and after steps for each phase of execution
- Environment variables can be configured from both Web UI, and configuration file
- Windows only
There are at least four different hosted continuous integration services available that support Deveo or any other repository hosting service. The features and the maturity of the services differ, but most of the services allow you to get things done. This evaluation was merely a scratch on the surface and only used very basic test setup. If you use any of the services, you'll probably notice more severe differences among them.
We are going to write step-by-step guides on setting up the aforementioned services with Deveo. If you have a preference over which service we should write a guide about, post a comment below.