CI/CD platform

This article will help you to find the fastest way to get the most value out of Last.Backend CI/CD.

CI/CD User guide

Last.Backend CI/CD platform is a powerful system allows you to build and test your source code. To start building you need to put .lb.yml file in the root of your repository.

This short guide will help you to setup CI/CD for your repository.

The .lb.yml file contains 4 sections:

  • workspace

  • pipeline

  • services

  • registry



Workspace


Workspace section describes the build and test environments. Here you can set up the working directory, where your code will be placed. This section is not required. If not presented - the default path:data will be used.

Reference

  • Path: Path parameter defines the root directory where your source code will be pulled and stored for steps and plugins.

Example

1workspace:
2  path: /data



Steps


The steps section defines the test and build process, executed step-by-step. The top-level steps section defines sets of steps that are executed sequentially. Each step starts in a new container inside pipeline pod.

The first pipeline step - is your repository cloning process. It defines automatically, so do not need to setup it manually.

After repo clone, the pipeline will be executed in steps definition order.

Step Example

 1workspace:
 2  path: /data
 3
 4steps:
 5  - name: image
 6    image: node:latest
 7    commands:
 8      - npm test
 9      - npm build
10    when:
11      event:
12        - push
13        - pull_request
14        - tag
15  - name: plugin
16    plugin: s3
17    settings:
18      BUCKET: demo
19      TOKEN: token
20    when:
21      event:
22        - push
23        - pull_request
24        - tag

The commands are executed in the workspace directory. This directory is described in workspace section and is available on each step in the pipeline.

Last.Backend CI/CD uses container exit code to define step execution status. If the container’s exit code is non-zero - that means that step is failed.

Steps can be 2 types: from container images or from plugins.

Image Reference

Container image step reference:

  • Name required: Step name. Required option.

  • Image required: Container image to be run as a service. Last.Backend pipeline supports all types of container registries including private too.

  • Commands required: Commands describe exec commands executed inside the container. These commands are converted in a simple shell script.

  • Environments: Environments are the set of “key:value” for the container. Not that boolean values will be converted to quoted strings.

  • When: Conditional option to define when step should be executed or not. Last.Backend supports git hook type as a condition in this version in the event options, and the branch condition in the same name option.

Plugin Reference

Plugin step reference:

  • Name required: Step name. Required option.

  • Plugin required: Plugin name to execute. Last.Backend search plugin in plugin list by its name.

  • Settings required: option for plugin type. Available plugin settings are described on the plugin page. Note that some settings are required for plugin runtime.

  • Environments: Environments are the set of “key:value” for the container. Not that boolean values will be converted to quoted strings.

  • When: Conditional option to define when step should be executed or not. Last.Backend supports git hook type as a condition in this version in the event options, and the branch condition in the same name option.



Services


Services are additional containers needed to test and build your source code. These containers should be started in the background at the beginning of your pipeline. The common reason to use services is if your tests need some additional apps like databases or caches or whatever.

Service Example

Here is a typical service example:

1workspace:
2  path: /data
3
4services:
5  - name: postgres
6    image: postgres

Service containers are available to access by localhost and. In the previous example, you can connect to postgres at tcp://localhost:5432.

After pipeline execution service containers will be destroyed and all data will be erased.

Service Reference

  • Name required: Service name.

  • Image required: Container image to run as a service. Last.Backend pipeline supports all types of container registries including private too.

  • Command: Command describes exec command to start your service

  • Environments: Environments are the set of “key:value” for the container. Not that boolean values will be converted to quoted strings.



Registry


The registry is a special section describes build and push container images from source code. This is not a required section. With Last.Backend CI/CD you can build and push multiple container images from one repository.

Registry Example

 1workspace:
 2  path: /data
 3
 4registry:
 5  - name: lastbackend
 6    settings:
 7      auto_tag: true
 8      dockerfile: ./Dockerfile
 9      context: .
10    when:
11      branch:
12        - master
13      event:
14        - push

Last.Backend CI/CD is supported build and push images only in an internal Last.Backend container hub. If you need to push images into your own registry - ping us, we’ll add this feature.

Registry Reference

  • Name required: Container image name to build and push. Required option.

  • Tags: You can provide a list of images tags for this container image.

  • Settings: this option describes container image build settings. Here you can set up:

    • auto_tag boolean: this option sets container image tag from build code branch. For example, if the worker build a container image from master branch - the container image tag will be master.

    • auto_tag_suffix string: adds additional suffix before image tag. For example, if the source code is from master branch and auto_suffix_tag is “lb-”, then the tag name for container image will be lb-master

    • dockerfile string: the path for dockerfile the build worker will use for image build. If not provided, the system will build an image from dockerfile located in the root of the repository.

    • context string: the context path for the image build process. If this option is not provided, the context path will be the root of the build repository.

  • When: Conditional option to define when step should be executed or not. Last.Backend supports git hook type as a condition in this version in the event options, and the branch condition in the same name option.



Examples


Here are some popular examples you can use in your pipeline.

Language Examples


C

1steps:
2- name: test
3  image: gcc
4  commands:
5  - ./configure
6  - make
7  - make test

C++

1steps:
2- name: test
3  image: gcc
4  commands:
5  - ./configure
6  - make
7  - make test

Clojure

1steps:
2- name: test
3  image: clojure
4  commands:
5  - lein test

D

1steps:
2- name: test
3  image: dlanguage/dmd
4  commands:
5  - dub test

Dart

1steps:
2- name: test
3  image: google/dart
4  commands:
5  - pub get
6  - pub run test

Elixir

1steps:
2- name: test
3  image: elixir:1.4
4  commands:
5  - mix local.rebar --force
6  - mix local.hex --force
7  - mix deps.get
8  - mix test

Erlang

1steps:
2- name: test
3  image: erlang:21
4  commands:
5  - rebar get-deps
6  - rebar compile
7  - rebar skip_deps=true eunit

Go

1workspace:
2  path: /go/src/github.com/lastbackend/lastbackend
3
4steps:
5- name: test
6  image: golang
7  commands:
8  - go get
9  - go test

Gradle

1steps:
2- name: test
3  image: gradle:jdk10
4  commands:
5  - gradle assemble
6  - gradle check

Groovy

1steps:
2- name: test
3  image: gradle:2.5-jdk8
4  commands:
5  - ./gradlew assemble
6  - ./gradlew check

Haskell

1steps:
2- name: test
3  image: haskell
4  commands:
5  - cabal install --only-dependencies --enable-tests
6  - cabal configure --enable-tests
7  - cabal build
8  - cabal test

Haxe

1steps:
2- name: test
3  image: haxe
4  commands:
5  - haxelib install build.hxml
6  - haxe build.hxml

Java

1steps:
2- name: test
3  image: maven:3-jdk-10
4  commands:
5  - mvn install -DskipTests=true -Dmaven.javadoc.skip=true -B -V
6  - mvn test -B

Maven

1steps:
2- name: test
3  image: maven:3-jdk-10
4  commands:
5  - mvn install -DskipTests=true -Dmaven.javadoc.skip=true -B -V
6  - mvn test -B

Node

1steps:
2- name: test
3  image: node
4  commands:
5  - npm install
6  - npm test

PHP

 1steps:
 2- name: install
 3  image: composer
 4  commands:
 5  - composer install
 6
 7- name: test
 8  image: php:7
 9  commands:
10  - vendor/bin/phpunit --configuration config.xml

Perl

1steps:
2- name: test
3  image: perl
4  commands:
5  - cpanm --quiet --installdeps --notest .
6  - perl Build.PL
7  - ./Build test

Python

1steps:
2- name: test
3  image: python
4  commands:
5  - pip install -r requirements.txt
6  - pytest

R

1steps:
2- name: test
3  image: r-base
4  commands:
5  - R -e 'install.packages(c("package1","package2"))'
6  - R CMD build .

Ruby

1steps:
2- name: test
3  image: ruby
4  commands:
5  - bundle install --jobs=3 --retry=3
6  - rake

Rust

1steps:
2- name: test
3  image: rust:1.30
4  commands:
5  - cargo build --verbose --all
6  - cargo test --verbose --all

Swift

1steps:
2- name: test
3  image: swift:4
4  commands:
5  - swift build
6  - swift test



Service Examples


CouchDB

 1steps:
 2- name: test
 3  image: couchdb:2.2
 4  commands:
 5  - sleep 15
 6  - curl http://127.0.0.1:5984
 7
 8services:
 9- name: couchdb
10  image: couchdb:2.2

ETCD

 1steps:
 2- name: test
 3  image: alpine:3.8
 4  commands:
 5  - apk add curl
 6  - sleep 15
 7  - curl http://127.0.0.1:2379
 8
 9services:
10- name: etcd
11  image: quay.io/coreos/etcd:latest
12  command: "/usr/local/bin/etcd --data-dir=/etcd-data --name node --initial-advertise-peer-urls
13            http://127.0.0.1:2380 --listen-peer-urls http://127.0.0.1:2380 --advertise-client-urls
14            http://127.0.0.1:2379 --listen-client-urls http://127.0.0.1:2379 --initial-cluster
15            node=http://127.0.0.1:2380"

ElasticSearch

 1steps:
 2- name: test
 3  image: alpine:3.8
 4  commands:
 5  - apk add curl
 6  - sleep 15
 7  - curl http://127.0.0.1:2379
 8
 9services:
10- name: elastcsearch
11  image: elasticsearch:5-alpine

MariaDB

 1steps:
 2- name: test
 3  image: mariadb
 4  commands:
 5  - sleep 15
 6  - mysql -u root -h 127.0.0.1 --execute="SELECT 1;"
 7
 8services:
 9- name: mariadb
10  image: mariadb
11  environments:
12   - MYSQL_ALLOW_EMPTY_PASSWORD: yes
13   - MYSQL_DATABASE: test

Memcached

 1steps:
 2- name: test
 3  image: ubuntu
 4  commands:
 5  - apt-get update -qq
 6  - apt-get install -y -qq telnet > /dev/null
 7  - (sleep 1; echo "stats"; sleep 2; echo "quit";) | telnet 127.0.0.1 11211 || true
 8
 9services:
10- name: memcached
11  image: memcached:alpine
12  command: memcached -vv

MongoDB

 1steps:
 2- name: test
 3  image: mongo:4
 4  commands:
 5  - sleep 15
 6  - mongo --host 127.0.0.1 --eval "db.version()"
 7
 8services:
 9- name: mongo
10  image: mongo:4
11  command: mongod --smallfiles

MySQL

 1steps:
 2- name: test
 3  image: mysql
 4  commands:
 5  - sleep 15
 6  - mysql -u root -h 127.0.0.1 --execute="SELECT 1;"
 7
 8services:
 9- name: mysql
10  image: mysql
11  environments:
12    MYSQL_ALLOW_EMPTY_PASSWORD: yes
13    MYSQL_DATABASE: test

Nats

 1steps:
 2- name: test
 3  image: ruby:2
 4  commands:
 5  - gem install nats
 6  - nats-pub -s tcp://127.0.0.1:4222 greeting 'hello'
 7  - nats-pub -s tcp://127.0.0.1:4222 greeting 'world'
 8
 9services:
10- name: nats
11  image: nats:1.3.0

Postgres

 1steps:
 2- name: test
 3  image: postgres:9-alpine
 4  commands:
 5  - psql -U postgres -d test -h 127.0.0.1
 6
 7services:
 8- name: postgres
 9  image: postgres:9-alpine
10  environments:
11    POSTGRES_USER: postgres
12    POSTGRES_DB: test

Redis

 1steps:
 2- name: test
 3  image: redis
 4  commands:
 5  - sleep 5
 6  - redis-cli -h 127.0.0.1 ping
 7  - redis-cli -h 127.0.0.1 set FOO bar
 8  - redis-cli -h 127.0.0.1 get FOO
 9
10services:
11- name: redis
12  image: redis

Rethink

 1steps:
 2- name: test
 3  image: node:9
 4  commands:
 5  - npm install -s -g recli
 6  - recli -h 127.0.0.1 -j 'r.db("rethinkdb").table("stats")'
 7
 8services:
 9- name: rethink
10  image: rethinkdb:2