How make the tests run faster?

zulhfreelancer · September 15, 2017, 4:09am

I’m using Drone to run rspec and everything is okay so far. But, I think it’s slow. On average, it took around 5 minutes, 20 seconds to finish.

My .drone.yml:

pipeline:
  build:
    image: zulhfreelancer/ruby_nodejs:latest
    commands:
      - gem update bundler
      - bundle install --path vendor/bundle
      - bundle exec rake db:create db:migrate
      - bundle exec rspec
    environment:
      - RAILS_ENV=test
      - SECRET_TOKEN=abc
      - DB_HOST=postgres
      - DB_USER=pg_user
      - DB_PASSWORD=pg_pass

services:
  postgres:
    image: postgres:9.4.5
    environment:
      POSTGRES_PASSWORD: pg_pass
      POSTGRES_USER: pg_user
      PGDATA: /var/lib/postgresql/data
      POSTGRES_DB: useradmin_test
  redis:
    image: redis:2.8.22

cache:
  - vendor/bundle
  - .git

Any idea or suggestion how to make the tests finish faster?

FYI, I’m running DroneCI on 1GB VPS. I only have Drone service inside that VPS. No other processes running except Drone.

bradrydzewski · September 15, 2017, 4:10am

I am not sure this question can be answered without knowing where the bottlenecks are. Which steps or commands are taking a long time? How long are they taking?

zulhfreelancer · September 17, 2017, 6:02pm

My tests now run faster. Here is my updated .drone.yml file:

pipeline:
  restore-cache:
    image: drillster/drone-volume-cache
    restore: true
    mount:
      - vendor/bundle
      - .git
    volumes:
      - /tmp/cache:/cache

  build:
    image: zulhfreelancer/ruby_nodejs:latest
    commands:
      - gem update bundler
      - bundle install --path vendor/bundle --jobs=5 --retry=3
      - bundle exec rake db:create db:migrate
      - bundle exec rspec
    environment:
      - RAILS_ENV=test
      - DB_HOST=postgres
      - DB_USER=pg_user
      - DB_PASSWORD=pg_pass

  rebuild-cache:
    image: drillster/drone-volume-cache
    rebuild: true
    mount:
      - vendor/bundle
      - .git
    volumes:
      - /tmp/cache:/cache

services:
  postgres:
    image: postgres:9.4.5
    environment:
      POSTGRES_PASSWORD: pg_pass
      POSTGRES_USER: pg_user
      PGDATA: /var/lib/postgresql/data
      POSTGRES_DB: useradmin_test
  redis:
    image: redis:2.8.22

I’m facing another problem. I want the tests to only run when there is new commit inside master and development branch. I looked at the docs and I tried to use the when with branch and event but it doesn’t work for me.

I also tried to add them in all steps inside my pipeline (restore-cache, build and rebuild-cache) but when I push non-master-non-development branch, Drone still get triggered and I can see the clone, postgres and redis running in the Drone web UI (the orange bars).

How to completely skip the entire pipeline when I push to non-master/non-development branch? In other words, how to not invoke Drone when I push new commits to my code repo ie feature/report?

bradrydzewski · September 17, 2017, 8:03pm

See http://docs.drone.io/hooks/

zulhfreelancer · September 18, 2017, 3:41am

I tried that too. Doesn’t work for me.

bradrydzewski · September 18, 2017, 4:09am

using the branches block as shown in the below example is the solution for limiting entire pipeline execution to specific branches.

pipeline:
  build:
    image: golang
    commands:
      - go build
      - go test

+branches: [ master, develop ]

I can confirm this syntax works as documented.

bradrydzewski · September 18, 2017, 4:11am

you can see real-world examples of valid syntax here: https://github.com/drone/drone/blob/master/.drone.yml

also, when claiming things don’t work, please provide example configuration. It is very difficult to provide support when we are not provided steps and samples to reproduce.

zulhfreelancer · September 18, 2017, 11:24am

Okay, seems like it worked for me now. My latest .drone.yml file:

pipeline:
  restore-cache:
    image: drillster/drone-volume-cache
    restore: true
    mount:
      - vendor/bundle
      - .git
    volumes:
      - /tmp/cache:/cache
    when:
      event: [push, pull_request, tag]

  build:
    image: zulhfreelancer/ruby_nodejs:latest
    commands:
      - gem update bundler
      - bundle install --path vendor/bundle --jobs=5 --retry=3
      - bundle exec rake db:create db:migrate
      - bundle exec rspec
    environment:
      - RAILS_ENV=test
      - DB_HOST=postgres
      - DB_USER=pg_user
      - DB_PASSWORD=pg_pass
    when:
      event: [push, pull_request, tag]

  rebuild-cache:
    image: drillster/drone-volume-cache
    rebuild: true
    mount:
      - vendor/bundle
      - .git
    volumes:
      - /tmp/cache:/cache
    when:
      event: [push, pull_request, tag]

services:
  postgres:
    image: postgres:9.4.5
    environment:
      POSTGRES_PASSWORD: pg_pass
      POSTGRES_USER: pg_user
      PGDATA: /var/lib/postgresql/data
      POSTGRES_DB: useradmin_test
    when:
      event: [push, pull_request, tag]
  redis:
    image: redis:2.8.22
    when:
      event: [push, pull_request, tag]

branches: [master, development]

But the whens did not work for me. When I submit PR (and set target branch for the PR branch to master or development), the tests didn’t run.

I want the tests to run when:

master get new commit(s)
development get new commit(s)
someone submit a PR and set the target branch to master or development

Did I miss anything here?

zulhfreelancer · September 18, 2017, 11:25am

I also tried this. But, it doesn’t work for me too:

pipeline:
  restore-cache:
    image: drillster/drone-volume-cache
    restore: true
    mount:
      - vendor/bundle
      - .git
    volumes:
      - /tmp/cache:/cache

  build:
    image: zulhfreelancer/ruby_nodejs:latest
    commands:
      - gem update bundler
      - bundle install --path vendor/bundle --jobs=5 --retry=3
      - bundle exec rake db:create db:migrate
      - bundle exec rspec
    environment:
      - RAILS_ENV=test
      - DB_HOST=postgres
      - DB_USER=pg_user
      - DB_PASSWORD=pg_pass

  rebuild-cache:
    image: drillster/drone-volume-cache
    rebuild: true
    mount:
      - vendor/bundle
      - .git
    volumes:
      - /tmp/cache:/cache

services:
  postgres:
    image: postgres:9.4.5
    environment:
      POSTGRES_PASSWORD: pg_pass
      POSTGRES_USER: pg_user
      PGDATA: /var/lib/postgresql/data
      POSTGRES_DB: useradmin_test
  redis:
    image: redis:2.8.22

branches: [master, development]
event: [push, pull_request, tag]

bradrydzewski · September 18, 2017, 3:12pm

I’m sorry I’m a bit confused

Can you please clarify what is working and what is not working? It would help if I better understand what is working for you, so we can focus on what isn’t working.

Can you clarify what this means? When you say “tests” didn’t run, do you mean the pipeline didn’t run? A specific step in your pipeline didn’t run? Or a specific command didn’t run?

zulhfreelancer · September 19, 2017, 11:51am

Sorry for the misunderstanding.

What I meant was, after add branches: [master, development] like below, the tests (pipeline) only get triggered after I push code to master or development branch (which is what I want and solved now):

redis:
    image: redis:2.8.22
    ...

branches: [master, development]

What is not working now is, the tests (pipeline) didn’t get triggered after I submit Pull Request (PR) and set the target branch for that PR to master or development branch. I tried to add when block like this…:

redis:
  ...
  when:
      event: [push, pull_request, tag]

…in all blocks (restore-cache:, build:, rebuild-cache:, postgres: and redis:) but it didn’t work for me. I also tried to make it like this…:

redis:
    image: redis:2.8.22
    ...
branches: [master, development]
event: [push, pull_request, tag]

…but it didn’t work too.

Yes, what I meant by ‘tests’ is my ‘pipeline’. All steps inside my pipeline didn’t run. I really appreciate your help so that I can trigger the entire pipeline when:

master get new commit(s)
development get new commit(s)
someone submit a PR and set the target branch to master or development

Thank you.

zulhfreelancer · September 20, 2017, 3:04am

@bradrydzewski am I missing anything?