After some help with drone and the amazon autoscaler

Drone Support
1

I am trying out autoscaler and hitting a few issues my current docker compose config is this.

The server and the autoscaler are both running, there are a few things I am confused about.

The server jobs are being run on the server itself so no agents and not through the auto scaler which surprised me I was not expecting the server to be able to do that.

Should the server be sending things to the autoscaler, the docs do not really mention how the two integrate do they even or is autoscaler completely separate ? is there something in the server that indicates they are talking to each other.

Currently the agent works in that I can query it from the command line, and I can submit an api request to start a new instance and see it pop up in amazon how ever I can not get this same behaviour when triggering a job.

version: '2'
services:
  drone-server:
    image: drone/drone:1.0.0-rc.2
    ports:
      - "9000:9000"
      - "80:80"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./data/drone:/var/lib/drone/
    env_file: .env

  drone-autoscale:
    image: drone/autoscaler
    ports:
      - "8080:8080"
    depends_on:
      - drone-server
    volumes:
      - ./data/drone:/var/lib/drone/
    env_file: .env

Anyone got a working config they can share so I can compare ?

2

this would imply you installed Drone using the single-server installation instructions, instead of the multi-server installation instructions. If you want to use the autoscaler you need to use the latter.

as a side note, port 9000 is no longer used and does not need to be opened with 1.0

3

Thanks for that got a few more questions now, you where correct I already had a single server setup as I was trying to add the auto scaler to an existing server.

The drone auto scaler seems to try and connect to docker on the public ip using the docker port this seems incorrect as you would open up the connection to the world can this be specified ?

Also I can see the scaler uses ami-80ca47e6 which seems like an old instance and does not expose the docker connection to outside connections, is this the correct image to be using with the latest verison ?

Also by using a load of really bad security practices and opening up everything to the web I managed to get a connection but it error-ed with this message.

{"arch":"amd64","error":"invalid character '\u003c' looking for beginning of value","level":"warning","machine":"agent-id","msg":"runner: cannot get queue item","os":"linux","time":"2018-12-12T11:06:49Z"}

So my questions are basically.

  1. Can I tell drone to use the internal ip for better security ?
  2. Is the default image above correct and should it just work or has it changed ?
  3. how to resolve the error above ?

my settings are below, obviously I have redacted the important values.

DRONE_ORG=ORGANIZATION_HERE
DRONE_DATABASE_DRIVER=sqlite3
DRONE_DATABASE_DATASOURCE=/var/lib/drone/drone.sqlite
DRONE_LOGS_DEBUG=true
DRONE_SERVER_PROTO=http
DRONE_SERVER_HOST=drone.example.com
DRONE_TLS_AUTOCERT=false
DRONE_USER_CREATE=username:droneadmin,machine:false,admin:true,token:randomtokenhere

# Git repository config
DRONE_REPOSITORY_FILTER=GIT_ORGANIZATION_HERE
DRONE_GITHUB_SERVER=https://github.com
DRONE_GITHUB_CLIENT_ID=CLIENT_ID_HERE
DRONE_GITHUB_CLIENT_SECRET=CLIENT_SECRET_HERE
DRONE_GIT_ALWAYS_AUTH=true

# drone screts and rpc keys
DRONE_SECRET=DRONE_RANDOM_SECRET_HERE
DRONE_SERVER_TOKEN=DRONE_RANDOM_SECRET_HERE
DRONE_RPC_SECRET=DRONE_RPC_RANDOM_SECRET_HERE
DRONE_AGENT_TOKEN=DRONE_RPC_RANDOM_SECRET_HERE

# Drone Autoscaler config, access keys
DRONE_POOL_MIN=1
DRONE_POOL_MAX=3
DRONE_RUNNER_CAPACITY=2
AWS_ACCESS_KEY_ID=AWS_ACCESS_KEY_HERE
AWS_SECRET_ACCESS_KEY=AWS_SECRET_ACCESS_KEY_HERE

# Amazon new instance config
DRONE_AMAZON_REGION=eu-west-2
DRONE_AMAZON_SUBNET_ID=subnet-id
DRONE_AMAZON_SECURITY_GROUP=security-group-id
DRONE_AMAZON_SSHKEY=amazon-ssh-key-name
DRONE_AMAZON_TAGS=name:drone-agent,owner:instance-owner
# DRONE_AMAZON_IMAGE=ami-01bee3897bba49d78
4

This is by design. Drone behaves in the same way as Docker Machine and allows you to connect to the remote Docker instance from your local machine for debugging: https://autoscale.drone.io/cli/drone-server-env/. The daemon is configured with private key authentication and cannot be accessed by unauthorized users. But if you prefer private IP you can set the DRONE_AMAZON_PRIVATE_IP=true flag.

Also I can see the scaler uses ami-80ca47e6 which seems like an old instance and does not expose the docker connection to outside connections, is this the correct image to be using with the latest verison ?

The link you provided is to a unit test. The default image list is found here: