I’m hoping I can get some assumptions validated for me and some help. I have a fully working drone setup with gitea triggering builds. Everything is working, until I try to publish to my own private registry. I can publish to the registry without an issue from the command line, so I’m fairly sure the problem isn’t there.
When I get t the publish step, here is the output:
/usr/local/bin/dockerd --data-root /var/lib/docker --host=unix:///var/run/docker.sock
2 Registry credentials or Docker config not provided. Guest mode enabled.
3 + /usr/local/bin/docker version
4 Client: Docker Engine - Community
5 Version: 19.03.8
6 API version: 1.40
7 Go version: go1.12.17
8 Git commit: afacb8b7f0
9 Built: Wed Mar 11 01:22:56 2020
10 OS/Arch: linux/amd64
11 Experimental: false
12
13 Server: Docker Engine - Community
14 Engine:
15 Version: 19.03.8
16 API version: 1.40 (minimum version 1.12)
17 Go version: go1.12.17
18 Git commit: afacb8b7f0
19 Built: Wed Mar 11 01:30:32 2020
20 OS/Arch: linux/amd64
21 Experimental: false
22 containerd:
23 Version: v1.2.13
24 GitCommit: 7ad184331fa3e55e52b890ea95e65ba581ae3429
25 runc:
26 Version: 1.0.0-rc10
27 GitCommit: dc9208a3303feef5b3839f4323d9beb36df0a9dd
28 docker-init:
29 Version: 0.18.0
30 GitCommit: fec3683
31 + /usr/local/bin/docker info
32 Client:
33 Debug Mode: false
34
35 Server:
36 Containers: 0
37 Running: 0
38 Paused: 0
39 Stopped: 0
40 Images: 0
41 Server Version: 19.03.8
42 Storage Driver: btrfs
43 Build Version: Btrfs v4.7.3
44 Library Version: 101
45 Logging Driver: json-file
46 Cgroup Driver: cgroupfs
47 Plugins:
48 Volume: local
49 Network: bridge host ipvlan macvlan null overlay
50 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
51 Swarm: inactive
52 Runtimes: runc
53 Default Runtime: runc
54 Init Binary: docker-init
55 containerd version: 7ad184331fa3e55e52b890ea95e65ba581ae3429
56 runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
57 init version: fec3683
58 Kernel Version: 4.4.59+
59 Operating System: Alpine Linux v3.11 (containerized)
60 OSType: linux
61 Architecture: x86_64
62 CPUs: 4
63 Total Memory: 15.48GiB
64 Name: d9873a7a7bd3
65 ID: UKRN:Z22Z:AWY4:ZUXX:B77Z:APUM:TBY4:HFK7:I3WN:Q56D:U7CW:YX3C
66 Docker Root Dir: /var/lib/docker
67 Debug Mode: false
68 Registry: https://index.docker.io/v1/
69 Labels:
70 Experimental: false
71 Insecure Registries:
72 127.0.0.0/8
73 Live Restore Enabled: false
74 Product License: Community Engine
75
76 WARNING: No kernel memory limit support
77 WARNING: No kernel memory TCP limit support
78 WARNING: No cpu cfs quota support
79 WARNING: No cpu cfs period support
80 WARNING: bridge-nf-call-iptables is disabled
81 WARNING: bridge-nf-call-ip6tables is disabled
82 + /usr/local/bin/docker build --rm=true -f Dockerfile -t 1bcdde65b9186304cbfc01fc422370037dac4921 . --pull=true --label org.label-schema.schema-version=1.0 --label org.label-schema.build-date=2020-12-03T21:42:08Z --label org.label-schema.vcs-ref=1bcdde65b9186304cbfc01fc422370037dac4921 --label org.label-schema.vcs-url=http://gitea.relativity/paradigm/DroneTest.git
83 Sending build context to Docker daemon 10.73MB
Googling around for the “inappropriate ioctl” issue, I find that it is commonly caused by having a mismatch of btrfs-progs versions. I’m wondering if this is a privilege issue or a mismatch between what is in a container (either drone/agent or plugins/docker) and what’s on the synology DSM (which is generally older).
For the privilege issue I have tried adding the privileged flag:
This entry in your logs indicates a problem with login. The most common root cause is the username and password are not being passed to the plugin correctly. Another common root cause is providing username and password, but failing to configure the registry setting in the yaml which lets Docker know which registry it is logging into.
You can see this guide for more details, including some common issues, and an example for using a custom registry with the plugin:
Since my registry is my own, I didn’t setup any authentication for it, so I didn’t think I needed that part. I fixed it by adding authentication, and testing it out. I did have to add the insecure: true to get it working since I didn’t setup SSL. Unfortunately I still get the same “inappropriate ioctl” bit at the end:
btrfs-progs in a container that has a version mismatch from what’s on the host
permissions that need to be escalated somewhere, although I’m now running the drone/agent as a privileged container, and I added privileged: true for the plugins/docker part of the config file:
Can confirm that this is a Synology problem Threw docker on another system and got the agent up and running in a few minutes. Works like a charm now.
Just to leave some key words in case some body else trips across this, the Synology DS918+ (among other models) running DSM 6 has older docker and btrfs-progs which throw the inappropriate ioctl when you try to run the plugins/docker drone plugin. The (unfortunate) easiest solution is to run the drone/agent on another docker box. The drone server works fine on the synology box, only the drone/agent can not be there.
There is a better option than running drone agents on another box. You have to instruct the drone docker plugin to use vfs as storage_driver. Vfs storage driver does not need to use the outdated btrfs kernel module of the Synology DSM.
To do so just add under steps-settings the following line to the .drone.yml file in your repository:
storage_driver: vfs
As per docker docs:
“The VFS storage driver is not a union filesystem; instead, each layer is a directory on disk, and there is no copy-on-write support. To create a new layer, a “deep copy” is done of the previous layer. This leads to lower performance and more space used on disk than other storage drivers. However, it is robust, stable, and works in every environment. It can also be used as a mechanism to verify other storage back-ends against, in a testing environment.”
So, even though its slower, it is robust and does the job.