Would be good to have functionality similar to ‘workspace’ on a docker pipeline where you can set the working directory for all subsequent git operations and commands.
You can customize the workspace path with the following syntax:
workspace:
path: /foo/bar
The workspace path is appended to the pipeline root directory. If you customized the workspace path using the above configuration, the workspace path would be the following:
/tmp/drone-${RANDOM}/foo/bar
Ok so there is no current way to define an absolute path as a working dir/workspace?
Everything always ends in the tmp folder?
Ok so there is no current way to define an absolute path as a working dir/workspace?
Everything always ends in the tmp folder?
Correct, we do this to prevent race conditions when two pipelines execute on the same machine at the same time. It also prevents using the same home directory for two different projects, which may have different clone credentials that would otherwise sourced from the same file path.
You could always fork the ssh runner and change the behavior to your liking, with the caveat that you may run into race conditions or unexpected behavior as a result.
Why not limit concurrent executions to 1 by default to get around the race condition?
This is the standard behavior in Jenkins and if you do more then 1 build concurrently it will immediately fail when there is a file lock it runs into.
Additionally /tmp is not always guaranteed to have enough space for large projects.
e.g the OS disk for Azure VMs is 30GB by default which does not leave alot of room to play with on the /tmp.
Why not limit concurrent executions to 1 by default to get around the race condition?
This runner was created with concurrent use as one of the design goals. Treating pipelines as ephemeral with no state left behind was also a design goal (within reason, keeping in mind there is no isolation or way to fully enforce this). Different teams may have different requirements, however, which is why anyone can create their own runner with behavior that is optimized for their use cases.
Setting aside whether or not designing this runner to support concurrency was the right or wrong decision, we do not want to introduce fundamental breaking changes given the large number of active Drone installations.
Additionally /tmp is not always guaranteed to have enough space for large projects.
You can change the temporary directory location using the TMPDIR environment variable. For example, TMPDIR=/workspace would create the pipeline root directory at /workspace/drone-${RANDOM}
What is the suggested way to get the code that is pulled/cloned into this TMPDIR location into the real working directory?
In our case we have all our nodejs code running within a directory in the $HOME folder and the pm2 process manager is running this code which has it’s own config (pm2.yaml) which uses absolute path names to point to the files which it runs.
Currently I am getting around this issue using the following commands in drone.yml
commands:
- npm install
- rsync --archive --delete --exclude=config.json . /home/user/some_folder/
- redis-cli flushdb
- node --max-old-space-size=8192 /home/user/some_folder/jobs/sqlrunner.js run
- pm2 reload all
Is there a better way to do this without using that rsync command?
Also since the working folder is ephemeral that means you must always do a clone operation first?
Or is there a way to tell it to just do a pull on an existing folder?
We have one repo in particular that is huge in size as it contains many SQL files and having to clone it everytime will slow things down significantly.