Managing secrets for a code base is a tricky proposition. For too long, traditional DevOps approaches have seen teams build elaborate out-of-band workflows to manage system secrets, API keys and tokens, and traditional passwords. This tends to be an error-prone and painful process. This is never a good scenario to be faced with when considering options around security.
As an illustration, a quick GitRob spree on your local GitHub organisation can reveal some serious vulnerabilities that are literally just waiting there for someone to exploit. I ran this on an open source project whose name I will not disclose, and got some very interesting results that could lead to a lot of mischief.
Recently, some new software has hit the scene: BlackBox, a utility built by the smart people at StackExchange. Blackbox is a suite of scripts that aims to marry the ever-warring security and convenience into something that should satisfy both the development and security teams.
Blackbox uses GPG to encrypt files before they're sent to your DVCS (assuming GitHub for the remainder of this article), and decrypts them when needed, either by you, your team, or an automated system. Blackbox is so promising, that in fact ThoughtWorks have identified it on their tools to assess for the January 2015 Tech Radar
There really is a lot to cover in terms of workflow, but for today we'll keep it simple, and walk through the most typical steps of starting a blackboxed project, adding files, editing files, and adding collaborators.
The following instructions assume some basic groundwork has been laid down.
keybase login, or
Start up your gpg-agent so your passphrased keys are stored in memory:
$ eval $(gpg-agent --daemon)
Let Blackbox initialize some housekeeping files:
The next command is provided by blackbox, just copy and paste your version:
$ git commit -m'INITIALIZE BLACKBOX' keyrings .gitignore
Blackbox requires admins to be managed via a simple text list file, and a keyring of GPG keys. The keyring is used so that all admins can have their own keys, and still decrypt files. As we've said earlier, we're fans of Keybase, a crypto service that helps manage GPG transmissions. I'll be encrypting files using my Keybase ID.
To see a list of GPG keys on your machine, and to get the UID you need to provide blackbox with, run:
$ gpg --list-keys
Choose your key, and give the UID to blackbox:
$ blackbox_addadmin keybase.io/gee_forr
Let's get Blackbox to manage a couple of secretive files for us. To do this, we register a new file with a simple command. In this demo, I'll be encrypting a Rails app's
database.yml files. You can add as many, or as few files as you wish. Really paranoid people might want to consider even blackboxing the
$ blackbox_register_new_file config/secrets.yml $ blackbox_register_new_file config/database.yml $ blackbox_register_new_file db/schema.rb
Blackbox does a couple of clever things here. Firstly, it adds the GPG version of the file to the git repo, and secondly, it adds the unencrypted file's name to the repo's
.gitignore file so that you won't accidentally add the decrypted file back into source control.
To edit a file, you have a number of options. You can decrypt, then edit, then encrypt, or you can do that all in one command.
$ blackbox_edit config/secrets.yml.gpg
This will prompt for your passphrase, decrypt the file, and open up
$EDITOR. Writing and quitting the editing session will re-encrypt the file.
Whilst you're editing secrets.yml, why not remove the production reference to an env var, and put the secret key right in there? Don't forget to keep on committing changes as you go along.
If you want to have a file decrypted so that you can run a development environment, just run
blackbox_edit_start on the file name. This will decrypt it and do nothing else. Running
blackbox_edit_end on the file name will re-encrypt.
This is a slightly finicky process. It involves all the prerequisites outlined above, and then implements a "vouchsafe" process to allow new people onto the project. Here's another reason we love Keybase. Keybase provides a way for someone to verify another person using their social media logins. If you know someone's Twitter or GitHub handles, you can find their Keybase fingerprint from there, and rest assured that the person you are "indoctrinating" (Blackbox's term, not ours) is who they claim to be.
On your new colleagues machine, the following commands are run from within the project's repo:
$ blackbox_addadmin keybase.io/ridhwana
Here, Ridhwana adds her Keybase UID. This outputs a git command suggestion for committing the files.
Push the application for the new admin back to the central git repo:
$ git push
Once the application for admin rights is back on a central git repo, a current admin can continue the indoctrination process on their machine.
Fetch the latest code with the new proposed admin:
$ git pull
Add the new public keyring to GPG
$ gpg --import keyrings/live/pubring.gpg
Now decrypt and re-encrypt all the files using the new keyring:
And finally push all these changes back to your central git repo
$ git commit -a $ git push
From this point on, both admins should be able to work with these files securely.
Standard practice in these cases, with or without a tool like Blackbox, should be to regenerate any keys or tokens, and change any passwords. Once that's done, running the
blackbox_removeadmin command will help to remove their key from the keyring, and re-encrypt all files.
Your CI server, or orchestration servers can have a GPG key without a passphrase added. This is done by creating a sub-key from another key. These servers should be locked down extremely tight. If someone breaks into a server like this, you've already lost, and fretting over a machine-use-only key will be the least of your worries.
We hope you've enjoyed this article, and are now in a position to simplify your secrets process. If you have any questions, comments or suggestions around this topic, feel free to drop us a line in the comments below.