Managing server secrets without exposing them
Field: server security, secrets management, passwords
For: SME owners whose tools rely on API keys and passwords, and their IT person
Experience: how I store the secrets of my own servers, after a complete overhaul of my method
A well-locked server can still lose everything through a single file. API keys, database passwords and access tokens often sit in plain text on the disk, in configuration files that get copied, backed up and shared without a second thought. Here is how I separate the secrets of people from those of machines, how the latter stay encrypted until the moment a service needs them, and what to prepare for the day one of them leaks.
Background
Every modern service relies on secrets: the key that bills your calls to an AI model, the ERP database password, the token that lets you send emails or post on a social network. An SME server quickly piles up several dozen of them. Securing the server keeps intruders out; but if someone does get in, or a copy escapes, they must not find every key to the house.
The problem: plain-text secrets everywhere
A secret readable in plain text leaks through paths nobody watches:
- The server copy: a snapshot taken by the hosting provider, a backup or a disk image carries the configuration files as they are.
- The code repository: a configuration file pushed by mistake to a shared repository publishes the keys with it.
- The compromised service: a hacked program reads everything its account is allowed to read, including other services' keys.
- The work message: a key pasted into a chat, a ticket or a conversation with an AI assistant stays in a history that others can read.
- The eternal key: created three years ago, never changed, it still opens everything, and nobody remembers where it was copied.
The solution: two worlds, five rules
1. A vault for people
The credentials people use (web accounts, banking access, cards) go into a password manager. I use Vaultwarden, a server compatible with the Bitwarden apps, hosted on my own server and reachable only through the VPN. Sign-ups are closed as soon as the first account is created. The server's automated processes have no access to it at all.
2. Encrypted files for machines
Service secrets are stored in a file encrypted with sops and age. Only the values are encrypted: the names stay readable, so you can see which keys exist without being able to read them. This encrypted file can be backed up and versioned like any other file, because it reveals nothing without the decryption key.
3. Decrypt in memory, at startup
When the server starts, a small service decrypts the secrets into an in-memory space, then launches the applications that need them. Plain-text secrets never touch the disk, and they vanish when the machine shuts down. A stolen disk or a snapshot only yields encrypted files.
4. Two keys, and a copy on paper
One key pair for the server, another for the administrator, and every encrypted file for both. The server can encrypt a secret for the human, but cannot read what does not concern it. The administrator's key is also kept on paper, somewhere safe away from the office: if these keys are lost, every secret is lost with them.
5. Plan the rotation before the leak
A key always ends up leaking or having to change. The procedure is prepared in advance: where the key is used, how to create a new one, how to replace it everywhere, and how to revoke the old one. The day a secret shows up somewhere it should not be, all that is left is to follow that sheet.
Questions to ask your IT person
- If a copy of our server were stolen tomorrow, which passwords and keys would be found in plain text?
- Who can read our services' keys, and can the program that handles email read the accounting system's keys?
- How long does it take to change a leaked key, everywhere it is used?
- Where is the key that opens everything else, and who can use it if you are away?
What I learned running it
- Environment variables are not a vault. A secret passed to a container remains visible to anyone who can inspect that container. Protection comes from what surrounds the secret (who has access to what), not from where you put it.
- Not everything can be encrypted. Some tokens are renewed automatically by the software that uses them, and must stay where it writes them. I listed them and protected them with access rights rather than forcing their migration.
- AI assistants write everything down. One day, a technical report containing the location of a key ended up in the history of another work session. Since then, the rule is strict: anything touching secrets stays in a dedicated private space, never in a conversation or a message between tools.
- A regular check beats a good intention. After an update, a plain-text secret can reappear in a configuration file. An automatic audit that looks for plain-text values flags it before it travels in a backup.
Under the hood (for your IT person)
- Human vault: Vaultwarden behind the reverse proxy, reachable only through the VPN, sign-ups disabled, admin interface protected by a token.
- Machine encryption: sops encrypts values with AES256_GCM using a data key, itself encrypted for each age recipient (service key and administrator key).
- Decryption: a oneshot systemd service decrypts to a user tmpfs; applications and containers start after it and read an environment file held in memory.
- Rights: the service account has neither sudo nor access to the human vault; membership of the docker group is close to root rights, which is handled separately.
- Rotation: one sheet per secret (uses, creation, replacement, revocation), re-encryption of the file, controlled restart, then a check that no plain-text value remains.
What I take away
A well-kept secret is not a hidden secret, it is a secret that is useless to whoever steals it: encrypted on disk, readable only by the service that needs it, and easy to replace. It is the same logic as guardrails for AI agents: an agent with no access to a key cannot leak it. And it is what makes backups safe: a copy of your encrypted files gives away no password.
Going further
I can take stock with you of your servers' secrets, find the ones lying around in plain text, and set up a vault and a rotation procedure. See Your tools and your data, or book a 20-minute discovery call.
The step-by-step guide
For your IT person, guide 3 in the Open Source Server Stack series, secrets and vault (CHF 39), sets all of this up in 2 to 3 hours: Vaultwarden behind the VPN, sops and age with two key pairs, in-memory decryption at startup, key backup, a rotation procedure and a final checklist. At the end, not a single secret is left in plain text on the server.
The Guides Pack (CHF 149) brings together guides 0 to 8 of the series: a complete stack, documented and tested in production, from the secure server to backups and monitoring. If you would rather have support, the assisted installation (CHF 597) installs the foundation with you, secrets included, in one or two video sessions, Complete Pack included, with 60 days of email support.
Sources
- OWASP, Secrets Management Cheat Sheet, accessed 15 November 2026.
- SOPS, Security (encryption model), official documentation, accessed 15 November 2026.
- age, official repository, accessed 15 November 2026.
- Vaultwarden, official repository, accessed 15 November 2026.
- Photo: Fixedsun / Wikimedia Commons (commons.wikimedia.org), licence CC0.
Tags: secrets management, passwords, Vaultwarden, sops, age, encryption, rotation, server security, SME