Berglas

Berglas is a command line tool and library for storing and retrieving
secrets on Google Cloud. Secrets are encrypted with Cloud KMS and
stored in Cloud Storage. An interoperable layer also exists with Secret Manager.

• As a CLI, berglas automates the process of encrypting, decrypting, and
  storing data on Google Cloud.

• As a library, berglas automates the inclusion of secrets into various
  Google Cloud runtimes

Berglas is not an officially supported Google product.

Setup

1. Install the Cloud SDK for your operating system. Alternatively,
   you can run these commands from Cloud Shell, which has the SDK
   and other popular tools pre-installed.

   If you are running from your local machine, you also need Default
   Application Credentials:

   gcloud auth application-default login
   

   This will open a web browser and prompt for a login to your Google account.
   On headless devices, you will need to create a service account. For more
   information, please see the authentication section.

2. Install the berglas CLI using one of the following methods:

   • Install a pre-compiled binary for your operating system:

     • darwin/amd64
     • linux/amd64
     • windows/amd64

     This will download the latest version built against the master branch. To
     download a specific version, specify a git tag in place of "master" in the
     URL.

     Depending on your operating system, you may need to mark the downloaded binary as executable:

     chmod +x /path/to/berglas
     

   • If you use Homebrew on macOS, you can install like this:

     brew install berglas
     

   • Use the official Docker container:

     docker pull gcr.io/berglas/berglas:latest
     

     This will pull the latest version built against the master branch. To use
     a specific version, specify a git tag in place of "latest" in the URL.

   • Install from source (requires a working Go installation):

     go get github.com/GoogleCloudPlatform/berglas/...
     go install github.com/GoogleCloudPlatform/berglas
     

3. Export your project ID as an environment variable. The rest of this setup
   guide assumes this environment variable is set:

   export PROJECT_ID=my-gcp-project-id
   

   Please note, this is the project ID, not the project name or project
   number. You can find the project ID by running gcloud projects list or
   in the web UI.

4. Export your desired Cloud Storage bucket name. The rest of this setup guide
   assumes this environment variable is set:

   export BUCKET_ID=my-secrets
   

   Replace my-secrets with the name of your bucket. Set only the name,
   without the gs:// prefix. This bucket should not exist yet!

5. Enable required services on the project:

   gcloud services enable --project ${PROJECT_ID} \
     cloudkms.googleapis.com \
     storage-api.googleapis.com \
     storage-component.googleapis.com
   

6. Bootstrap a Berglas environment. This will create a new Cloud Storage bucket
   for storing secrets and a Cloud KMS key for encrypting data.

   berglas bootstrap --project $PROJECT_ID --bucket $BUCKET_ID
   

   This command uses the default values. You can customize the storage bucket
   and KMS key configuration using the optional flags. Run berglas bootstrap -h for more details.

   If you want full control over the creation of the Cloud Storage and Cloud
   KMS keys, please see the custom setup documentation.

7. (Optional) Bootstrap a Berglas environment specifying a bucket location. By
   default the berglas bucket is created in the multi-regional location US.
   You can specify your location by using the following command. Please see the
   list of supported locations in the GCP bucket location documentation
   page

   export BUCKET_LOCATION=europe-west1
   berglas bootstrap \
     --project $PROJECT_ID \
     --bucket $BUCKET_ID \
     --bucket-location $BUCKET_LOCATION
   

   This command uses the default values. You can customize the storage bucket
   and KMS key configuration using the optional flags. Run berglas bootstrap -h for more details.

   If you want full control over the creation of the Cloud Storage and Cloud
   KMS keys, please see the custom setup documentation.

8. (Optional) Enable Cloud Audit logging on the bucket:

   Please note this will enable audit logging on all Cloud KMS keys and all
   Cloud Storage buckets in the project, which may incur additional costs.

   1. Download the exiting project IAM policy:

      gcloud projects get-iam-policy ${PROJECT_ID} > policy.yaml
      

   2. Add Cloud Audit logging for Cloud KMS and Cloud Storage:

      cat <<EOF >> policy.yaml
      auditConfigs:
      - auditLogConfigs:
        - logType: DATA_READ
        - logType: ADMIN_READ
        - logType: DATA_WRITE
        service: cloudkms.googleapis.com
      - auditLogConfigs:
        - logType: ADMIN_READ
        - logType: DATA_READ
        - logType: DATA_WRITE
        service: storage.googleapis.com
      EOF
      

   3. Submit the new policy:

      gcloud projects set-iam-policy ${PROJECT_ID} policy.yaml
      

   4. Remove the updated policy from local disk:

      rm policy.yaml
      

CLI Usage

1. Create a secret:

   berglas create ${BUCKET_ID}/foo my-secret-data \
     --key projects/${PROJECT_ID}/locations/global/keyRings/berglas/cryptoKeys/berglas-key
   Successfully created secret: foo
   

2. Grant access to a secret:

   berglas grant ${BUCKET_ID}/foo --member user:user@mydomain.com
   

3. Access a secret's data:

   berglas access ${BUCKET_ID}/foo
   my-secret-data
   

4. Spawn a child process with secrets populated in the child's environment:

   berglas exec -- myapp --flag-a --flag-b
   

   This will spawn myapp with an environment parsed by berglas.

5. Access data from a specific generation of a secret:

   berglas access ${BUCKET_ID}/foo --generation 1563925940580201
   my-previous-secret-data
   

6. Revoke access to a secret:

   berglas revoke ${BUCKET_ID}/foo --member user:user@mydomain.com
   

7. Delete a secret:

   berglas delete ${BUCKET_ID}/foo
   Successfully deleted secret if it existed: foo
   

In addition to standard Unix exit codes, if the CLI exits with a known error,
Berglas will exit with one of the following:

• 60 - API error. Berglas got a bad response when communicating with an
  upstream API.

• 61 - Misuse error. You gave unexpected input or behavior. Please read the
  error message. Open an issue if you think this is a mistake.

The only exception is berglas exec, which will exit with the exit status of
its child command, if one was provided.

Integrations

• App Engine (Flex) - When invoked via App Engine Flex,
  Berglas resolves environment variables to their plaintext values using the
  `berglas://reference syntax. This integration works with
  any language runtime because berglas serves as the entrypoint to the Docker
  container. See examples/appengineflex for examples
  and invocations.

• App Engine (Standard) - When invoked via App Engine,
  Berglas resolves environment variables to their plaintext values using the
  berglas://reference syntax. This integration only works
  with the Go language runtime because it requires importing the auto/
  package. See examples/appengine for examples
  and invocations.

• Cloud Run - When invoked via Cloud Run, Berglas resolves
  environment variables to their plaintext values using the berglas://
  reference syntax. This integration works with any language
  runtime because berglas serves as the entrypoint to the Docker container. See
  examples/cloudrun for examples and invocations.

• Cloud Functions - When invoked via Cloud Functions,
  Berglas resolves environment variables to their plaintext values using the
  berglas:// reference syntax. This integration only works
  with the Go language runtime because it requires importing the auto/
  package. See examples/cloudfunctions for examples
  and invocations.

• Cloud Build - When invoked via Cloud Build, Berglas
  resolves environment variables to plaintext values using the berglas://
  reference syntax. This integration only works with volume
  mounts, so all Berglas secrets need to specify the ?destination parameter.
  See examples/cloudbuild for examples and invocations.

• Kubernetes - Kubernetes pods can consume Berglas secrets by installing a
  MutatingWebhook. This webhook mutates incoming pods with the
  berglas:// reference syntax in environment references to
  resolve at runtime. This integration works with any container, but all pods
  requesting berglas secrets must set an command in their Kubernetes manifests.
  See examples/kubernetes for samples and installation
  instructions.

• Anything - Wrap any process with berglas exec -- and Berglas will
  parse any local environment variables with the berglas:// reference
  syntax and spawn your app as a subprocess with the
  plaintext environment replaced.

Logging

Both the berglas CLI and berglas library support debug-style logging. This logging is off by default because it adds additional overhead and logs information that may be security-sensitive.

The default logging behavior for the berglas CLI is "text" (it can be changed
with the --log-format flag). The default logging behavior for the berglas
library is structured JSON which integrates well with Stackdriver (it can be
changed to any valid formatter and you can even inject your own logger).

Examples

Examples are available in the examples/ folder.

Library Usage

Berglas is also a Go library that can be imported in Go projects:

import (
	_ "github.com/GoogleCloudPlatform/berglas/pkg/auto"
)

When imported, the berglas package will:

1. Download and decrypt any secrets that match the Berglas environment
   variable reference syntax in the environment

2. Replace the value for the environment variable with the decrypted secret

You can also opt out of auto-parsing and call the library yourself instead:

import (
	"context"
	"log"
	"os"

	"github.com/GoogleCloudPlatform/berglas/pkg/berglas"
)

func main() {
	ctx := context.Background()

	// This higher-level API parses the secret reference at the specified
	// environment variable, downloads and decrypts the secret, and replaces the
	// contents of the given environment variable with the secret result.
	if err := berglas.Replace(ctx, "MY_SECRET"); err != nil {
		log.Fatal(err)
	}

	// This lower-level API parses the secret reference, downloads and decrypts
	// the secret, and returns the result. This is useful if you need to mutate
	// the result.
	if v := os.Getenv("MY_SECRET"); v != "" {
		plaintext, err := berglas.Resolve(ctx, v)
		if err != nil {
			log.Fatal(err)
		}
		os.Unsetenv("MY_SECRET")
		os.Setenv("MY_OTHER_SECRET", string(plaintext))
	}
}

For more examples and documentation, please see the godoc.

Authentication

By default, Berglas uses Google Cloud Default Application Credentials. If you
have gcloud installed locally, ensure you have application default
credentials:

gcloud auth application-default login

On GCP services (like Cloud Build, Compute, etc), it will use the service
account attached to the resource.

To use a specific service account, set the GOOGLE_APPLICATION_CREDENTIALS
environment variable to the filepath to the JSON file where your credentials
reside on disk:

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/my/credentials.json

To learn more, please see the Google Cloud Service Account
documentation.

Authorization

To control who or what has access to a secret, use berglas grant and berglas revoke or the associated API methods. These methods use Cloud IAM
internally. The following information is relevant only if you plan to grant IAM
permissions manually.

Most operations require access to the Cloud KMS key and the Cloud Storage
bucket. You can read more about Cloud KMS IAM and Cloud
Storage IAM in the documentation.

Any service account or entity using Berglas will need to authorize using the
cloud-platform scope.

Create

To create a secret, the following role is required on the Cloud Storage bucket:

roles/storage.objectCreator

To create a secret, the following role is required for the Cloud KMS key:

roles/cloudkms.cryptoKeyEncrypter

Access

To access a secret, the following role is required on the Cloud Storage bucket:

roles/storage.objectViewer

To access a secret, the following role is required for the Cloud KMS key:

roles/cloudkms.cryptoKeyDecrypter

Delete

To delete a secret, the following role is required on the Cloud Storage bucket:

roles/storage.objectAdmin

To delete a secret, no permissions are needed on the Cloud KMS key.

Implementation

This section describes the implementation. This knowledge is not required to use
Berglas, but it is included for security-conscious/curious users who want to
learn about how Berglas works internally to build a threat model.

When encrypting a secret:

1. Berglas generates an AES-256-GCM data encryption key (DEK) using Go's crypto
   package for each secret. (N.B. each secret has its own, unique DEK).

2. Berglas encrypts the plaintext data using the locally-generated DEK,
   producing encrypted ciphertext, prepended with the AES-GCM nonce.

3. Berglas encrypts the DEK using the specified Cloud KMS key, also known as a
   key encryption key (KEK). This process is called envelope
   encryption.

4. Berglas stores the Cloud KMS key name, encrypted DEK, and encrypted ciphertext
   as a single blob in Cloud Storage.

When decrypting a secret:

1. Berglas downloads the blob from Cloud Storage and separates the Cloud KMS key name,
   encrypted DEK, and ciphertext out of the blob.

2. Berglas decrypts the DEK using Cloud KMS. This is part of envelope encryption.

3. Berglas decrypts the ciphertext data locally using the decrypted DEK.

Security & Threat Model

Berglas makes certain security tradeoffs in exchange for a better UX. In
particular, KMS crypto key IDs are stored on the secret object's metadata. An
attacker with permission to write objects to your Cloud Storage bucket could
overwrite existing secrets. As such, you should follow the principles of least
privilege and revoke default ACLs on the bucket as described in the setup guide.

For more information, please see the security and threat model.

FAQ

Q: Should I use Berglas or Secret Manager?

Berglas is compatible with Secret Manager and offers
convenience wrappers around managing secrets regardless of whether they reside
in Cloud Storage or Secret Manager. New projects should investigate using Secret
Manager directly as it has less operational overhead and complexity, but Berglas
will continue to support Cloud Storage + Cloud KMS secrets.

Q: Is there a size limit on the data I can encrypt?

Berglas is targeted at application secrets like certificates, passwords, and
API keys. While its possible to encrypt larger binary files like PDFs or images,
Berglas uses a a GCM cipher mode to encrypt data, meaning the data must fit in
memory and is limited to 64GiB.

Q: Why do you use envelope encryption instead of
encrypting the data directly with Cloud KMS?

Envelope encryption allows for encrypting the data at the application layer,
and it enables encryption of larger payloads, since Cloud KMS has a limit on the
size of the payload it can encrypt. By using envelope encryption, Cloud KMS
always encrypts a fixed size data (the AES-256-GCM key). This saves bandwidth
(since large payloads are encrypted locally) and increases the size of the data
which can be encrypted.

Q: Why does Berglas need permission to view my GCP resource?

Berglas communicates with the API to read the environment variables that were
set on the resource at deploy time. Otherwise, a package could inject arbitrary
environment variables in the Berglas format during application boot.

Q: I renamed a secret in Cloud Storage and now it fails to decrypt - why?

Berglas encrypts secrets with additional authenticated data including the name
of the secret. This reduces the chance an attacker can escalate privilege by
convincing someone to rename a secret so they can gain access.

Q: Why is it named Berglas?

Berglas is a famous magician who is best known for his secrets.

Contributing

Please see the contributing
guidelines.

License

This library is licensed under Apache 2.0. Full license text is available in
LICENSE.