Using AWS Secrets Manager to Securely Store and Retrieve App Secrets in Node.js.

Almost every application we build uses some internal ‘secrets’. Examples of such secrets would be the private key against which JWT’s are generated for authentication or an API key used to make an API call to another service. Typically, we store these secrets as environment variables. In most Node.js apps, we store the secrets in a .env file and use the dotenv npm package to retrieve them.

The Problem

Today, most applications are deployed to production using CI/CD tools. All we do is commit our changes to a git repository (most commonly GitHub) and then our CI/CD workflow kicks in and handles the rest. For deployment to an AWS EC2 instance, we can configure a similar setup using tools like CodePipeline and CodeDeploy provided by AWS. However, a common problem that arises in setting up such a workflow is configuring the ‘secrets’ or the environment variables. As we know, it is a security best practice to not commit the .env file to our git repository (especially if it’s public!) as it would expose those secrets to unwanted seekers. Setting the secrets manually is a big NO in the workflow of a CI/CD pipeline as it completely ruins the whole point of CI/CD automation. Thus, our target is to develop an efficient technique to retrieve these secrets in a production environment running on an AWS EC2 instance.

The Solution

AWS Secrets Manager is a fully managed service offered by Amazon Web Services that helps you store, manage, retrieve and rotate your application secrets in a secure and efficient manner.

For the use case and it’s problem described above, Secrets Manager is the most efficient solution.

Let’s explore AWS Secrets Manager via the AWS console. If you do not have an AWS account, create one here.

Login to your dashboard and search for ‘Secrets Manager’ in the search bar at the top of the page. On clicking ‘Secrets Manager’ , you get redirected to the Secrets Manager Dashboard and click on ‘Store a new secret’.

Click on the highlighted button

You are now redirected to a page that asks you what kind of secret you want to store. For our purpose, we shall select ‘other type of secrets’, and fill in two secrets as sample data.

Creating new secret

Click on next. In the next page you are required to give a secret name. Other fields are optional. We shall leave them blank for now.

Click on next. The next page gives you the option to configure rotation (ie, changing values periodically). You can write a lambda function to configure rotation of your secrets. However for our demo app, we do not need rotation of secrets. So we’ll select disable rotation and proceed by clicking on next.

Rotation Configuration

In the next page, we get to review all the configuration we’ve done so far and there are also code samples in various languages to interact with secrets manager at the bottom of the page. Click on ‘Store’ at the bottom.

Your secret is now created. You can view the list of your secrets here.

Now that we have configured Secrets Manager to store our secrets, let’s go ahead with the process of retrieving them in a Node.js app running on an EC2 instance.

[NB: This article assumes the reader’s familiarity with the process of deploying Node.js apps to EC2 instances using CodePipeline and CodeDeploy. ]

Retrieving the secrets programmatically

Prerequisites:

1. Create an EC2 instance (preferably running Ubuntu) and install Node.js.
2. Attach suitable IAM role to the EC2 instance to enable access to Secrets Manager.

Once the above prerequisites are met, we can start building our Node.js app.

In your project directory, setup a basic node.js project using the command

npm init -y

We need the following external modules :

• express : to manage api routes
• cors : to allow cross origin requests
• dotenv : to interact with .env file
• aws-sdk : to interact with AWS Secrets Manager

Install the dependencies : npm install express cors dotenv aws-sdk --save

Now create a file called retrieveSecrets.js and add the following code :

In the above code, we setup the aws-sdk at first. For that, we create an instance of SecretsManagerand store it in client [Note : if you try to execute this from a platform other than EC2 instance, you would need to set credentials as well. Learn more here. However, as we shall be deploying this app to an EC2 instance with relevant IAM roles attached, we need not do that].

Next, we return a Promise. Inside the promise, we call the getSecretValue method on the client object by passing the secret we want to retrieve in the SecretId variable. In the callback, we check for errors and reject the promise, if any. Else, we work on the returned data object. This object has a property called SecretString which contains a stringified json format of the stored secrets. We parse that in json and store it in secretsJSON .

Finally, we iterate over the secretsJSON object and create a string suitable for writing to a .env file and resolve the promise with that secretsString .

Now create the app.js file which shall be the entry point to our app:

In the above code ,we first require the dependencies and modules and setup a basic express app. Next, we configure a route to serve the value of the retrieved secrets.

Finally, we start the server on port 4000 and in the callback we perform three operations :

1. Retrieve the secretsString returned by the retrieveSecrets function.
2. Write this string to a .env file at the root of the project.
3. Configure the dotenv module to successfully retrieve the secrets from .env file.

That’s it! We’re all set to deploy this app to a suitable EC2 instance. [Note that we need to expose the port 4000 of the instance by using security groups or configure nginx to handle requests].

Upon successful deployment, visit the url : http://<your-instance-ip>:4000

This is the response :

Response with the secrets

Voila! Our secrets are successfully retrieved! This would also mean a .env file has been successfully created at the root of our project directory. To verify this, ssh into the instance and cd into the project directory. Execute this command : cat .env

You should get an output like this :

Showing the contents of .env file on EC2 server

That’s it! We’ve successfully retrieved secrets from secrets manager and stored it in a .env file on the server!

Find the code for this demo here.

Thanks for reading through.

Hope you found this useful 😊.