Building Functional Web Apps with MongoDB

In this tutorial, we’ll use Begin to quickly develop a to-do list application. We’ll use AWS Lambda as our serverless platform and MongoDB Atlas as our database provider.

Let’s get started.

Prerequisites

You will need to have git and Node.js installed to your local computer to follow along with this tutorial. (Learn more about installing git and installing Node.js.)

You’ll also need a GitHub account. (Learn more about signing up with GitHub.)

This tutorial also assumes some familiarity with such things as:

• Text editors
• Terminal / CLI
• Git and version control
• General software development using JavaScript

You do not need to be an expert in any of these things to follow along though.

Getting started

First, click the Deploy to Begin button below. This starts the process of authorizing Begin with your GitHub account. (You may be prompted to log in to GitHub, and/or be asked to create a Begin username.)

Name your app & repo

You’ll then be prompted to name your new app and repository – this is optional. Feel free to use the default app and repo name if you like!

Note: your Begin app name and repository name cannot be changed later.

Once you’ve clicked the Create... button, Begin will spin up your new project on GitHub (under github.com/{your GH username}/{your repo name}).

By default your Begin app repo is created as a public GitHub repo; it can be set to private by granting Begin additional permissions from this screen (or later from the Settings screen found in the left nav of your Begin app).

Your first deploy

After creating your app, you’ll be taken to its Activity stream. Welcome to the main backend interface of your Begin app!

From the Activity view, you’ll be able to watch your app build & deploy in real-time. Any time you push to main, you’ll see a new build get kicked off in Begin.

Each build undergoes several predefined build steps (learn more about build steps here); these build steps may install your app’s dependencies (install), test your code’s syntax (lint), generate any files or assets needed to run your app (build), and/or run an automated test suite (test).

If no build steps fail, then the build containing your latest commit to main is automatically deployed to your staging environment.

Go ahead and click the Staging link in the upper left corner of your left nav to open your new app’s staging URL. You should now see your new app:

Get set up locally

Next, let’s get your new site running in your local environment (i.e. the computer you work on).

First, head to your GitHub repo (from the first card in your Activity, or the left nav). Find the clone or download button and copy the git URL.

Then head to your terminal and clone your repo to your local filesystem.

git clone https://github.com/your-github-username/your-new-begin-app.git

Once you’ve got your project cloned on your local machine, cd into the project directory and install your dependencies:

cd your-new-begin-app
npm install

Now you are all set to work on your app locally!

Project structure

Now that your app is live on staging and running locally, let’s take a quick look into how the project itself is structured so you’ll know your way around. Here are the key folders and files in the source tree of your new app:

.
├── public/
│   ├── index.html
│   └── index.js
├── src/
│   ├── http/
│   │    ├── get-todos/
│   │    ├── post-todos/
│   │    ├── post-todos-000id/
│   │    └── post-todos-delete/
│   └── shared/
│        └── mongodb-client.js
└── app.arc

public/index.html & public/index.js

public/index.html is the page served to the browser. This is where our JSON data will be appended to a DOM element of our choosing. public/index.js is where we will write our function that fetches the JSON data from get /todos and displays it in our HTML page.

Your app utilizes built-in small, fast, individually executing cloud functions that handle HTTP API requests and responses. (We call those HTTP functions, for short.)

The HTTP function that handles requests:

• get /todos is found in src/http/get-todos/
• post /todos is found in src/http/post-todos/
• post /todos/:id is found in src/http/post-todos-000id/
• post /todos/delete is found in src/http/post-todos/delete.

In the next section, we will go more in-depth about how to manipulate data stored in MongoDB from an HTTP Function.

src/shared/mongodb-client.js

The mongodb-client.js file is where we will write common code that will be used in all of our HTTP functions. This code will automatically get copied into each HTTP function during the hydration phase of the install build step.

app.arc

Your app.arc file is where you will provision new routes and functions.

Infrastructure-as-code is the practice of provisioning and maintaining cloud infrastructure using a declarative manifest file. It’s like package.json, except for cloud resources like API Gateway, Lambda, and DynamoDB (all of which Begin apps use).

By checking in your Begin app’s project manifest (app.arc) file with your code, you can ensure you have exactly the cloud resources your code depends on. This is crucial for ensuring reproducibility and improving iteration speed.

Access MongoDB from HTTP Functions

Let’s dig into how we connect to MongoDB Atlas and manipulate data via HTTP Functions. Open src/http/get-todos/index.js:

In the first four lines of the function:

• We require the http middleware from @architect/functions.
• Include some helper functions from our mongodb-client.js package which are shared with all of our HTTP functions
• Setup our handler to call the async functions clientContext and read in that order.

// src/http/get-todos/index.js
const { http } = require('@architect/functions')
const { clientConnect, clientClose, clientContext } = require('@architect/shared/mongodb-client')

exports.handler = http.async(clientContext, read)

Over in src/shared/mogodb-client.js we will find the clientContext function.

// src/shared/mongodb-client.js
async function clientContext(req, context) {
 process.env.ARC_ENV === 'testing' ?
   context.callbackWaitsForEmptyEventLoop = true :
   context.callbackWaitsForEmptyEventLoop = false
}

This function may look weird, and that’s because it is. When we are running locally the architect sandbox spins up a new Node.js process to handle each incoming request. Once the request is fulfilled the process exits. When the function runs as an AWS Lambda we will have the option of keeping the connection to MongoDB open to serve multiple requests.

See Best Practices Connecting from AWS Lambda

Now let’s look at the read function from src/http/get-todos/index.js. The first line in the function is where we await the clientConnect helper function.

// src/http/get-todos/index.js
const client = await clientConnect;

Because this is a promise, it will only resolve once. This means that only the first call to this function will incur the penalty of setting up a connection with the database. If the HTTP function is already warm the promise will return the already connected client. If you find the coldstart time of your function to be too long while the database connection is set up then provisioned concurrency could help.

// src/shared/mongodb-client.js
const { MongoClient } = require('mongodb')

const client = new MongoClient(process.env.MONGODB_URL,
 { useNewUrlParser: true,  useUnifiedTopology: true })

module.exports = { clientConnect: client.connect(),
  clientClose, clientContext }

Next we will fetch out todos from our collection by:

// src/http/get-todos/index.js
const db = client.db('todo-app')
const collection = db.collection('todos')
let todos = await collection.find({}).toArray();

• We’ll specify which database we want to use.
• Tell the database which collection we want to load.
• And finally, load every todo stored in our collection by finding all the objects and having them returned to us as an array.

The next line in src/http/get-todos.js is:

// src/http/get-todos/index.js
await clientClose()

When running on AWS this function is a no-op but when running locally it closes the connection to MongoDB. Once again see Best Practices Connecting from AWS Lambda

// src/shared/mongodb-client.js
async function clientClose() {
 if (process.env.ARC_ENV === 'testing') {
   await client.close()
 }
}

Finally we’ll return the todos as the HTTP functions response

// src/http/get-todos/index.js
return {
  statusCode: 200,
  cacheControl: 'no-cache, no-store, must-revalidate, max-age=0, s-maxage=0',
  json: todos
}

Since we are using the arc.http.async helper function we can use some shortcuts when we return a payload from the function.

• statusCode sets the response to 200 OK
• cacheControl sets the cache-control header
• json set the content-type header to application/json; charset=utf8 while also JSON encoding the todos array for us.

Full code listings for:

• src/http/get-todos/index.js
• src/shared/mongodb-client.js

Getting Up and Running with MongoDB Atlas

Let’s start by setting up our MongoDB Atlas account. If you don’t already have one, sign up for one here.

MongoDB Atlas can be deployed in minutes and used for FREE with an M0 sized cluster.

When you are signed up and logged into the MongoDB Atlas dashboard, the first thing we’ll do is set up a new cluster. Click the Build a Cluster button to get started.

You’ll see that MongoDB Atlas offers three deployment options:

• Serverless: For applications with variable or infrequent traffic
• Dedicated: For applications with sophisticated workload requirements
• Shared: For exploring MongoDB with basic configuration controls

From here, select the Shared Clusters option, which will have the free tier we want to use.

Finally, for the last selection, you can leave all the defaults as is and just hit the green Create Cluster button at the bottom. Depending on your location, you may want to choose a different region, but I’ll leave everything as is for the tutorial. The cluster build-out will take about a minute to deploy.

While we wait for the cluster to be deployed, let’s navigate to the Database Access tab in the menu and create a new database user. We’ll need a database user to be able to connect to our MongoDB database. On the Database Access page, click on the Add New Database User button.

Give your user a unique username and password. Be sure to write these down as you’ll need them soon. Make sure that this database user can read and write to any database by checking the Database User Privileges dropdown. It should be selected by default, but if it’s not, set it to read and write to any database.

Next, we’ll also want to configure network access by navigating to the Network Access tab in the dashboard. For the sake of this tutorial, we’ll enable access to our database from any IP as long as the connection has the correct username and password. In a real-world scenario, you’ll want to limit database access to specific IPs that your application lives on, but configuring that is out of scope for this tutorial.

Click on the green Add IP Address button, then in the modal that pops up click on Allow Access From Anywhere. Click the green Confirm button to save the change.

By now our cluster should be deployed. Let’s hit the Clusters selection in the menu and we should see our new cluster created and ready to go. It will look like this:

One final thing we’ll need to do is add a database to save our todos. To do this, click on the Databases tab in the menu and then the Browse Collections button.

Click on the Add My Own Data button.

For the Database name use “todo-app” and for Collection name use “todos” then click the Create button.

Congratulations, you’ve now set up a database where we can store our todos.

Before we leave the Atlas console let’s grab our MongoDB Atlas connection string. To get this value, click on the Connect button on the Clusters overview page.

From here, select the Connect your application option:

and you’ll be taken to a screen that has your connection string.

Note: Your username will be prepopulated, but you’ll have to update the password value. Copy this connection string as we will be using it in the next section.

Developing Locally

One of the main advantages of using Begin is the ability to develop your application business logic locally.

At the root of your project create a file called .env. This is where we will store the connection string we generated in the previous step.

# .env

@testing

MONGODB_URL=mongodb+srv://<username>:<password>@<hostname>.mongodb.net/<cluster>?retryWrites=true&w=majority

Now, in the same folder that you ran npm install previously run:

npm start

Navigate to localhost:3333/ and you will see a copy of your application running locally. Experiment with adding, editing and deleting some todo list items then come back here, we’ll wait.

Deploy Your Site

Head back over to Begin and open the environments panel. It is here that we will need to set our MONGODB_URL environment variable. Scroll down to the Staging section and add a new environment variable called MONGODB_URL and give it the value of your MongoDB Connection string you used earlier. Then repeat the same process for the Production section.

Now in the top left-hand corner click on the Staging link. Try adding, editing and deleting todos. Notice that it is even faster than the local development as our HTTP functions are able to re-use the database connection?

Once you are satisfied that everything is working it’s time to deploy to production. Click the Deploy to production button in the upper left, pick a version, leave an optional message summarizing your changes, and Ship it!

When your next build is done, click the production link in the upper left corner to see the latest release of your app.

Congratulations!

You now have a good idea how HTTP functions work on Begin and how to use MongoDB as a backend data store.