How-to: resize and transform images automatically without relying on 3rd party APIs

Photo by Randall Brunder on Unsplash

TL;DR

I got fed up with too many "A"s in my JAMstack and built an image processing plugin to eliminate one of them. If you are only interested in the Architect plugin for transforming images and want to skip my hot takes feel free to jump to Part 2.

Part 1: What’s my beef with 3rd party APIs?

About a year ago (before I worked for Begin), I was building a serious side project (meaning it made no money, but I had big dreams). It required managing lots of images. I used static hosting with a SPA frontend framework. I’m not naming names because I am not trying to throw shade on anyone in particular. All of these tools and techniques have a place and time when they work well. And a place where they break down, but I’m getting ahead of myself.

So stop me if you’ve heard this one before: dealing with images on the web is a pain in the app. To be performant requires many sizes and types for each image. And it would be best if you had some naming convention or system to organize them so you can use them correctly. This used to mean a bespoke manual process or custom scripts. But then there was JAMstack.

The JAMstack way

At its core, the JAMstack architecture has three tools for handling any problem:

1. Client-Side JavaScript
2. Build processes that produce static assets
3. 3rd Party APIs

When you only have a hammer, every problem looks like a nail. For dealing with images, any client-side solution means you already sent the wrong image over the network, leaving only the last two hammers.

Most frontend meta-frameworks now include some image component to generate these assets at build time. They are super popular and a big part of the rapid adoption of these frameworks. They are also why your “Hello World” may be hella fast, but your real site takes 30 minutes to build.

Here come the APIs

There are 3rd party APIs to do almost anything you can imagine, including image processing. Just upload your image to one of these services and request the converted version from a special url. This is an awesome solution to the problem.

But APIs are like npm packages. You probably need a few to get anything done, but they multiply fast until the fate of your app is no longer in your hands.

The whole JAMstack movement was built on the premise that you can outsource almost any part of your app to other services through an API. It is the release valve for anything the architecture itself can’t do.

So what is wrong with offloading complexity to other services?

You can’t remove complexity without paying for it eventually. Either literally with money or through the increasing complexity of coordinating all these services (see the Law of Conservation of Complexity).

No, really, what is wrong with a Frankenstein app approach?

Some specific downside of depending on lots of 3rd party solutions bolted together are:

1. Cost: Each service has different limits and pricing structures that combine to make it hard to figure out what you will pay as your app scales.
2. Coordination: Depending on one service can be tricky, but linking many of them together becomes exponentially difficult. Just the lack of local development and testing support for these services means large parts of your app may be untestable.
3. Changes: These services are constantly changing. As much as you hope for proper API versioning, you are not guaranteed to get it. Even additive changes result in unplanned work and unscheduled downtime.

Climbs off soapbox

There are three kinds of apps: a toy, a side project, and a business. For the first type, it matters less. Use whatever you can to learn what you want to know. For the last one, you probably have a team of intelligent people with spreadsheets to weigh the trade-offs, so once again, do what you will. But for the serious side project that you hope will eventually become a business or that you want to run long term for other reasons, beware of depending on too many 3rd party services. At least choose a way of building that allows you to build some of this into your project. Architect and Begin are both excellent choices, in my opinion.

Part 2: An Architect plugin for transforming images

If you wanted to skip the hot takes and see the tech, you are in the right place. Let me show you the image plugin I built. And to be clear the real hero of this story that I am trying to promote is the way of building apps that makes this kind of bite size dynamic functionality easy to build in.

What it does

What I needed is to drop in large images and be able to use any size version of those images by just requesting them. The plugin itself is very basic. It does not do anywhere near as much as the 3rd party APIs it replaces. But for me, it does all that I need it to do.

How to Install it

You can install the plugin for any Architect app with npm install @ryanbethel/arc-image-plugin. You can then add it to your app manifest like this:

#app.arc
@app
image-app

@http
get /
get /transform/*  #transform route for arc-image-plugin

@plugins
ryanbethel/arc-image-plugin

Currently, you need to add the get /transform/* in line 7 to register a route for the handler. This is because plugins cannot currently add new routes easily, but that is coming soon. You also need to add the handler itself and the config file below for the same reason. Again will we will add this into the plugin in a future version.

// src/http/get-transform-catchall/index.js
let arc = require('@architect/functions')
let { imageHandler } = require('@ryanbethel/arc-image-plugin')

exports.handler = arc.http.async(imageHandler)

#src/http/get-transform-catchall/config.arc
@aws
memory 1152
timeout 30

How to use it

The Architect framework serves static assets from a local folder that becomes an S3 bucket when deployed to AWS. You drop your giant.jpeg image in the public folder, and then once deployed, you can access it from anywhere in your app at http://example.com/_static/giant.jpeg or with a root relative path at /_static/giant.jpeg.

Architect includes built in fingerprinting, but lets ignore that for the moment for simplicity. With the image plugin, you can request the same image by swapping the “_static” for “transform” and include query parameters to get a different size (/transform/giant.jpeg?width=100&height=100). This will scale the image to fit in those dimensions while maintaining the aspect ratio.

The first time you make a request, it is transformed in a lambda and that new version is saved to a cache S3 bucket. The next request for that size is served from the cache. The uncached request can take 1-3 seconds, but the cached response takes only around 100ms. Scale to fit, cover, and contain transforms are supported as well as grayscale.

Fingerprinting and Updates

Best practice for modern web development uses fingerprinting of static assets. This is so that if you change giant.jpeg, but use the same filename, all the many layers of cache between a user and your site will not block them from seeing your updates. If you use fingerprinting through Architect (see fingerprinting in the docs), a request to giant.jpeg is turned into a request to giant-123abc.jpeg. And transform requests become /transform/giant-123abc.jpeg?width=100. Then when you update the file the fingerprint changes to giant-456xyz.jpeg, and the plugin will generate a new version because you are now requesting /transform/giant-456xyz.jpeg?width=100.

If you are interested, you can see the code for the plugin at https://github.com/ryanbethel/arc-image-plugin.

Local Development

One of the most valuable features of using Architect is local development support for almost everything. I built this plugin to have the same. It uses a local temp directory as a cache for transformed images, and it works just as it would when deployed.

What if I don’t want to write my own plugin

You are welcome to install and use this plugin for your own project. It is brand new, and you should consider it under development. Pull requests are welcome or if you want to fork it and make it your own that’s great too.

If you are new to Architect or if you are using Begin.com (which does not yet have full support for plugins), you can replicate most of what it does in a single route/lambda in your app. That is what I did before I rolled it into a plugin to be more portable. If you aren’t using Begin or Architect, you should try them out.