Using Cloud Pub/Sub on Node.js from Elm

Why Elm?

Elm is a pure functional language with a syntax similar to Haskell, but optimized for writing front-end web applications. It has a transpiler that converts Elm modules into JavaScript, as well as a very small and fast runtime to interface with browsers.

Elm is pretty firmly positioned as a front-end tool, to make dynamic, single-page applications for browsers. But with the help of the elm-node package, much more is possible!

Caveats

This article will walk you through building an Elm application that runs in Node.js, and uses the Node.js Pub/Sub libraries for communicating with Google Cloud Pub/Sub. Please note that we do not recommend doing this for your production work at this time; this article is mostly for the curious!

Why “pure functional”?

Many of us are familiar with basic functional patterns, like passing a lambda function to map() in JavaScript.

Pure functional refers to programs that describe what it is that you want to figure out, declaratively. Most programs are written in an imperative fashion, which is to say that you, as the programmer, are telling the computer what steps to take, and in what order, and managing the state of the program. This introduces a great burden on the programmer, because one must constantly think about how to maintain the consistency of state, as well as what might go wrong to handle unexpected errors from unknown side-effects.

JavaScript tools have been developed to help with the state problem at least, such as Redux, whose model was inspired by Elm. Redux lets you declare your program’s state as an immutable structure that is updated using mutators. However, many things can still go wrong at runtime, because the compiler isn’t helping you to understand the possible flows through the logic.

A truly pure functional language has no flow control. There are no steps to run, and since your program is basically a mathematical equation, the compiler is able to fully understand it, and catch most logic errors at compile time. In fact, it is even possible to write proofs about pure functional programs.

Elm … on Node?

Because Elm is simply a transpiler, it’s also possible to run the resulting code using the Node.js runtime, rather than a browser. This gives you the ability to build very lightweight server processes using all of the existing Node compatible libraries from npm/yarn, while staying in Elm for as much of the application as you like. Elm’s lack of side effects also means that many logic and data structure modules can be shared between client and server.

This article will walk you through building an Elm application that runs in Node.js, and uses the Node.js Pub/Sub libraries for communicating with Google Cloud Pub/Sub.

My series of “Using Pub/Sub From” articles:
Using Cloud Pub/Sub from Kotlin
Using Cloud Pub/Sub on Node.js from Kotlin/JS

Also of interest:
Things I wish I knew about Google Cloud Pub/Sub: Part 1

Project setup

Begin by installing a recent version of Node.js. (Or using nvm!)

You will also want the commands for Elm and elm-node, so install them globally:

npm install -g elm elm-node

You can then create an elm-node project:

elm init
elm-node --example-elm > src/Main.elm
elm-node --example-js > src/index.js
elm-node --js src/index.js src/Main.elm

Right away, I found an error in the generated sample code — index.js needs to refer to Elm.Main, not Elm.MainWithJs. These types of errors may be fixed by the time you try it.

The generated program is extremely simple. Some program declarations are made in Elm, including a port to talk with JavaScript. The JavaScript code then subscribes to the port and responds to calls.

App design

Because Elm and JavaScript are interfacing across two very different kinds of coding, we need to think about how they will communicate. The Elm guide discusses this in light detail. The recommendation is essentially to try to avoid piping every JavaScript call through to Elm. Instead, make an interface that makes sense in Elm. This also helps with the Elm/JavaScript transition, which relies on passing nothing more complicated than JSON. (No object handles or anything.)

For this very simple example, I’ve chosen to expose an interface to Elm that lets the Elm code request that messages be sent or ack’d, and then is able to receive messages in an event-driven manner. For a real application, you’d probably need something more complex, like passing numbers or JSON across to refer to specific topics or subscriptions, and caching those with a Map on the JavaScript side.

The ultimate recommendation of the Elm designers is to rewrite client libraries in native Elm, but given the size and complexity of the Node Pub/Sub libraries, this is probably not feasible at this time. So we will use ports!

Port definitions

The place we must start here is to define our ports. To define our ports, we must first decide what sort of capabilities and interface we want to have on the Elm side. How much will be declared in Elm, and how much will reside in JavaScript/TypeScript?

Let’s briefly pick this apart a bit!

First, we have the definitions of the messages we will be passing back and forth. Our message payload for this example will be a simple string, so the MessageToSend will simply be an alias for String. When receiving a message, we must also know the Pub/Sub message ID, so MessageReceived has both the payload and the ID.

The ports themselves are basically declarations of messages that may be passed between Elm and JavaScript. The log port will let us print log messages, publish lets us ask for a message to be sent, receive goes from JavaScript to Elm for a received message, and ack lets us ack a Pub/Sub message. One thing to note is that I’ve chosen to make the parameter to publish a record rather than just a string. This is to make it simpler to add more parameters later on, if we wanted to.

You can see that the messages that go from Elm to JavaScript are functions returning type Cmd msg. This is a generic type that specifies a command the Elm program requests in order to get to the next state. Messages from JavaScript to Elm use Sub msg, which is a function returning type Sub msg. This means that we are subscribing to messages with a parameter type of MessageReceived, and will use those to trigger a state update.

Elm model

Now that we’ve defined our ports, let’s define our Elm-side model.

In this case, our very simple example is just sending one word of this sentence at a time (thus toSend for what’s left to send, and receivedMessage for what we’ve received). Also, we define Msg here as the type we will take in to trigger a state update. In this case, the only message we’ll receive is Received, with a parameter of type MessageReceived. But you might have others here in a more complex application.

Elm Main

If this looks more like a data structure than a program, you wouldn’t be wrong! In Elm, they are one and the same.

The main symbol we export in this Elm module is main, and it will be a worker program (no UI). The init function will specify the initial state. The subscriptions function will list the messages we’re willing to receive. And update will define how to get an updated state from a received message, as well as further actions.

This defines the init function, which returns a tuple. This includes the initial state (toSend has all but the first word, and receivedMessage hasn’t received any yet) as well as next steps to take. The Cmd.batch function builds an array of commands to execute, in any order, probably all at once. Note that these are our ports from above.

This one is very simple! We’re simply saying that we are able to receive message type Received (defined above as part of Msg).

This is the core of the logic in the Elm app. Basically when the Elm app is passed type Received with a message (again, defined above) and a previous state model, we will return a new model. The model will update toSend to be one fewer word than the previous model; we will also add the new received word to receivedMessage. (Note that these updates are not done in-place, but this is returning a new model; so we must use let if we want those values right away.) Finally, the Cmd batch will print out what we’ve received, ack the message in Pub/Sub, and potentially publish the next word. (All of which may happen at once, in any order.) If there’s nothing left to send, then no further commands are issued.

Running the app

Your process will no doubt vary here, but for the complete sample, I have created an npm run script that you can use to start it. You should see something like this:

> npm run run
[…]
Use Ctrl+C to exit.
Elm starting Pub/Sub Elm test
JS publishing: Mary
JS: received: 1 : Mary
Elm received so far: ‘Mary’
JS publishing: had
JS acking: 1
JS: received: 2 : had
Elm received so far: ‘Mary had’
[…etc…]

Note that, because our app is publishing messages sequentially, the final message string will always be in order. A nice exercise for the reader would be to have it publish more than one message (one per word) at a time, and you can see that they may come out of order. From there, you might play with assembling them in order again, and so on.

Next steps

Check out a full working example that runs on the Pub/Sub emulator:

https://github.com/feywind/elm-node-pubsub

Have you used Elm on the server side with Node.js? Do you find this interesting and wish to hear more? Please feel free to leave a comment about this post or things that you find interesting/promising in regards to Elm on Google Cloud Platform!