Writing a Publish & Subscription (PubSub) Service from Scratch

Overview

The Publish & Subscribe pattern is generally defined as a software architecture paradigm where senders of messages, called publishers, send their messages to an intermediary broker which utilizes any form of classification to notify receivers, called subscribers, of those messages. The receivers who have subscribed to those classifications do not need to know who emitted the initial message, but rather only that the message falls under the classification that they have subscribed to. There are several libraries that implement Pub Sub models across a wide variety of domains, and even for the article written here, there are libraries that solve the same problem that can be applied to UI and State management such as mroderick’s PubSubJS. There’s SignalR, with which you can natively configure a Pub Sub model within asp.NET. JMS within Java. Google even has Pub/Sub Client libraries that you can implement within C++, C#, Go, Java, Node.js, PHP, Python, and Ruby.

So why should you write your own Pub Sub model? The ability to write a pub sub service from scratch in different environments across different languages allows you to maintain a sensible software architecture for clients within a tight security and package approval process. These constraints may force you to build your own homegrown implementation instead of using your favorite Pub Sub library or API. In some legacy code environments, you may have requirements that don’t allow you to use the syntax due to its un-availability within that version. Additionally, many frontend frameworks make it slightly easier to build tightly coupled components. This can quickly become overly complex when passing data between parents and children. A quick reminder of the structure of a Pub Sub service and how it works internally, can provide insight into the types of problems that you can solve. In this article, we will focus on the implementation of a Pub Sub service within the paradigm of UI & State Management.

Basic Pub Sub Architecture:

We chose UI & State management because it still illustrates the purpose and scope of a Pub Sub service implementation without involving the complicated cross platform elements that some many network-based Pub Sub models employ. Some frontend frameworks, especially those that are unopinionated, leave it up to the developer to choose their own method of handling complex UI transformations from client data model changes. A Pub Sub service, within this context, is a simple and efficient way to share client data model updates among many components, without utilizing unnecessary resources. These resources in traditional models are usually spent creating observers or setters to underlying data which can be an expensive process.

Features

1. Publish, Subscribe, Un-Subscribe, Get Topics
2. Call Debouncing (Conserving procedural calls)
3. Group Chat Communication Demonstration

Software Architecture

Official Terminology

First, some terms to consider while building a simple pub-sub model:

Topics

Your “single source of truth”. A series of data which is held in a single local client area to be subscribed to. In this simple Pub Sub service, our “topics” will essentially just be a multi-dimensional JSON store. For each topic within a Pub Sub service, you may have many subscribers. Each of those subscribers can act as publishers in order to mutate and notify others of the changes.

Subscribe

An action that, when called, stores the data’s access path and a subscription object which contains a callback reference into a Subscription array. Informally, the act of subscribing means that you would like to be informed when there’s an update to the information subscribed to.

Publish

An action which, when called, notifies others who have subscribed to a topics path that a change has been made. The Subscribers can then do what they need to in order to update their local views. Informally, the act of publishing means that you have an update that you would like to share with other subscribers of the topic.

Get

An action which, when called, retrieves the topic from the Pub Sub service repository. An equivalent action can be performed by accessing the topics member variable. Informally, the act of getting means that you want to receive the information, but not necessarily to subscribe to the updates.

Un-Subscribe

An action which, when called, removes the data’s callback reference from the Subscriber object in order to prevent memory leakages. Informally, the act of un-subscribing means that you are telling the service that you will no longer b available to receive updates.

Debounce

A class which, when instantiated and called, allows you to debounce a function call until some delayed period when calls stop being made. This is useful when you have multiple different datasets mutating a published state in quick succession. It allows you to delay sending a message to the subscriber until some specified time interval after.

Un-Official Terminology

These are other terms that I will use within this context when describing certain features of this Pub Sub

Pluck

A function which, when called, follows the path to the subscribed object in order to retrieve the source data.

Theoretical Client Story

In a request to build a chat system, a client has requested that this system be able to do the following tasks:

*Note our “messenger” is the user who is sending messages.

1. Keep track of the messenger’s name.
2. Keep track of the messenger’s conversation history.
3. Be able to change the messenger’s name at any time.
4. Have the messenger’s name be updated within the conversation history.
5. Keep track of the date and time of each message.

Design Notes

Notice that each of these items mentioned within the client story, make no mention of an architected approach towards solving the problem. But we have 2 main domains of data here:

1. Users
2. Messages

With an additional domain to separate the client’s rendering:

3. Settings

A conversation, even though it is described, is not necessarily a saved entity. Rather it can also be a view generated by code, which reduces data redundancy. Each messenger is essentially a user, with the caveat being that the currently authenticated user physically using the application is the messenger. Rather than creating a dataset which models the business logic directly, we have decided to split the information into more atomic elements which can be virtually joined in order to meet the business requirements.

For sake of clarity, the only additional domain of information that I would be storing will be the currently authenticated user’s settings. For example, which conversation should I open at any given time? What user am I? Technically I could store this information within the user domain to determine this, but then I would be directly storing the users view information within the user data itself. This means that every time the user updates the conversation they are viewing, they would also be updating every component that relies on general user information. This seems unnecessary, so I have separated that additional domain of data as well below.

Where’s the Code & Full Guide?

∗∗∗ The working code, guide, demo for this article can be found here.

What You Need

→ Intermediate knowledge of JavaScript

→ Basic knowledge of Object-Oriented Design (modules, classes)

→ Basic knowledge of HTML

Technology Stack

→ HTML

→ JavaScript

This article will refrain from using any specific frontend framework to remain agnostic to your desired project.

What’s Next?

This model can be expanded to do MANY more things that aren’t directly covered in the source guide like:

• Scoped data changes within subscriptions.
• Partial updates data updates, reducing data size.
• GIT style change handling (to multi-component merge conflicts).

Additional Notes

Frontend Frameworks (i.e. angular, react, riot)

The following comments are recommendations based on prior experience implementing “Pub Sub” models:

1. Store data subscriptions within the component models that handle the business logic of an application.
2. Subscriptions should be created within the mount lifecycle and saved within the component state for re-use later on with any bound event listeners from your UI components.
3. Decoupling generic layout components from wrapping business logic is good practice if you plan on re-using those components across your application’s business domains. By creating generic components and wrapping them up in business logic, you can bootstrap the reusability or your components and generate UI Entity Frameworks that work for your development teams.

Off the Shelf Recommendations

As previously written in the introduction. There are several libraries that provide working Pub Sub services that can simply be pulled into your projects. Many, however, do not contain atomic data stores. Here are some recommendations we can make depending on your project type:

• Simple Pub Sub within TypeScript (Single Message Only):
  • Mitt
• Simple Pub Sub within JavaScript (Single Message Only):
  • Mroderick’s PubSubJS
  • Sahadar’s pubsub.js
• Simple Pub Sub within Python (Single Message Only):
  • PyPubSub
• Simple Pub Sub within C# (Single Model Only):
  • Upta’s pubsub
• Managed cross-language, networked Pub Sub for cross-device/language communication, with store and forwarding:
  • Google’s Pub/Sub Message and Event Streams

‍