Categories: architecture, idempotence

Idempotence. What is it and why should I care?

Reading Time: 5 minutes

Introduction

Idempotence is an area within software engineering that should be understood and addressed.

Definition

Idempotency can be defined as :

The property of certain operations whereby they can be applied multiple times without changing the result beyond the initial application

Source: Wikipedia

Let me explain how this definition translates into two concrete examples within the context of Software Engineering.

From an API perspective, “the property of certain operations” can be referred to as the processing of a HTTP request, whereas from a Client perspective (such as an Mobile/Web app) it could be the attempt to perform a HTTP request to a server.

For the next part of our definition “they can be applied multiple times without changing the result beyond our initial application“. Let us define two examples whereby our operations may or may not change the result beyond our initial application, providing idempotent and non idempotent examples.

API Operation

An API could be performing an operation (processing a HTTP request) to allow a user to participate in a music event by inserting a row in a participations database table. A user should only be able to participate once in a music event. Supposing this operation was non idempotent, what could happen?

Non idempotent example

– The user uses their client device (web/mobile) to participate in a music event by clicking a button “sign up to this music event on Monday 5th  August”

– The users request starts the operation on the server handling the HTTP operation.

– The first operation the server performs is inserting a new row in the participations table. In order to insert one in this table, there should be no row within the events table that consists of this given users participation for this music event. This operation completes successfully, great!

– The second operation adds a new row in the events table with type “user participated” to state that a new user has taken part in this given music event. Throughout this process however the database client connection closes due to an unknown reason. This should not happen!

– The user receives an error on his client device stating that something went wrong. The button on the application to participate in a music event is still green , meaning that they try to take part again!

– This time the user receives a success message stating that they have successfully participated. The server HTTP operation has inserted the participation and the associated event into the database.

It seems that all has gone as plan? No, not at all!

If we work out how many records are in the database within the events and participations table we now have the following:

Participations: 2 Rows

Events – 1 Row

What does this mean? It means that the application has violated its rule of only one participation per user, due to the route being non idempotent. Why? Because the client connection was closed upon the row insertion into the events table for the first request.

Making the operation idempotent

The engineer has made this route non idempotent by the following:

  • Performing the participation insertion based on whether the associated event is present or not. This event within the logic of the application is inserted after the participation insertion

In order to make this route idempotent, the insertion should not be based on the associated event being present. The event should however be present but only to store additional metadata, not determining whether a participation should be inserted or not.

Firstly after validating a HTTP schema, the next requirement should be to make this route idempodent in performing an:

Insert or update database operation

This can be referred to as an upsert operation.

Why does this help?

Every time the route is requested a maximum of one  participation will be created. If the operations after the upsert fails, for example the event creation, then no more participations will be created on the 2nd/3rd or 4th attempt of the request being made. The operations main task of allowing a user to participate is now idempotent.

Client operation

Lets take a new scenario by focusing on a mobile app which allows users to chat together. This application involves users sending messages to participants that they choose.

Non idempotent example

Lets take the scenario of when a user uses the application in an area which does not have much signal. Below is the step by step process:

– The user opens the application to see they have 3g signal. They start talking to their friends and exchange 4-5 messages.

– The user then enters a tunnel, as they are in a train,  the application completely loses signal.

– The user does not realise this and clicks send. The application does nothing, the send button stays green and the user attempts to send the message again.

– The user then realizes that they have no signal as the two messages they tried to send did not appear on the conversation screen.

– The user then comes out of the tunnel and regains signal. The mobile now sends the same message twice which is received by the recipient.

This result is not only annoying for the user but it could be easily avoided by focusing on idempotence during the development stage of sending a message. Imagine if a group chat occurred between 50 participants where 10 of them were all on the train at the same time trying to send the same message. The conversation would be a complete muddle once network connectivity restored due to the mass duplication of messages being displayed.

Idempotent example

Within the previous example, the client was able to re-send the same message multiple times (regardless if the previous message was successfully delivered or not making it non idempotent).

In order to make the client request idempotent the following would need to occur throughout sending a message:

The send button on the conversation should be un-clickable, if a message has already been sent but has not received a response

This would mean that the message could only be sent a maximum of once if communication problems occured.

Semi Idempotent Example

Shown below is an interesting example where a user finds a glitch where the client semi handles idempotency (until the application is reloaded). Seen though the server is 100% idempotent, the action still completes as normal and no duplications occur. This situation is not ideal for the end user, but is better than no idempotency at all.

idempotence example
semi idempotent example

Conclusion

Being idempotent is a mindset that needs to be adopted early by the Software Engineer for a given task, such as allowing a user to participate in a music event (Back end developer) and preventing sending multiple messages which are the same (front end developer). The ideal scenario is having both the back end and the front end idempotent. This indicates that it should be impossible to send a message multiple times from the client if it has not received a response from the server. It is also impossible to insert more than one row in a database for a previously broken request.

Please follow and like us:

6 comments

Leave a Reply

Your email address will not be published. Required fields are marked *

*