NubeSync: Modern .NET Offline Data Sync

POSTED JULY 2020



Prologue

If you are a .NET developer and created a mobile app in the recent years, there is quite a chance you went with Xamarin Forms and an Azure Mobile Client backend.

Well at least that is what we did, as Azure had it all: push notifications, authentication, database connection – and most important for us: offline data sync.

Since most of our apps are “forms-over-data-line-of-business” stuff it is vital to make them work flawless even in low to none connection scenarios.
The offline sync implementation was not perfect and lacked some polish in certain areas like conflict handling and performance, but that were compromises we could accept.



Going down

But times have changed and so did Microsofts investment into its Azure Mobile Client backend, the project was abandoned, and users should transition to the shiny new App Center Data. Unfortunately, this project was canceled before it even left beta status.

So, with several Azure Mobile Client based apps in production and even more of them in the pipeline, how to go on about this?

Using Azure Mobile Client for a new greenfield app is a no-go, since even the template to create a mobile backend has been removed from the Azure portal.



What We Want

It showed that the head start Azure Mobile Client was giving us has melted away and left us stuck in a framework that is no longer maintained.

We don’t want to end up in that spot ever again, so we decided that whatever we were using in the future had to be based on widely adopted standard technologies that run everywhere – and can be hosted anywhere.

Good news is, that most of the Azure backends capabilities (like authentication and database connection) are sort of baked into ASP.NET Core nowadays – so the only part left out for us is offline data sync.



NubeSync to the rescue!

Therefore, we took matters in our own hands and developed a bi-directional offline data sync framework (how hard can it be, huh?) – titled NubeSync.

And since we weren’t the only ones struggling with the retirement of the Azure Mobile Client backend we decided to open-source our effort, to hopefully give others in the .NET community a starting point for implementing offline sync in their mobile apps.

Some parts are very similar to the Azure Mobile Client and some parts are different.

In this and the following blog posts I’d like to outline some basic principles of the NubeSync framework.



The Client

Client data is usually persisted into a SQLite database in an offline scenario and since one of our goals was to stick to widely adopted standard technologies, we picked SQLite as the default client database (although other kinds of storage, e.g. file based ones could be easily implemented).

When a record is saved to the SQLite database via the NubeSync client the changes are split into “operations”. That means every change that was made to the record becomes a separate operation, that is stored in a special table in the SQLite database.

All those operations are then pushed to the server, where the server applies those operations to the cloud-hosted main database.

This is a different approach than the original Azure Mobile Client implemented that brings several benefits:

Merge conflicts can be handled more easily, since not only the current version of a record is sent to the server, but a whole “audit log” of the changes is available, making it possible to resolve conflicts on a “per-field” basis.
That means when a record was changed by several users this might not even result in a conflict, when every user changed a different field of the record!

Syncing “operations-style” also enables handling conflicts solely at the server, making it easier to change conflict handling in the future – you don’t have to consider old versions of the app in the field, that still might handle conflicts the old way.

It can also bring performance improvements, when not the whole record has to be sent to the server but only the fields that were changed on the client.

When the client is done pushing its changes, all records that were changed in the meantime by other clients are pulled from the server and the client database is up to date.



The Server

Implementing the server is straight forward, when the user sends operations (= all changes the user made to the database) the server processes them and updates the main database accordingly.

Since clients cannot manipulate records directly (only via operations) there is no need to implement POST/PATCH/DELETE methods in the controller – all that is required is a GET controller that returns the current records of the database.



The Future

Currently we are using NubeSync in greenfield projects, and if it proves itself to be battle-ready there, we will transition existing Azure Mobile Client based apps to it.

Currently we are sticking to Azure SQL as a database backend, but NubeSyncs architecture allows the backend storage to be easily changed, basically to everything that is Entity Framework Core compatible – CosmosDB would be a popular example.

And since NubeSync is not only limited to Xamarin apps we will also use it in other client apps that has to be offline capable – be it Blazor, Desktop, UWP and maybe even a Flutter client.



Getting Started

If you are in a similar situation as we are and would like to give NubeSync a try (or just get some inspiration for rolling your own offline sync), check out the documentation and some samples on https://github.com/stefffdev/NubeSync