What to Expect from Realm JavaScript v12
Kenneth Geisshirt5 min read • Published May 22, 2023 • Updated Apr 02, 2024
Rate this article
The Realm JavaScript team has been working on Realm JavaScript version 12 for a while. We have released a number of prereleases to gain confidence in our approach, and we will continue to do so as we uncover and fix issues. We cannot give a date for when we will have the final release, but we would like to give you a brief introduction to what to expect.
You will continue to see version 11 releases as bugs are fixed in Realm Core — our underlying database. All of our effort is now focused on version 12, so we don’t expect to fix any more SDK bugs on version 11, and all new bug reports will be verified against version 12. Moreover, we do not plan any new functionality on version 11.
You might expect many breaking changes as we are bumping the major version, but we are actually planning to have as few breaking changes as possible. The reason is that the next major version is more breaking for us than you. In reality, it is a complete rewrite of the SDK internals.
We are changing our collection classes a bit. Today, they derive from a common Collection class that is modeled over ReadonlyArray. It is problematic for Realm.Dictionary as there is no natural ordering. Furthermore, we are deprecating our namespaced API since we find it out of touch with modern TypeScript and JavaScript development. We are dropping support for Altas push notifications (they have been deprecated some time ago). Other changes might come along during the development process and we will document them carefully.
The goal of the rewrite is to keep the public API as it is, and change the internal implementation. To ensure that we are keeping the API mostly untouched, we are either reusing or rewriting the tests we have written over the years. We implemented the ported tests in JavaScript and rewrote them in TypeScript to help us verify the new TypeScript types.
Realm JavaScript has historically been a mixture of C++ and vanilla JavaScript. TypeScript definitions and API documentation have been added on the side. A good portion of the API does not touch a single line of JavaScript code but goes directly to an implementation in C++. This makes it difficult to quickly add new functionality, as you have to decide if it can be implemented in JavaScript, C++, or a mixture of both. Moreover, you need to remember to update TypeScript definitions and API documentation. Consequently, over the years, we have seen issues where either API documentation or TypeScript definitions are not consistent with the implementation.
Realm JavaScript builds on Realm Core, which is composed of a storage engine, query engine, and sync client connecting your client device with MongoDB Atlas. Realm Core is a C++ library, and the vast majority of Realm JavaScript’s C++ code in our old architecture calls into Realm Core. Another large portion of our old C++ code is interfacing with the different JavaScript engines we are supporting (currently using NAPI [Node.js and Electron] and JSI [JavaScriptCore and Hermes]).
Our rewrite will create two separated layers: i) a handcrafted SDK layer and ii) a generated binding layer. The binding layer is interfacing the JavaScript engines and Realm Core. It is generated code, and our code generator (or binding generator) will read a specification of the Realm Core API and generate C++ code and TypeScript definitions. The generated C++ code can be called from JavaScript or TypeScript.
On top of the binding layer, we implement a hand-crafted SDK layer. It is an implementation of the Realm JavaScript API as you know it. It is implemented by using classes and methods in the binding layer as building blocks. We have chosen to use TypeScript as the implementation language.
We see a number of benefits from this rewrite:
Deliver new features faster
First, our hypothesis is that we are able to deliver new functionality faster. We don’t have to write so much C++ boilerplate code as we have done in the past.
Provide a TypeScript-first experience
Second, we are implementing the SDK in TypeScript, which guarantees that the TypeScript definitions will be accurate and consistent with the implementation. If you are a TypeScript developer, this is for you. Likely, your editor will guide you through integrating with Realm, and it will be possible to do static type checking and analysis before deploying your app in production. We are also moving from JSDoc to TSDoc so the API documentation will coexist with the SDK implementation. Again, it will help you and your editor in your day-to-day work, as well as eliminating the previously seen inconsistencies between the API documentation and TypeScripts definitions.
Facilitate community contributions
Third, we are lowering the bar for you to contribute. In the past, you likely had to have a good understanding of C++ to open a pull request with either a bug fix or a new feature. Many features can now be implemented in TypeScript alone by using the building blocks found in the binding layer. We are looking forward to seeing contributions from you.
Generate more optimal code
Last but not least, we hope to be able to generate more optimal code for the supported JavaScript engines. In the past, we had to write C++ code which was working across multiple JavaScript engines. Our early measurements indicate that many parts of the API will be a little faster, and in a few places, it will be much faster.
As mentioned earlier, all new functionality will only be released on version 12 and above. Some new functionality has already been merged and released, and more will follow. Let us briefly introduce some highlights to you.
First, a new unified logging mechanism has been introduced. It means that you can get more insights into what the storage engine, query engine, and sync client are doing. The goal is to make it easier for you to debug. You provide a callback function to the global logger, and log messages will be captured by calling your function.
Second, full-text search will be supported. You can mark a string property to be indexed for full-text search, and Realm Query Language allows you to query your Realm. Currently, the feature is limited to European alphabets. Advanced functionality like stemming and spanning across properties will be added later.
Last, a new subscription API for flexible sync will be added. The aim is to make it easier to subscribe and unsubscribe by providing
subscribe() and unsubscribe() methods directly on the query result.While Realm JavaScript version 12 will not bring major changes for you as a developer, we believe that the code base will be at a better place. The code base is easier to work with, and it is an open invitation to you to contribute.
The new features are additive, and we hope that they will be useful for you. Logging is likely most useful while developing your app, and full-text search can be useful in many use cases. The new flexible sync subscription API is experimental, and we might change it as we get feedback from you.