How does Realm indexes work with string properties and partial match search

If I have a string property that has content similar to a paragraph (usually several sentences of content) and want to speed up search for that property when I do “contains” queries, will making that property an Index help speed that up? Or does indexing only help if you are searching an exact match for the string content?

In other words, I have a property that has string content that i am looking to find partial matches with for example “contains” the phrase “project management”. Does indexing still help make queries like this faster or does it need to be searching for “exact” matches of the entire string for indexing to help?

Is there any documentation that explains how the indexing works behind the scenes? Thanks!

Hey @Shawn_Murphy,

Adding a string index will only optimize queries for equality matching in strings currently, though we may optimize other cases in the future. If you are looking for the technical details of the string index, you can check out the source code comments. Querying with the “contains” operator uses the Boyer-Moore algorithm.

It sounds like what you are looking for is full text search. It is not trivial to implement, but it is a frequently requested feature, you can add your vote to indicate community interest here: Full text search support for Realm – MongoDB Feedback Engine

Thanks a lot @James_Stone ! This is very helpful info and just what I was looking for. Much appreciated.

Best,
Shawn

@James_Stone Apologies for the late reply. If a String property is indexed, does that speed up querying in all operations that test against that property, such as .where() and .filter() (assuming we use NSPredicate or Realm Queries in those functions instead of a closure-based approach?

Suppose I have this, for instance:

let allFoos: Results<Foo> = someRealm.objects(Foo.self)
let theOneFoo: Foo = allFoos.filter({ $0.someIndexedStringProperty == "someuniquestring" })

If someIndexedStringProperty is the primaryKey for Foo, Realm is blazingly fast at retrieving that Foo using:

let theFoo: Foo = someRealm.object(ofType: Foo.self, forPrimaryKey: value)

I’m looking to get similar query performance on another indexed String property that is unique across all Foos, but is not the primaryKey. So far, I haven’t been able to do that.

Hi @Bryan_Jones, yes there should be a marked improvement in performance when querying using equality matching on an indexed property vs a non-indexed property. If you are not seeing this, can you open an issue in the appropriate SDK (looks like realm-swift) and share your schema with the indexed property and the query? We should be able to help you sort out what’s going on there.

@James_Stone I do indeed see a difference in performance. It does seem slower than a query by primaryKey, but querying with == on an indexed vs. non-indexed string property definitely shows a difference.

My use case is constructing an NSOutlineView on-the-fly as the user expands each level. The reason is that the OutlineView shows the entire filesystem from / on down, so constructing the entire tree of model objects (800,000+) ahead of time is slow and wasteful. (Realm also doesn’t have support for a live-object “tree” collection, so I have to build the datasource level-by-level manually.)

Realm’s query performance using the “materialized paths” approach seems to be sufficient to create each level of the tree’s data on-the-fly in the time it takes to animate the level open.

To follow up here for those interested query performance and how an index can help, I’m happy to say that we now have support for full text search. There’s a nice intro here.