Custom Analyzers
Overview
An Atlas Search analyzer prepares a set of documents to be indexed by performing a series of operations to transform, filter, and group sequences of characters. You can define a custom analyzer to suit your specific indexing needs from the Atlas UI.
Syntax
A custom analyzer has the following syntax:
"analyzers": [ { "name": "<name>", "charFilters": [ <list-of-character-filters> ], "tokenizer": { "type": "<tokenizer-type>" }, "tokenFilters": [ <list-of-token-filters> ] } ]
Attributes
A custom analyzer has the following attributes:
Attribute | Type | Description | Required? |
|---|---|---|---|
name | string | Name of the custom analyzer. Names must be unique within an index, and may not start with any of the following strings:
| yes |
charFilters | list of objects | Array containing zero or more character filters. See
Usage for more information. | no |
tokenizer | object | Tokenizer to use to create tokens. See
Usage for more information. | yes |
tokenFilters | list of objects | Array containing zero or more token filters. See
Usage for more information. | no |
Usage
To use a custom analyzer when indexing a collection, include the
following in your index definition analyzers field:
Optional. Specify one or more character filters. Character filters examine text one character at a time and perform filtering operations.
Required. Specify the tokenizer. An analyzer uses a tokenizer to split chunks of text into groups, or tokens, for indexing purposes. For example, the whitespace tokenizer splits text fields into individual words based on where whitespace occurs.
Optional. Specify one or more token filters. After the tokenization step, the resulting tokens can pass through one or more token filters. A token filter performs operations such as:
Stemming, which reduces related words, such as "talking", "talked", and "talks" to their root word "talk".
Redaction, the removal of sensitive information from public documents.
Note
The text passes through character filters first, then a tokenizer, and then the token filters.
Example Collection
The Character Filters, Tokenizers, and
Token Filters pages contain sample index definitions and
query examples for character filters, tokenizers, and token filters.
These examples use a sample minutes collection with
the following documents:
{ "_id": 1, "page_updated_by": { "last_name": "AUERBACH", "first_name": "Siân", "email": "auerbach@example.com", "phone": "(123)-456-7890" }, "title": "The team's weekly meeting", "message": "try to siGn-In", "text": { "en_US": "<head> This page deals with department meetings.</head>", "sv_FI": "Den här sidan behandlar avdelningsmöten", "fr_CA": "Cette page traite des réunions de département" } }
{ "_id": 2, "page_updated_by": { "last_name": "OHRBACH", "first_name": "Noël", "email": "ohrbach@example.com", "phone": "(123) 456 0987" }, "title": "The check-in with sales team", "message": "do not forget to SIGN-IN. See ① for details.", "text" : { "en_US": "The head of the sales department spoke first.", "fa_IR": "ابتدا رئیس بخش فروش صحبت کرد", "sv_FI": "Först talade chefen för försäljningsavdelningen" } }
{ "_id": 3, "page_updated_by": { "last_name": "LEWINSKY", "first_name": "Brièle", "email": "lewinsky@example.com", "phone": "(123).456.9870" }, "title": "The regular board meeting", "message": "try to sign-in", "text" : { "en_US": "<body>We'll head out to the conference room by noon.</body>" } }
{ "_id": 4, "page_updated_by": { "last_name": "LEVINSKI", "first_name": "François", "email": "levinski@example.com", "phone": "123-456-8907" }, "title": "The daily huddle on tHe StandUpApp2", "message": "write down your signature or phone №", "text" : { "en_US": "<body>This page has been updated with the items on the agenda.</body>" , "es_MX": "La página ha sido actualizada con los puntos de la agenda.", "pl_PL": "Strona została zaktualizowana o punkty porządku obrad." } }
Example Custom Analyzers
The Atlas Search Visual Editor includes the following built-in custom analyzers based on a common-use template to help you get started:
Email Parser - Use this to tokenize email addresses up to 200 characters. For example, you can apply this analyzer on the
page_updated_by.emailfield in the Example Collection.Phone Numbers - Use this to create a single token from a US-formatted phone number. For example, you can apply this analyzer on the
page_updated_by.phonefield in the Example Collection.Dash-Separated IDs - Use this to create tokens from hyphen-delimited text. For example, you can apply this analyzer on the
messagefield in the Example Collection.
You can use these built-in custom analyzers or create your own custom analyzer using the Atlas Search Visual Editor or JSON Editor. To learn more about creating your own custom analyzers, see the following pages:
Note
When you add a custom analyzer using the Visual Editor in the Atlas UI, the Atlas UI displays the following details about the analyzer in the Custom Analyzers section.
Name | Label that identifies the custom analyzer. |
|---|---|
Used In | Fields that use the custom analyzer. Value is None if
custom analyzer isn't used to analyze any fields. |
Character Filters | Atlas Search character filters configured
in the custom analyzer. |
Tokenizer | Atlas Search tokenizer configured in the
custom analyzer. |
Token Filters | Atlas Search token filters configured in
the custom analyzer. |
Actions | Clickable icons that indicate the actions that you can perform on the custom analyzer.
|