Hello everyone, I have ran into a problem regarding the language analyzer’s stopwords. I couldn’t find a solution anywhere for it, I hope someone has a better understanding of this issue.
I have these documents in my collection:
{title: “Head of IT”}, {title: “Customer Head of IT”}, {title: “Head Chef”}, {title: “Head of Engineering”}.
The main problem: When I do an Atlas text search for “Head of IT”, and I sort the results by relevance I get them back in the following order: “Head of IT”, “Head Chef”, “Customer Head of IT”, “Head of Engineering”. After a bit of investigation I found that this is because the lucene.english analyzer is stripping out IT from the text search, because “it” is a stopword (it strips out “of” as well, but that was expected).
This is what my index looks like:
{
"mappings": {
"dynamic": false,
"fields": {
"title": [
{
"dynamic": true,
"type": "document"
},
{
"analyzer": "lucene.english",
"type": "string"
}
]
}
},
"analyzers": [
{
"charFilters": [],
"name": "stopwordRemover",
"tokenFilters": [
{
"tokens": [
// list of stopwords
],
"type": "stopword"
},
{
"stemmerName": "english",
"type": "snowballStemming"
}
],
"tokenizer": {
"type": "standard"
}
}
],
"synonyms": [
{
"analyzer": "lucene.english",
"name": "synonyms",
"source": {
"collection": "synonyms"
}
}
]
}
What I’d like to happen is that MongoDB should ignore in some way the stopwords coming from the language analyzer and only take into consideration my stopwords defined in the index. If there is some workaround that’s good as well, but this would be a crucial thing for me. Thanks in advance!