Newly defined index does not appear in Atlas shared cluster using API or CLI

[Vardan_Sargsyan92 (Vardan Sargsyan)]

Hi

I am using MongoDB realm and I am trying to define the TTL index on my realm DB property. There are a couple of options to define an index on property. If I define the index manually it works as it should be and I am able to see the index and drop it in the atlas (Atlas → Browse Collections → Select Collection → Indexes). I have many clusters and would like to automate the process of creating indexes for all of them. To achieve that I need to either create an index using realm atlas API or create an index with Mongo-Cli.

For both cases, I have the same issue. I create an index and get a success message (“Index is being created” from the CLI and “{ }” from the API), but when I navigate to the Atlas → Browse Collections → Select Collection → Indexes screen and refresh the data I don’t see any new indexes, even a day after the added index does not appear.
When I try to create an index again on the same property I am getting this error
{"detail":"Unexpected error.","error":500,"errorCode":"UNEXPECTED_ERROR","parameters":[],"reason":"Internal Server Error"} for both API and CLI.

So any ideas why this happens and why I am not able to see the newly added indexes?

Here is a sample of creating a single index on string property with Atlas-API

https://cloud.mongodb.com/api/atlas/v1.0/groups/6155e35b3fe240082e0e464d/clusters/msoisales-dev-eu2-atlas/index

{

           "db": "msoisales-dev-eu2-realm",

           "collection": "MasterItem",

           "keys": [{ "Name": 1 }],

           "collation": { "locale": "en_US", "strength": 2 }

}

[Jason_Tran]

Hi @Vardan_Sargsyan92,

It does sound like the index is being created from your description (or possibly has been).

For mongocli index creation, could you try the following and advise accordingly:

1. Have the Atlas UI open at the Database Deployments section (where you see all your clusters). Ensure the cluster where you are creating the index exists there in the Database Deployments section.
2. Run the mongocli create index command
3. After running the above command, quickly switch back to the Atlas UI and check if the following are shown:

• A blue bar at the top of the database deployments section:

• The following entry (or similar) exists in the Project Activity Feed:

User performed a Data Explorer CRUD operation, which modifies data
Username: abcdefgh /// this is a altered public api key for example purposes
Operation: createRollingIndex
Database: testdb
Collection: testcoll
Changed by an API key with Public key: abcdefgh from 159.xx.xx.xx

Note: The blue bar indiciating changes are being deployed could be for another change to a resource in your project but it’s a good thing to check during the time of index creation as well for troubleshooting purposes

Could you then also provide further information regarding:

• The mongocli version being used
• The full mongocli command used to create the index (Please redact any personal or sensitive information before sending here)

I did some testing using a M10 cluster with mongocli version 1.23.1 and was able to see the index created from mongocli in the Atlas UI using the following command:

$ mongocli atlas clusters indexes create mongocliindex --clusterName "testCluster" --collection "testcoll" --db "testdb" --key "test:1"

Your index is being created

If you have multiple projects in your Atlas organization, you can possibly try specifying the --projectId option when creating the index to ensure the index is created on the correct project. The project ID can be found in the Project Settings page

Regards,
Jason

[Vardan_Sargsyan92 (Vardan Sargsyan)]

HI @Jason_Tran. Thank you for the quick response.

I see the blue bar when I create a new index on the property but when it completes I am not able to see the index for any of the clusters. I don’t see it on activity feed too.

I specify the project Id with the command mongocli config set
before creating an index I call the following commands:

mongocli config set public_api_key $public_api_key 
mongocli config set private_api_key $private_api_key 
mongocli config set org_id $org_id 
mongocli config set project_id $project_id
mongocli config set mongosh_path $mongosh_path

Here are some details which could be helpful.

The command:

mongocli atlas clusters indexes create testIndex --clusterName <clusterName> --collection <collection> --db <db> --key <field:value> --projectId <project Id>

MongoCli version - 1.23.1
The cluster tier - M0 Sandbox (it’s a Dev cluster)
Type - Replica Set-3 nodes
Linked Realm app to that cluster
version 5.0.6

[Vardan_Sargsyan92 (Vardan Sargsyan)]

Update!

I just noticed that the cluster update appears on the projects activity feed. But when I navigate to the Atlas → Browse Collections → Select Collection → Indexes I don’t see the index there.

One more thing which I have noticed is the different operation types on the activity feed depending on how I create an index.

When I do it with MongoCli or API the project’s activity feed shows the Operation: createRollingIndex whereas when I create an index manually the feed shows Operation: createIndex

[Jason_Tran]

Thanks for providing all those details @Vardan_Sargsyan92,

The cluster tier - M0 Sandbox (it’s a Dev cluster)

The mongocli index creations are done in a rolling fashion (as is the Atlas API for creating a rolling index). Due to this method of index creation and the current cluster tier in use for your dev environment (M0), the index will not be created as per the Atlas M0 (Free Cluster), M2, and M5 Limitations documentation :

M0 free clusters and M2/M5 shared clusters do not provide the ability to build indexes with a rolling build.

In saying so, I understand that this behaviour is not expected as an error response to the request or suitable alternative may better indicate the index was not built / being built. I have escalated this internally and was confirmed that this is an issue with mongocli not returning an error when it should.

In the meantime, the index creation command via mongocli that you’ve provided should work as expected if done against a M10+ tier cluster as these tier of clusters do not share the above limitations as the free/shared tier clusters.

Regards,
Jason

[Jason_Tran]

In addition to the above as a workaround for creating indexes for your M0,M2/M5 tier clusters, you can create them through mongosh using db.collection.createIndex()or the Atlas UI via Data Explorer as you have done before.

Regards,
Jason

[Vardan_Sargsyan92 (Vardan Sargsyan)]

Yeah. It would be nice to have the opportunity to get a message that the index was not built as a request’s response. It would be much better than just UNEXPECTED_ERROR.

Thank you @Jason_Tran for clarifying. Now it’s clear why I was not able to see the indexes. I’ll try to use mongosh. Thank you for a workaround.

[Jason_Tran]

Hi @Vardan_Sargsyan92,

Just as an update, the response should now be as follows for M0,M2 and M5 tier clusters:

$ mongocli atlas clusters indexes create indexnew --clusterName "M0Cluster" --collection "testcoll" --db "testdb" --key "testkey:1"

/// Redacted the project ID for my test environment

Error: POST https://cloud.mongodb.com/api/atlas/v1.0/groups/<PROJECT_ID>/clusters/M0Cluster/index: 400 (request "TENANT_ROLLING_INDEXES_UNSUPPORTED") Rolling index builds are not supported on shared-tier clusters. Upgrade to Dedicated to use this feature.

Thank you for raising this initially

Regards,
Jason

[Vardan_Sargsyan92 (Vardan Sargsyan)]

HI @Jason_Tran. Thank you for the update. That’s awesome.

By the way, I have tried the mongosh and it works fine with shared clusters. But I guess I need to create another topic on how to run a js file inside the script . I do database connection and then try to run an index creation command inside js file but no luck. I am investigating the following posts and hope they will help.

post 1

post 2