Realm app auto deployment - not deploying Rules, Schema (but is deploying Functions, Triggers, etc.)

Hi Mongo Community,

Frequent reader (as of late) - first time poster.
I’ve configured the GitHub auto-deployment to deploy our Realm app from our DEV env to our “next,” env - QA. I’ve gone through the tutorials, and our company has had a recent “accelerated” consulting engagement - and this was all POC’d and working fine with two personal/free-tier Projects & Realm-apps (exporting with realm-cli beta 4).

However, what I see now is that “mechanically” the auto-deployments are working (evident by the History tab) - but it seems that ONLY the Functions, Triggers, Values & Auth Provider of the app are deploying (and NOT the Schema, Rules, and GraphQL Schema/Customer Resolvers) - even though I clearly see all the json files under the data_sources/mongodb-atlas sub-structure (with the db and collections).

Other notes:

  1. current realm-cli is 2.0.0.-beta.5
  2. the realm_config.json only has the realm app name and config version fields (all other fields removed), and the data_sources/mongodb-atlas/config.json has had its config.clusterName field removed - see screenshots below.

Screenshots:

So with that - is there something that I’m fundamentally missing? I’m happy to provide any other info that might help.

Thanks in advance!

Regards,
Chris

1 Like

Hi @Christopher_Roth.

I wanted to check a couple of things.

Was the app exported in the new format, if you’re not sure then it would be good to export it again?

What happens if you try to manually import the app using realm-cli?

Hi @Andrew_Morgan,
Thank you for the prompt follow-up - very much appreciated.
There’s nothing too propriety/sensitive about the info in the exported realm-app structure - nonetheless, if it’s OK, I’m going to attempt to email that do you directly. It’s a text file w/ the result of the Windows cmd “tree” commend (more’ing it in a command prompt allows it to display pretty).
And then I’ll have to follow-up later in the day today with the result of importing the app directly into our QA env via realm-cli - and I appreciate the suggestion.
Regards,
Chris.

Hi! I have the same problem. And yes it works with realm-cli, schema imported properly. But if use automatic deploy again it resets back and deletes all schemas.

Ah, interesting (but sorry to hear Mikhail). I appreciate you sharing your experience; I expect to confirm the same later today (I have a number of other priorities to work on this morning).

Hi @Andrew_Morgan,
Just a quick follow-up that I got back to it, and can confirm (Mikhail’s observation) that when importing the Realm app via the realm-cli, the entire configuration imports successfully (the Schema & Rules look good - and all other artifacts).
Regards,
Chris.

This looks like a similar issue I was having with deploying the Realm App via GitHub. Everything except the schema rules and webhooks imported. I’m guessing you don’t use webhooks and therefore don’t see that they are not there either. Realm app export and import via GitHub fails

@Christopher_Roth I liked your screenshots and description of the problem. Much better than mine!

@ConstantSphere - thanks for contributing! In fact, I did read you post - but wasn’t quite sure if it was the same issue, and so submitted this post. But, it does seem that we encountered the same GitHub auto-deploy issue; and perhaps I was hasty, but I didn’t notice (or couldn’t see) any realm app structural difference, compared to when this was working for me (granted, in the context of different projects/clusters) just a few weeks ago - at least not beneath data_sources/mongodb-atlas/(db name)/… directories. And you’re correct, our app does not (yet?) have any web hooks.

Hi - just thought I’d post a follow-up. @Andrew_Morgan - assuming you received the tree-view .txt file of my exported realm app structure, is it OK / what you expected? Should I fall back to an earlier realm-cli version (to beta 4 if that’s possible)? Any other action that I can/should take? Thanks in advance!

Hi @Christopher_Roth, sorry - I got distracted by other activities!

I just created a new app, exported it, added it to GitHub, and then linked it to a new app.

The only change I had to make to make it work was to change name in /realm_config.json.

I no longer need to remove config.clustername from /data_sources/mongodb-atlas/config.json - looks like that’s been fixed.

I checked the directory structure for data_sources and mine seems to match yours.

Does the linked data source have the same name (mongodb-atlas) in the app you’re trying to import into?

Hi @Andrew_Morgan - thanks and no worries. I appreciate the update & will take another crack at it here this afternoon (and be prepared to do a realm-cli import, if it doesn’t appear to work). And yes, our target env’s realm-app has the same linked source name of mongodb-atlas - image

Thanks and I’ll follow-up with what I see. Regards!

Hi @Andrew_Morgan - apologies, it wasn’t until today that I was able to get back to this. I still see the same results - and that being that the Rules & Schema content are deleted upon a “successful” auto-deployment via GitHub.
I’d un-installed and re-installed the realm-cli (now beta 6); being on windows, I toggled the git core.autocrlf configuration setting (to ensure that only LF chars remain in the files upon commit into GitHub) - but all resulted in the same.
So, I re-imported the app into our QA env via the realm-cli import command (which works) to ensure the app’s successful deployment in that env, but I’m back to being at a loss as to what else to try.
Is there a new GitHub Mongo app that I should re-install in GitHub?
I think perhaps I should re-engage the Mongo engineer that conducted our accelerated consultation - again, at that time (April 22/23) - this all worked.
Thanks!

Hi @Christopher_Roth. We noticed recently that a deployment to our production system (using the “old directory structure”) appeared to work, but it was actually enforcing the previous schema rules. Checking the UI showed the new schema rules were in place but it wasn’t until we added a single space to the file via the UI and deployed via the UI that it actually took effect. Have you seen any behaviour like that on your system?

1 Like

Hi @ConstantSphere, thanks for following-up & I think I follow - and no, I don’t believe I’ve yet to see this (but will keep an eye out for this scenario as we make subsequent deployments). That said, we’re early in our project’s life, only have have two envs so far (Dev and QA), and haven’t performed many deployments (our project’s had a bit of swirl recently, not Mongo-related). But, we’ll be getting back to deploying up to QA (and higher) from Dev here on a regular cadence shortly.

I think our situation is that a) our project has only ever been on the “new Realm directory structure,” and b) I went ahead and opened a Mongo Support ticket (for this topic) last week, and demo’d the behavior to two Mongo support engineers late last week. And from that session, I think I we identified our issue - and that is, in our GitHub repo… the Realm app content (its full directory structure) was checked into a sub-directory within the repo (and not into its “root”), and even though the Realm deployment config specified that sub-directory (to deploy from), well - that was the issue. I created a second/new GitHub repo (with no sub-directory for the Realm app, and not other content), and just committed the Realm app into it (at the root) - making the typically realm_config.json and data sources config.json updates - and when merging from one branch to another, it all deploys fine to the target/QA env (incl. the Rules & Schema).
So now I’m just waiting for one of the Mongo support engineers to replicate my scenario (given a GitHub repo with a sub-directory that contains the Realm app content), and confirm whether he sees the same / what we all observed when working together.
I’ll update this post once that’s been confirmed (or refuted).
Thanks!

1 Like

I have seen this exact same behavior, and discovered the same workaround. It has me rather concerned, what if an update to production doesn’t get applied?

I just tried to make a repro for this issue with a new project, but wasn’t able to reproduce the behavior with a new application. I can consistently reproduce this in an existing application though, I will file a support ticket when I have a window where I can leave my dev environment in a bad state for support to examine.

Any updates on this?

On my own system much of this now works properly but not everything. About a month ago I built up the courage to try another deployment from GitHub with my “old” directory structure and it mostly worked. By that I mean that the deployment appeared successful in the deployment log, all of the Rules, Webhooks and functions worked and the app remained connected to the cluster. Realm then synced back to GitHub with the “new” directory structure which was fine except that all of my hosted files were missing and therefore my entire website disappeared!

Despite that - this is the best it’s ever worked. If you don’t use the hosting feature this might all be fine. As it happened I could quite easily rebuild the hosting folder via my CI chain so no big harm done. I now have the new directory structure in github and can successfully deploy to github and onto Realm.

However, there is still an issue that I can’t change anything in the Realm UI whist I have “Enable Automatic Deployment” on, otherwise all my hosted files get deleted both in Realm and in my githib repo! This is only really an issue on my dev environment as I would never change anything via the UI in prod.

Hi @Thomas_Juel_Andersen - sorry, I’d never followed-up to my last Post. Mongo was able to reproduce my issue (of the auto-deployments not deploying the Schema/Rules, if the Realm app structure is versioned-controlled in a sub-directory structure of a GitHub repo, and not at the root) - so, from my Support Case, a ticket was logged with the Realm engineering team. So, we pivoted to just having a flat GitHub repo for our Realm App (so that this was no longer an issue for us), and all of our auto-deployments have been successful since (over the past few months, deploying weekly or over two weeks). That said, I just followed-up on my Support Case (asking if there are any updates form the Realm engineering team); I hadn’t received any updates since late June (other than it was in the Realm engineering team’s hands). And, I haven’t (yet?) observed the issues that Simon’s described.
Thanks.