Realm Environments?

[Tim_N_A]

How are MongoDB Atlas + Realm projects supposed to organize environments?

Are Realm environments supposed to all use the same database?

Are we expected to migrate entire Realm applications from project to project throughout a development cycle?

What’s the correct to organize MongoDB and Realm intro environments?

Why do I always have trouble finding documentation on this online?

Thanks for any help!

Double posting from old thread in other topic: How do I organize LOCAL-DEV, TEST, and PRODUCTION environments? - #3 by Tim_N_A

[Sumedha_Mehta1 (Sumedha Mehta)]

Hi Tim - @Lauren_Schaefer 's talk on this might be a good place to start: Building CI/CD Pipelines for MongoDB Realm Apps - YouTube

[Lauren_Schaefer]

Thanks for the pointer to my talk, Sumedha! Here is a blog post that covers the same content: How to Build CI/CD Pipelines for MongoDB Realm Apps Using GitHub Actions | MongoDB

[Tudor_Suditu]

How to Build CI/CD Pipelines for MongoDB Realm Apps Using GitHub Actions | MongoDB I was following the github actions tutorial from you @Lauren_Schaefer and I would still have some questions:

Supposing I have a simple realm app with a trigger, trigger configuration json is bound to a specific database. How can I use a single repo to manage dev/ prod environments seamlessly, so that I only need to do a PR when I’m confident that dev environment is fine? I would totally avoid using UI to deploy changes for prod unless there is a simple setting which could only be done there.

My thinking was something like this:

1. dev project → dev atlas cluster → dev realm app
2. prod project → prod atlas cluster → prod realm app

I would have dev project connected to my repository develop branch. Once I’m confident everything’s good, I would do a PR into master for release. Merging the PR would call github actions as you described which will:

1. checkout code
2. update all triggers json files database setting with production DATABASE ( they are named differently at the moment)
3. update realm.json with production APP_ID and APP_NAME
4. use mongo cli to push updated code

This doesn’t seem like a good option to me, since setting an automated deploy on the production application for master branch would not allow me to do any changes on the realm UI without breaking the master branch I guess, since mongo will overwrite dev code… ( with this approach, master branch will have dev settings, but relying on github actions to do all the required json manipulations )

I’m pretty confused about the best way of doing it. Your blog post it’s focused on the mobile application and I cannot see anything related to triggers which are tied to database / collections. Would you advise me which option would work best conceptually for my use case? Thanks a lot in advance!

[Sumedha_Mehta1 (Sumedha Mehta)]

Hi Tudor -

I’m interested in learning more here

This doesn’t seem like a good option to me, since setting an automated deploy on the production application for master branch would not allow me to do any changes on the realm UI without breaking the master branch I guess, since mongo will overwrite dev code… ( with this approach, master branch will have dev settings, but relying on github actions to do all the required json manipulations )

Is there a reason or scenario why you anticipate using both Github Deploy + UI to deploy changes to your app? From what I can tell, your concern is mainly around when you are doing both frequently.

The other thing I’m not sure I understand is why making a change to your UI on the dev app (which is tied to your dev branch), doesn’t appropriately propogate to your prod app, since the same GH action will trigger a deploy to prod, changing the environment.

[Tudor_Suditu]

Attaching the build.yml started from the skeleton that you did. It kinda seems to work, however, adding a trigger with preimages set to enabled, then obviously I need to change trigger configuration for preimage in each json with some scripts… I could do that , but I have some doubts that the same structure will hold for you guys. Looking forward to hearing from you how the environments issue would be solved so that it doesn’t involve human interaction to manually create triggers, which obviously would be error prone.

name: Build

on:
  push:
    branches:
      - "main"

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2

      # SET ENVIRONMENT VARIABLES WE WILL USE IN LATER STEPS
      - name: "Set env vars"
        if: ${{ github.ref == 'refs/heads/main' }}
        run: |
          echo "REALM_APP_ID=${{ secrets.PROD_REALM_ID }}" >> $GITHUB_ENV
          echo "CLUSTER_NAME=cicd-prod-cluster" >> $GITHUB_ENV
          echo "DATABASE_NAME=production" >> $GITHUB_ENV
          echo "APP_NAME=cicd-prod" >> $GITHUB_ENV
      - name: "Install the Realm CLI"
        run: |
          npm install -g mongodb-realm-cli
      - name: "Authenticate Realm CLI production"
        if: ${{ github.ref == 'refs/heads/main' }}
        run: |
          realm-cli login --api-key="${{ secrets.PROD_REALM_API_PUBLIC_KEY }}" --private-api-key="${{ secrets.PROD_REALM_API_PRIVATE_KEY }}" --realm-url https://realm.mongodb.com --atlas-url https://cloud.mongodb.com
      - name: update triggers configurations
        run: |
          for filename in triggers/*.json; do
           echo "`jq '.config.database="${{ env.DATABASE_NAME }}"' $filename`" > $filename
           cat $filename
          done
      - name: update realm-config
        run: |
          echo "`jq '.app_id="${{ env.REALM_APP_ID }}"' realm_config.json`" > realm_config.json
          echo "`jq '.name="${{ env.APP_NAME }}"' realm_config.json`" > realm_config.json
          cat realm_config.json
      - name: update data sources
        run: |
          echo "`jq '.config.clusterName="${{ env.CLUSTER_NAME }}"' data_sources/mongodb-atlas/config.json`" > data_sources/mongodb-atlas/config.json
          cat data_sources/mongodb-atlas/config.json
      - name: deploy realm application
        run: |
          realm-cli push --remote="${{ env.REALM_APP_ID }}" -y

[Tudor_Suditu]

It seems to me that it’s kind of clunky to execute github actions on main branch push just so that I modify json configs and deploy altered code to production app. If at any point in time I want to add a hotfix from the UI in prod, then UI will overwrite the github content ( which will have dev configurations << github actions don’t alter the actual repository as shown above>> ) , and the main branch will eventually have prod values in configurations…

That means that at the next pull request from dev to master, we ll need to solve conflicts, something like that. I’m just wondering whether there’s a better approach or not into solving this.

[Tudor_Suditu]

Attaching final version of the build script. I already tested this one, and it seems to be working fine with triggers / secrets / config values for environments ( development.json, production.json ). I would be very interested in finding out if this approach is an anti pattern or not at least.

Current approach for our use case:

1. Developer uses the realm development application which has automated github deployments enabled for repo develop branch
2. When developers are confident that development version works as expected, we make sure all development.json configs have values for production in production.json
3. We issue a PR for colleagues to review the code and the underlying changes
4. On PR merge, the build script is ran which handles differences between database name, realm app name etc, and everything is getting updated.

PS: in this use case, we disabled automated deployments for production, so that all the updates will be flowing through our PRs and repo main branch in a safe manner.

name: Build

on:

  push:

    branches:

      - "main"

jobs:

  deploy-production:

    if: ${{github.ref == 'refs/heads/main'}}

    runs-on: ubuntu-latest

    steps:

      - uses: actions/checkout@v2

      # SET ENVIRONMENT VARIABLES WE WILL USE IN LATER STEPS

      - name: "Set env vars"

        run: |

          echo "REALM_APP_ID=${{ secrets.PROD_REALM_ID }}" >> $GITHUB_ENV

          echo "CLUSTER_NAME=xxxxx" >> $GITHUB_ENV

          echo "DATABASE_NAME=xxxxx" >> $GITHUB_ENV

          echo "APP_NAME=xxxxxxx" >> $GITHUB_ENV

      - name: "Install the Realm CLI"

        run: |

          npm install -g mongodb-realm-cli

      - name: "Authenticate Realm CLI production"

        run: |

          realm-cli login --api-key="${{ secrets.PROD_REALM_API_PUBLIC_KEY }}" --private-api-key="${{ secrets.PROD_REALM_API_PRIVATE_KEY }}"

      - name: update triggers configurations

        run: |

          for filename in triggers/*.json; do

           echo "`jq '.config.database="${{ env.DATABASE_NAME }}"' $filename`" > $filename

           cat $filename

          done

      - name: update realm-config

        run: |

          echo "`jq '.app_id="${{ env.REALM_APP_ID }}"' realm_config.json`" > realm_config.json

          echo "`jq '.name="${{ env.APP_NAME }}"' realm_config.json`" > realm_config.json

          echo "`jq '.environment="production"' realm_config.json`" > realm_config.json

          cat realm_config.json

      - name: update data sources

        run: |

          echo "`jq '.config.clusterName="${{ env.CLUSTER_NAME }}"' data_sources/mongodb-atlas/config.json`" > data_sources/mongodb-atlas/config.json

          echo "`jq '.config.namespacePreimageConfigs=(.config.namespacePreimageConfigs | if(type=="array" and length > 0) then map(.dbName="${{ env.DATABASE_NAME }}") else [] end) ' data_sources/mongodb-atlas/config.json`" > data_sources/mongodb-atlas/config.json

          cat data_sources/mongodb-atlas/config.json

      - name: deploy realm application

        run: |

          realm-cli push --remote="${{ env.REALM_APP_ID }}" --include-package-json -y

[Kevin_18580 (Kevin Boutin)]

Why wouldn’t you use environment variables like this (realm_config.json)?

{
    "app_id": "%(%%environment.values.realmAppId)",
    "config_version": 20210101,
    "name": "%(%%environment.values.realmAppName)",
    "location": "US-VA",
    "provider_region": "aws-us-east-1",
    "deployment_model": "GLOBAL",
    "environment": "%(%%environment.values.environment)"
}

[Jimpy]

@Kevin_18580 I tried to use environment variables as you suggested, and it worked for app_id and name, but not for the environment. It looks like the environment is kind of self-referencing itself, so how should it know which environment to use?

[Drew_DiPalma]

Just to confirm – App Services does not support environment variables for the Environment field as this would create a cyclical reference. Our recommendation here would be to hardcode the environment in the repo/branch and then update it as a part of the code promotion / CICD process.

[Kevin_18580 (Kevin Boutin)]

I am using the following in my environments/development.json:

{
    "values": {
        "cluster": "my-cluster",
        "environment": "development",
        "realmAppId": "triggers-someid",
        "realmAppName": "Triggers",
    }
}

Each environment json would have different values.

The project/cluster will have the deployment environment set and that determines which JSON file is used in the Realm application.

FYI: we are using 3 clusters (one for development, qa and production).

[Charles_Benedito]

what do you do to handle multiple enviroment using the same codebase?

I have two clusters:
development app on atlas linked to my development branch on github
production app on atlas linked to my main branch on github

but what determines the environment of the app is the environment value from the /realm_config.json file

so if the environment value from my realm_config.json is “development” and I merge my development branch to main, the production app will read the env vars from environments/development.json

how to set and forget what env files to use in which app