Yes exactly, you would need to add each.
But if you just want to have every Database and Collection from your cluster represented in your Federated Database instance we actually have a short hand for that, you just put in your storage configuration (via the UI editor) something like this:
"databases" : [
{
"name" : "*",
"collections" : [
{
"name" : "*",
"dataSources" : [
{
"storeName" : "<atlas-store-name>",
}
]
}
]
}
]