$group is a blocking stage, as in it needs to wait for every document to pass through before it can output your groups. The more data you can filter out in the $match stage, the better your performance will be.
However, you can see in your explain plan that the IXSCAN (index scan) phase is taking the longest. It also is looking at 3.4m keys but only return 10,000 documents. Make sure that you have an index which matches your query (the $match stage) and keep it as small as possible.