MongoDB is coming to London and we want to see you there. Tickets are going fast, but we’re putting two free day-passes up for grabs - all you need to do is share your very best mug shot online!
By now, most of you should have a MongoDB mug. If not, where have you been the past 6 years? We've shipped mugs around the world but we rarely get to see the Mug owners. To get in the draw to win two free tickets to MongoDB London, simply snap a photo (alone or with others) with your MongoDB mug(s) and tag it on Twitter with the hashtag #mongodbmug.
Good luck, and look forward to seeing you in November at MongoDB London.
What to Expect at MongoDB London
MongoDB London will not only reveal the why behind using MongoDB in a range of use cases, but crucially a varied roster of presenters will explain the how. MongoDB Co-Founder and CTO, Eliot Horowitz, will open the day and give a roadmap of what's to come in MongoDB 2.8.
At MongoDB London, you will see how you can build applications never before possible in your organization. Gary Collier, CTO at AHL Man Group, will discuss how his team created a single platform for all of its financial data with MongoDB. That project resulted in a 25x improvement in tick throughput and slashed the required disk storage by 40 percent. Additionally, James Stewart, will describe the process of digital transformation in the UK Government and Robert Hill, at Capgemini, will be showcase building a real-time view of your business to integrate all of your siloed data--also called the Single View of the Customer.
Alongside a number of user presentations, there will be several MongoDB experts giving talks on everything from the Internet of Things to examples of MongoDB best practices in Financial Services. Finally, you will get the chance to sit down with MongoDB experts at our "Ask The Experts" hall to answer your toughest questions about MongoDB.
Don't delay. Snap your MUG and send it over.
Sharding Pitfalls Part III: Chunk Balancing and Collection Limits
In Parts 1 and 2 we have covered a number of common issues people run into when managing a sharded MongoDB cluster. In this final post of the series we will cover a subtle, but important distinction in terms of balancing a sharded cluster as well as an interesting limitation that can be worked around relatively easily, but is nonetheless surprising when it comes up. 6. Chunk balancing != data balancing != traffic balancing The balancer in a sharded cluster cares about just one thing: Are chunks for a given collection evenly balanced across all shards? If they are not, then it will take steps to rectify that imbalance. This all sounds perfectly logical, and even with extra complexity like tagging involved the logic is pretty straight forward. If we assume that all chunks are equal, then we can rest assured that our data is being evenly balanced across all the shards in our cluster and rest easy at night. Although that is sometimes, perhaps even frequently, the case it is not always true - chunks are not always equal. There can be massive “jumbo” chunks that exceed the maximum chunk size (64MiB), completely empty chunks and everything in between. Let’s use an example from our first pitfall , the monotonically increasing shard key. For our example, we have picked just such a key to shard on (date), and up until this point we have had just one shard and had not sharded the collection. We are about to add a second shard to our cluster and so we enable sharding on the collection and do the necessary admin work to add the new shard into the cluster. Once the collection is enabled for sharding, the first shard contains all the newly minted chunks. Let’s represent them in a simplified table of 10 chunks. This is not representative of a real data set, but it will do for illustrative purposes: Table 1 - Initial Chunk Layout Now we add our second shard. The balancer will kick in and attempt to distribute the chunks evenly. It will do this by moving the lowest range chunks to the new shard until the counts are identical. Once it is finished balancing, our table now looks like this: Table 2 - Balanced Chunk Layout That looks pretty good at the moment, but lets imagine that more recent chunks are more likely to have more activity (updates say) than older chunks. Adding the traffic share estimates for each chunk shows that shard1 is taking far more traffic (72%) than shard2 (28%) despite the chunks seeming balanced overall based on the approximate size. Hence, chunk balancing is not equal to traffic balancing. Using that same example, let’s add another wrinkle - periodic deletion of old data. Every 3 months we run a job to delete any data older than 12 months. Let’s look at the impact of that on our table after we run it for the first time (assuming the first run happens on July 1st 2015). Table 3 - Post-Delete Chunk Layout The distribution of data is now completely skewed toward shard1 - shard2 is in fact empty! However, the balancer is completely unaware of this imbalance - the chunk count has remained the same the entire time, and as far as it is concerned the system is in a steady state. With no data on shard2, our traffic imbalance as seen above will be even worse, and we have essentially negated the benefit of having a second shard for this collection. Possible Mitigation Strategies If data and traffic balance are important, select an appropriate shard key Move chunks manually to address the imbalances - swap “hot” chunks for “cool” chunks, empty chunks for larger chunks 7. Waiting too long to shard a collection (collection too large) This is not very common, but when it falls on your shoulders, it can be quite challenging to solve. There is a maximum data size for a collection when when it is initially split which is a function of the chunk size and data size as noted on the limits page . If your collection contains less than 256GiB of data, then there will be no issue. If the collection size exceeds 256GiB but is less than 400GiB, then MongoDB may be able to do an initial split without any special measures being taken. Otherwise, with larger initial data sizes and the default settings, the initial split will fail. It is worth noting that once split the collection may grow as needed and without any real limitations as long as you can continue to add shards as data size grows. Possible Mitigation Strategies Since the limit is dictated by the chunk size and the data size, and assuming there is not much to be done about the data size, then the remaining variable is the chunk size. This is adjustable (default is 64MiB) and can be raised in order to let a large collection split initially and then reduced once that has been completed. The required chunk size increase will depend on the actual data size. However, this is relatively easy to work out - simply divide your data size by 256GB and then multiply that figure by 64MiB (and round up if it is not a nice even number). As an example, let’s consider a 4TiB collection: 4TiB divided by 256GiB = 16 64MiB x 16 = 1024MiB Hence, set the max chunk size to 1024MiB , then perform the initial sharding of the collection, and then finally reduce the chunk size back to 64MiB using the same procedure. . Thanks for reading through the Sharding Pitfall series! If you want to learn more about managing MongoDB deployments at scale, sign up for my online education course, MongoDB Advanced Deployment and Operations . Planning for scale? No problem: MongoDB is here to help. Get a preview of what it’s like to work with MongoDB’s Technical Services Team. Give us some details on your deployment and we can set you up with an expert who can provide detailed guidance on all aspects of scaling with MongoDB, based on our experience with hundreds of deployments.
Latinas in Tech: Andy Morales Coto
This spotlight is part of a blog series to amplify exceptional Latina talent in the tech industry. Through our partnership with Latinas In Tech, this article originally appeared on their site . Tell us about yourself, Andy. How did you get to where you are today? I’m originally from Costa Rica and have been living in NYC for the past six years. I’m a product designer, but I wasn’t always one: before coming to New York, I was working in multiple industries, as a game designer, a copywriter, and a digital marketer. But I guess most of that is just titles and places I come from, not really the way I got to be where I am. If I look more deeply, I would say that the moments that have led me to where I am today are a mixture of privilege and the fallout of self-discovery. I was born in an upper middle class family, the daughter of two public servants — a doctor and an engineer — and learned English pretty early on at their behest. I was able to go to private school my whole life, up until college, when I attended the University of Costa Rica, which is publicly funded by all Costa Ricans. I wouldn’t say I had a luxurious life growing up: there were certainly hand-me-downs from my sisters, but I also never had a problem buying a video game console if I wanted it — I’d just have to give up having a birthday party (and I did). Overall, I’d say my parents motivated me to follow my dreams, and would gladly take me to any classes I wanted (English, robotics, programming, drawing) from the time I was a little girl. In that sense, I always had a leg up, understood what was considered “excellence” in education, and pretty early on set my mind on studying abroad eventually. With that said, my comfortable life became, well, not comfortable at all when I came out at 19. College changed my life completely. Finally being able to understand who I was, I came out as queer to my very conservative parents, and the reception was extremely toxic. For the first time, I understood what it meant to not be able to afford a meal, or even a bus ticket. I walked miles to go to college several times, hell-bent on finishing my degree in communications (the closest thing to tech, I figured, without the toxicity of the homogeneity of computer science). Finally I graduated, but my whole perception of the world had changed: I became more empathetic and less judgmental of others, and I knew what depression and trauma were. Coming out made me a better human being with an understanding of my privilege, and I’m deeply grateful that I took that step. Coming out made me a better human being with an understanding of my privilege, and I’m deeply grateful that I took that step. I continued working for several years after graduating from college, did another degree in marketing while I worked, and finally got accepted into Parsons (NYC) on a scholarship to study transdisciplinary design. And here we are! Oh, also, and this is very important: I’m married to a lovely American and live with her and two fluffy tabby cats in Brooklyn. NYC is what I call home now (and probably forever). What inspired you to pursue a career in the tech industry? I think pretty early on I was in awe of technology, and I don’t just mean computers, but also cars, glasses, electricity, hammers. I’ve always admired anything that expands the possibilities of what a human can do. But my “aha moment” happened when I was 10 and accessed the internet at the University of Costa Rica. My mother was a teacher there and had access to connection before the rest of the country did. She’d sometimes let me use her computer, and I still remember using Netscape in complete fascination of what this meant for humanity: we would all be connected. That’s when it really clicked for me: I love this, I love computers. As a manager at MongoDB, what have been some of the most memorable and impactful projects you’ve worked on so far? I’m the most proud of the people I manage, and seeing them grow every day. My direct reports are infinitely more talented than I am in some ways, and I welcome that. I want to be surrounded by people more talented than I am, and they’re going to change the face of the design industry, I have no doubt. Watching them get better and better, lead projects of their own, and successfully navigate difficult stakeholder situations — well, it just puts a smile on my face! But, apart from that, a specific project I’ve enjoyed is Blue Sky, a yearly design-driven sprint that we do in conjunction with key stakeholders to create the “concept car” of the product I lead design for. This will be the second year we do Blue Sky, and we hope to use design thinking beyond the graphical user interface, partnering with product and engineering to imagine the future experience of MongoDB Realm in the CLI and the IDE. With each Blue Sky, design positions itself as a partner for our stakeholders, and our proposals coming out of the project tend to be implemented up to 75% of what we design. It’s exciting to become strategic partners in the direction the product will take. How has your culture (and/or other identity marker) shaped you as a leader? As a manager? Well, my culture is a mixture of queer culture, Costa Rican culture, and NYC culture. I think all of these shape me as a leader, because it means I am not a monolith as a person; I’ve learned to see the world through many different perspectives. Being able to compare and contrast how different cultures view or react to situations makes me self-aware, and puts me in a position where I strive to understand how others are reacting to situations, in the frame of their culture. I’d say this is empathy, which is a bit of a design cliche, but I actually think that it’s more than empathy — it’s vulnerability and sobering humility. Trust me, I wasn’t always super self-aware, but as I’ve gotten to know the world through different cultural lenses, I’ve realized that I have to be careful with how I help others be what they consider their very best. Whether it’s grappling with cultural expectations or navigating workplace biases, we fight through many challenges as Latinx women. What’s one you’re working through currently? I’m definitely sometimes worried about how I come off to my teammates, particularly those who are not Latin American. I can be emotionally vulnerable, honest, and bubbly: I cry at work at times, I am not afraid of jumping into difficult conversations, and I laugh loudly. Unfortunately, as a woman and as a Latina, these can be seen as vapid qualities, symbols of weakness. Why is she so loud, so emotional, so open to talking? In the past, I’ve tried to cover this up by being serious, talking softly but more deeply, and avoiding vulnerable conversation; as I’ve grown older, I’ve realized that inhibiting those qualities hinders me at work, because it makes me feel miserable, and that I end up gaining more supporters in the long term by being as open-hearted as I am. I definitely think I have my upbringing in Costa Rica to blame for that: it is not the norm for women to be like that at work, but while I was growing up I certainly saw more female bosses be open and vulnerable. I can be emotionally vulnerable, honest, and bubbly: I cry at work at times, I am not afraid of jumping into difficult conversations, and I laugh loudly. This, of course, sometimes brings some internal turmoil: Am I just not meant to be in this American culture? Am I borrowing from my Costa Rican experiences without giving back? There’s a certain sense of duty that you feel toward those who are in your home country, even if your current definition of home has changed (I consider myself more a New Yorker than anything else, by now). To be honest, I don’t have a solution to that sense of duty and loss, and I struggle with it pretty often. I deal with it by donating and helping others that want to chase their dreams in the USA, but I still struggle with it. It’s hard not to miss the place you grew up in. It’s a big piece of you, no matter where you go. Looking to the future, what inspires you, and what initiatives are you most excited about right now? I’m inspired by games, and I can’t wait to continue using playful design in every product I design. Tangentially, I design live action role playing (LARP) games, and I can’t wait to be able to play with my other designer friends again, hopefully at a house by the beach. What’s one piece of career advice you’ll never ever forget? One of my professors from grad school, Mathan Ratinam, told me once that throughout his career he learned that you are lucky if you get to choose a job for one of three reasons: you love the work, you love the mission, or you love the people. I’ve tried loving the work, and I’ve tried loving the mission, but let me tell you: if I don’t enjoy working with the people, I’m not going to be happy in the long term. Whenever I consider a career move, I don’t focus on the mission or the work as much anymore, because those haven’t brought me the happiness that I thought they would. People do. Whenever I consider a career move, I don’t focus on the mission or the work as much anymore, because those haven’t brought me the happiness that I thought they would. People do. How do you reset when you’re in a funk? I let myself cry/experience sadness first, I go to therapy (cannot stress this enough: if you can afford it, please go to therapy), and I practice Muay Thai. I just love kicking a bag and sweating the problems out, you know? Any podcasts or blog recommendations? I don’t really listen to podcasts or read blogs that often. I play games and I read books; those are my two sources of design inspiration. I’d say, if you can, play “Zelda: Breath of the Wild,” to see what the epitome of design is. Also, play any LARP from the Golden Cobra Challenge: http://www.goldencobra.org/ . You can print those for free and play them with people online. Bookswise, I’ve been reading Fall ; or, Dodge in Hell , by Neal Stephenson, but sometimes it hits too close to home. Is there anyone you’d like to shout out for their support along your career journey? My wife, Crystal Morales. She’s the best thing that has ever happened to me. She is the smartest career advisor I know, and the smartest person I know. Period. Mathan Ratinam, of course, whom I mentioned before. He has inspired me so many times and listened to me talk for hours on the phone. A champ. My friends who, during college, helped me get a meal when I couldn’t: Olalla, Edith, Diana (my best friend since then), Warren, Memo, MaJo. A big hug to them all. And my college teacher Andrea Alvarado, who understood the pains I was going through at home when I came out and, instead of failing me, gave me extra work to do, showing me that part of being compassionate is never being condescending. Andy is thriving as a lead product designer at MongoDB . If you’re ready to work with what sounds like an incredible group of people, here are three open roles you should check out! Product Manager, Server Sales Development Representative Lead Engineer, Docs Platform