The Rise of the Strategic Developer
The work of developers is sometimes seen as tactical in nature. In other words, developers are not often asked to produce strategy. Rather, they are expected to execute against strategy, manifesting digital experiences that are defined by the “business.” But that is changing. With the automation of many time-consuming tasks -- from database administration to coding itself -- developers are now able to spend more time on higher value work, like understanding market needs or identifying strategic problems to solve. And just as the value of their work increases, so too does the value of their opinions. As a result, many developers are evolving, from coders with their heads-down in the corporate trenches to highly strategic visionaries of the digital experiences that define brands. “I think the very definition of ‘developer’ is expanding,” says Stephen “Stennie” Steneker, an engineering manager on the Developer Relations team at MongoDB. “It’s not just programmers anymore. It’s anyone who builds something.” Stennie notes that the learning curve needed to build something is flattening. Fast. He points to an emerging category of low code tools like Zapier, which allows people to stitch web apps together without having to write scripts or set up APIs. “People with no formal software engineering experience can build complex automated workflows to solve business problems. That’s a strategic developer.” Many other traditional developer tasks are being automated as well. At MongoDB, for example, we pride ourselves on removing the most time-consuming, low-value work of database administration. And of course, services like GitHub Copilot are automating the act of coding itself. So what does this all mean for developers? A few things: First, move to higher ground. In describing one of the potential outcomes of GitHub Copilot, Microsoft CTO Kevin Scott said, ““It may very well be one of those things that makes programming itself more approachable.” When the barriers to entry for a particular line of work start falling, standing still is not an option. It’s time to up your strategic game by offering insight and suggestions on new digital experiences that advance the objectives of the business. Second, accept more responsibility. A strategic developer is someone who can conceive, articulate, and execute an idea. That also means you are accountable for the success or failure of that idea. And as Stennie reminded me, “There are more ways than ever before to measure the success of a developer’s work.” And third, never stop skilling. Developers with narrow or limited skill sets will never add strategic value, and they will always be vulnerable to replacement. Like software itself, developers need to constantly evolve and improve, expanding both hard and soft skills. How do you see the role of the developer evolving? Any advice for those that aspire to more strategic roles within their organizations? Reach out and let me know what you think at @MarkLovesTech .
4 Common Misperceptions about MongoDB
One year ago, in the middle of the pandemic, Dev Ittycheria, the CEO of MongoDB, brought me on as Chief Technology Officer. Frankly, I thought I knew everything about databases and MongoDB. After all, I’d been in the database business for 32 years already. I’d been on MongoDB’s Board of Directors and used the products extensively. And of course I’d done my due diligence, met the leadership team, and analyzed earnings reports and product roadmaps. Even with all that knowledge, this past year as MongoDB’s CTO has taught me that many of my preconceived notions were just plain wrong. This made me wonder how many other people might also have the wrong impression about this company. And this blog is my attempt to set those perceptions straight by sharing my four major revelations of the last year. My first revelation is that MongoDB is not trying to become this generation’s relational database. For years I assumed that MongoDB basically wanted to be a better, more modern version of Oracle when it grew up. In other words, compete with the huge footprint of Oracle and other commercial RDBMSs that have been the industry archetype for so long. I was way off. The whole point of MongoDB is to leave all those forms of archaic, legacy database technology in the historical dust. This was never supposed to be an evolution, but instead a revolution. Our founders not only envisioned the world's fastest and most scalable persistent store, but also one that would be programmed and operated differently. The combination of embedded documents and structures combined with automatic high availability and almost-infinite distribution capability all add up to a fundamentally different way of working with data, building applications, and running those applications in production. Oracle and (SQL*Server, etc) still hang their hats on E.F. Codd’s 51-year old vision of rows and columns. To obtain high availability and distribution of data, you need add ons, options packages, bailing wire and duct tape. And you need a lot of database administrators. Not cheap. Even after all that, you’re still trailing the technological edge. This is how wrong I was. Our durable competitive advantages over these legacy data stores make competing with those products almost irrelevant. We instead focus on the modern needs of modern developers building modern applications. These developers need to create their own competitive advantage through language-native development, reliable deployments to production, and lightning fast iteration. And the world is noticing; just check out the falling slope of Oracle and SQL*Server and the rising slope of MongoDB on the db-engines website. Which brings me to my second revelation: MongoDB was built for developers, by developers. I always knew that MongoDB was exceedingly fast and easy to program against. One time while I was bored in a meeting (yes, it happens here as well!), I built an Atlas database, loaded it with 350MB of data, downloaded and learned our Compass data discovery tool, built-in analytics aggregation pipelines, and our Charts package, and embedded live charts in a web page. This took me all of 19 minutes, end to end. To build something like that for engineers , it just has to be built by engineers , ones that are free to focus on all the rough edges that creep into products as features are added. I was first exposed to software planning and management over 40 years ago, and my LinkedIn profile shows a pretty diverse tour around the industry. Now, one year in, I can emphatically state that engineering and product at MongoDB are both different and better than any company I’ve ever had the privilege to work at. Our executive leadership gives engineering and product broad brushstokes of goals and desired outcomes, and then we work together to come up with detailed roadmaps, updated quarterly, that meet those goals in the way we think best, with no micromanagement. And we’re not afraid of 3-5 year projects, either. For example, multi-cloud was more than three years in the making. Also unlike any other company I’ve been at, we embrace the creation and re-payment of tech debt, rather than sweeping it under the rug. We do this through giving our product and engineering teams huge amounts of context, delivered with candor and openness. And one more essential thing; we have an empowered program management team that improves processes (including killing them) as fast as we create them. In short, we paint the targets for our teams and let them decide how and when to shoot. They even design the arrows and bows. It’s true bottoms-up engineering. Our engineers feel valued and understood. And that, in turn, empowers them to develop features that make our customers feel valued and understood, like a unified query language, or real-time analytics and charting directly in the console, or multi-region/multi-cloud clusters where all the networking cruft is taken care of for you. And this brings me to my third revelation: MongoDB is built for even the most demanding mission critical applications. Fast? Yes. Easy? Of course. But mission-critical? That’s not how I saw MongoDB when I used Version 2 for a massive student data project 10 years ago. While it was the only possible datastore we could have chosen for the amount of data and the speed of ingestion and processing needed, it was pretty hard to set up and use in a 24 x 365 environment. MongoDB had gotten ahead of itself in the early 2010’s. There was a gap between our capabilities and the expectations of the market. And it was painful. Other databases had had more than 30 years to solidify their systems and operations. We’d had five. But with Version 3 we added a new storage engine, full ACID transactions, and search. We built on it with Version 4. And then again with Version 5, released this week at our .Live conference. I knew about all this progress intellectually of course when I joined, but not viscerally. I came to realize that the security, durability, availability, scalability, and operability our platform offers (of course in addition to all the features that developers love too) was ideal for architecting fast-moving enterprise applications. And I found the proof in our customer list. It reads like a Who’s Who of major global banks, retailers, and telecommunications companies, running core systems like payments, IoT applications, content management, and real-time analytics. They use our database, data lake, analytics, search, and mobile products across their entire businesses, in every major cloud, on-premises, and on their laptops. And that leads me to my fourth and final revelation. MongoDB is no longer just a database. Of course, the database is still the core. But MongoDB now provides an enterprise-class, mission-critical application data platform. A cohesive, integrated suite of offerings capable of managing modern data requirements across even the most sprawling digital estates, and scaling to meet the level of any company’s ambition, without sacrificing speed or security. Since the day I was first introduced to MongoDB’s products, I’ve had tremendous respect and admiration for the teams and their work. After all, I’m a developer, first and foremost. And it always felt like they “got” me. But had I known then what I know now, I would have jumped on this train a long time ago. In fact, I might have camped out on their doorstep with my resume in hand. And who knows? Maybe a bunch of people reading this will do just that, and have their own revelations about how fulfilling and exciting it can be to be at a great company, with a great culture, producing great products. I’ll write another letter a year from now, and let you know how it’s going then. In the meantime, please reach out to me here, or at @MarkLovesTech .
Launched Today: MongoDB 5.0, Serverless Atlas, and the Evolution of our Application Data Platform
Today we welcome you to our annual MongoDB .Live developer conference. Through our keynote and conference sessions we'll show you all the improvements, new features, and exciting things we've been working on since last year’s conference. What I want to do in this blog post is provide you with a summary of what we are announcing, and resources to help you learn more. While it's easy to focus on what we are announcing at this year's event, we actually started out on this journey 12 years ago by releasing the world’s most intuitive and productive database technology to develop with — MongoDB. And we believe the applications of the NEXT 10 YEARS will be built on data architectures that continue to optimize for the developer experience, allowing teams like yours to innovate at speed and scale. So how are we building on this vision? Today I am incredibly proud to announce three big things: The General Availability (GA) of MongoDB 5.0, the latest generation of our core database. It includes native support for time series workloads, new ways to future-proof your applications, multi-cloud privacy controls, along with a host of other improvements and new features. The preview release of serverless instances on MongoDB Atlas, which makes it even easier for development teams who don’t want to think about capacity management at all to get the database resources they need quickly and efficiently. Major enhancements to Atlas Data Lake, Atlas Search, and Realm Sync, which allow engineering teams to reduce architectural complexity and get more value out of their data by leveraging a unified application data platform. MongoDB 5.0 GA MongoDB 5.0 is the latest generation of the database most wanted by developers . Our new release makes it even easier to support a broader range of workloads, introduces new ways of future-proofing your apps, and further enhances privacy and security. This major jump in version number from MongoDB 4.4 – our prior GA version – to 5.0 reflects a new era for MongoDB's release cadence: We want to get new features and improvements into your hands faster. Starting with MongoDB 5.0, we will be publishing new Rapid Releases every quarter, which will roll up into Major Releases once a year for those of you that want to maintain the existing annual upgrade cadence. You can learn more about the new MongoDB release cadence from our blog post published last October. Digging into MongoDB 5.0, here is what’s new and improved: Native Time Series Designed for IoT and financial analytics, our new time series collections, clustered indexing, and window functions make it easier, faster, and lower cost to build and run time series applications, and to enrich your enterprise data with time series measurements. MongoDB automatically optimizes your schema for high storage efficiency, low latency queries, and real-time analytics against temporal data. Running your time series applications on our application data platform eliminates the time and the complexity of having to stitch together multiple technologies yourself. You can manage the entire time series data lifecycle in MongoDB – from ingestion, storage, querying, real-time analysis, and visualization through to online archiving or automatic expiration as data ages. Time series collections can sit right alongside regular collections in your MongoDB database, making it really easy to combine time series data with your enterprise data within a single versatile, flexible database – using a single query API to power almost any class of workload. Our new time-series collections blog post gives you everything you need to get started. Future-proof with the Versioned API and Live Resharding Starting with MongoDB 5.0, the Versioned API future-proofs your applications. You can fearlessly upgrade to the latest MongoDB releases without the risk of introducing backward-breaking changes that require application-side rework. Using the new versioned API decouples your app lifecycle from the database lifecycle, so you only need to update your application when you want to introduce new functionality, not when you upgrade the database. Future-proofing doesn’t end with the Versioned API. MongoDB 5.0 also introduces Live Resharding which allows you to easily change the shard key for your collections on demand – with no database downtime – as your workload grows and evolves. The way I like to think about this is that we’ve extended the flexibility the document model has always given you down to how you distribute your data. So as things change, MongoDB adapts without expensive schema or sharding migrations. Next-Gen Privacy & Security MongoDB’s unique Client-Side Field Level Encryption now extends some of the strongest data privacy controls available anywhere to multi-cloud databases. And with the ability in 5.0 to reconfigure your audit log filters and rotate x509 certificates without downtime you maintain a strict security posture with no interruption to your applications. Run MongoDB 5.0 Anywhere MongoDB 5.0 is available today as a fully-managed service in Atlas . You can of course also download and run MongoDB 5.0 on your own infrastructure, either with the community edition of MongoDB, or with MongoDB Enterprise Advanced . The Enterprise Advanced offering provides sophisticated operational tooling via Ops Manager, advanced security controls, proactive 24x7 support, and more. MongoDB Ops Manager 5.0 enhancements include: Support for the automation, monitoring, and backup/restore of MongoDB 5.0 deployments. Improved load performance with parallelized client-side restores. A quick start experience for deploying MongoDB in Kubernetes with Ops Manager. And lastly, a guided Atlas migration experience that walks users through provisioning a migration host to push data from their existing environment into the fully managed Atlas cloud service. You can learn more about MongoDB 5.0 from our What’s New guide . New to MongoDB Atlas — Serverless Instances (Preview) We want developers to be able to build MongoDB applications without having to think about database infrastructure or capacity management. With serverless instances on MongoDB Atlas, now available in Preview, you can automatically get the database resources you need based on your workload demand. It’s really simple: the only decision you need to make is the cloud region hosting your data. After that, you’ll get an on-demand database endpoint that dynamically adapts to your application traffic. Serverless instances will support the latest MongoDB 5.0 GA release, Versioned API, and upcoming Rapid Releases so you never have to worry about backwards compatibility or upgrades. Pay only for reads and writes your application performs and the storage resources you use (up to 1TB of storage in preview) and leave capacity management to MongoDB Atlas’s best-in-class automation. We invite you to try it out today with a new or existing Atlas account. And the Preview release is just the beginning – we will be working with partners such as Vercel and Netlify to deliver an integrated serverless development experience in the coming months. In the longer term, we will continue to evolve our cloud-native backend architecture to abstract and automate even more infrastructure decisions and optimizations to deliver the best database experience on the market. The New MongoDB Shell GA The new MongoDB Shell has been redesigned from the ground up to provide a modern command-line experience with enhanced usability features and powerful scripting environment. It makes it even easier for users to interact and manage their MongoDB data platform, from running simple queries to scripting admin operations. A great user experience, even on a command-line tool, should always be a major consideration. With the new MongoDB Shell we have introduced syntax highlighting, intelligent auto-complete, contextual help and useful error messages creating an intuitive, interactive experience for MongoDB users. Check out this blog post for more information. MongoDB Charts and Atlas Data Lake: Better Together MongoDB Charts intuitive UI and ability to quickly create & share charts and graphs of JSON data is now integrated with Atlas Data Lake . You can now easily visualize JSON data stored in Amazon AWS S3 without any data movement, duplication or transformation. Furthermore, you can run Atlas Data Lake’s federated query to blend data across multiple Atlas databases and AWS S3, and visualize the results with Charts. By adding Atlas Data Lake as a data source in Charts, you can discover deeper, more meaningful insights in real time. Check out this blog post for more information. Atlas Search — More Relevance Features It’s incredibly important for modern applications to deliver fast and relevant search functionality: it powers discoverability and personalization of content, which in turn drives user engagement and retention. Atlas Search , which delivers powerful full-text search functionality without the need for a separate search engine, has several new capabilities for building rich end user experiences. We’ve recently added support for function scoring, which allows teams to apply mathematical formulas on fields within documents to influence their relevance, such as popularity or distance — e.g. closer restaurants with more or better reviews will show up higher in a list of results. In addition, you can now define collections of synonyms for a particular search index. By associating semantically equivalent terms with each other, you can respond to a wider range of user-initiated queries in your applications. Realm Realm lets you have simple, powerful local persistence on mobile phones, tablets and IoT devices like Raspberry Pi. The Realm SDKs provide a set of APIs that let developers store and interact with native objects directly, reducing the amount of code required as there is no need for ORMs or learning cryptic database syntax. In addition, we made MongoDB Realm Sync generally available earlier this year, making it easy to synchronize data between local storage on your devices and MongoDB Atlas on the backend. No need to worry about networking code or dealing with conflict resolution as we handle all of that for you. Today, we’re excited to announce support for Unity. You can now use Realm to store your game data, like scores and player state, and sync it automatically across devices. Realm's support for Unity is now Generally Available and ready for production workloads. We're also investing in support for more cross-platform frameworks — the Kotlin Multiplatform and Flutter/Dart SDKs are now both available in Alpha. And finally, the team is working towards Realm Flexible Sync, a new way to synchronize data with more granular control. Flexible Sync will allow you to — Build applications that respond dynamically to user's needs. Let your end users decide what data they need, and when. Use more precise permissions that can adapt over time. Check out this dedicated blog on our upcoming plans for Flexible Sync to learn more. Getting Started With everything we announced today, you can imagine it was a packed keynote! And there is so much more that we didn’t cover. You can get all of the highlights from our new announcements page where you will also find all the resources you need to get started.
The Foundations of IT Modernization
Lately, I’ve been thinking a lot about the term “modernization.” It’s a broad term that means different things to different people. To some, modernization means migrating legacy systems to the cloud. To others, it means rewriting applications, containerizing, or embracing microservices architecture. And to others still, modernization is synonymous with an equally amorphous (and ubiquitous) term: digital transformation. However you define it, modernization is all the rage right now. IDC says these investments are growing at a compound annual growth rate of 15.5%, and will reach $6.8 trillion by 2023 . (yeah, trillion, with a ‘t’). This frenzy of spending on technology, services, and skills is intended to bring aging systems and business processes up to date. In many cases, these investments are urgent and necessary, as companies of all shapes and sizes, in every industry, must accelerate their pace of innovation in order to survive. But the work of modernization is complex, costly, and technically challenging. It’s like renovating every room in a sprawling estate, while you’re still living in it. It’s hard to even know where to start. To that end, I can’t help but think about the words of the CIO of a $30 billion insurance company who had already been on a modernization journey for years. He said: “We tried everything to accelerate innovation...but, in the end, it was our data platform that was holding us back.” In other words, they were spending millions to fix up their estate, adding radiant heat, smart speakers, and a state-of-the-art home theater. But they were building on top of foundations that were first poured when disco was new. (I’m looking at you, relational databases, first conceived and implemented in the 1970s). In the digital economy, companies succeed or fail based on how fast they innovate. More often than not, that innovation takes the form of software and services, which in turn create value by storing, manipulating, and querying data. And what do you use to store, manipulate, and query all that data? Your application data platform. Years ago, that just meant ‘database with some scripts around it.’ Those days are gone. Now, an application data platform has to supply speed, governance, security, availability, and more. So let’s get back my modernization metaphor. You can’t build new solid things on top of creaky, unstable, old things. We all know the old things I’m talking about; databases that make you structure your data in a way that isn’t natural, languages written to be so precise to the computer that they are inscrutable to developers, ‘roach motel’ storage systems that don’t store things in modern, open formats. So if you want to modernize your infrastructure, or modernize your applications, or modernize the way you build software, shouldn’t you first modernize your data platform? Sure, it’s hard to renovate your house. But this is where you live. And if you want that house to last, make sure it’s built on solid foundations. What does that mean? It means that it’s not enough to design and build the right apps. If you want to be truly modern, look at how you input and output your data, how you query, manipulate, and store it, and how you program against it. Get those things right, and you dramatically increase your pace of innovation. No matter where you are in your own modernization journey, it’s not too late to do this. Don’t believe it? Hit me back on Twitter at @MarkLovesTech and I’ll show you how.
The Difference Between R and D
I used to believe that Research and Development (R&D) departments should work in lockstep with Product teams so that they can stay focused on commercially-viable innovations. After all, not every innovation has a market, and not every business has resources to bet on future markets. All of that changed when I met Dr. Michael Cahill, the head of MongoDB Labs in Australia. Michael came to MongoDB through our acquisition of WiredTiger back in 2014, an open source storage engine company he co-founded with Keith Bostic. He holds a PhD in Computer Science and a history of breakthrough innovation. He also has an enlightened point of view on the role of research in any technology company. “Researchers need time and space to pursue the theoretical,” he told me. “We want them to come up with crazy ideas, with much longer time horizons.” Michael is referring to the fundamentally different mindsets required of researchers versus developers. Our developers are focused on new products or features that can make an impact in the next 3-4 quarters. Our researchers are thinking about solving problems that have the potential to reshape entire markets for decades. Big difference. Funding this kind of innovation is challenging for the MBA set, and measuring the ROI of basic research is notoriously difficult. Progress can seem slow and difficult to quantify. Our researchers occupy a space that straddles art and science, industry and academia. They spend a lot of time reading, thinking, and tinkering. Ideas are freely shared, cultivated, iterated, and sometimes abandoned. This is the price of disruptive innovation. In fact, MongoDB would never exist if our founders had set out to simply improve upon relational databases. Instead, they wanted to invent an entirely new way to manage data. It was an ambitious idea that required long-term thinking. An idea that despite MongoDB’s current success, is still only in its infancy. An idea so humongous, Michael Cahill may have even called it “crazy.” Don’t get me wrong. The work of MongoDB Labs is firmly grounded in MongoDB’s core strategy: to constantly improve the way data is stored, accessed, secured and processed. Document databases are only the first act of this play. And I’m certain the next act is being written as we speak, by Michael and his team. Have a different approach to R&D? Think my ideas are “crazy”? Let me hear about it on Twitter at @MarkLovesTech
The Innovation Tax: How much are unproductive and unhappy developers costing you?
I am not someone who believes that developers should be coddled. And I don’t subscribe to a culture of entitlement for developers, or any other part of an organization for that matter, including the C-suite. We are all professional adults operating in the real world. We should treat each other like grownups, regardless of role or responsibility. From the coder to the financial analyst to the sales rep, we all bring our unique value to the company. So execs like me need to strive to understand, appreciate, and foster the critical skills every team member brings to the table. Let’s start with developers, one of my favorite cohorts. We’ve all heard the now overused adage of the digital age: “Every company is becoming a software company.” What this trope is trying to convey is that innovation in the digital space - application development - is a major force in driving new business creation and competitive advantage. The speed with which a new application can be deployed, coupled with the quantity of innovative features in it, is a direct lever on the success of a business. If applications are the currency of the new economy, then development teams are the market makers. In my experience, however, despite the relentless strategic emphasis on speed and innovation in the digital economy, these teams continue to be misunderstood, mismanaged, and marginalized inside both large and small companies. It’s not rational. Worse, it’s incredibly costly. I think about this as a tax on the amount of innovation that a company can produce. Companies pay this tax when they fail to understand the nature of the work developers do, or provide a safe and productive environment for them to do it. And if you don’t get that right, you’re not going to be in this game for very long. Though I don’t write any production code these days, at heart, I’m still a developer. And MongoDB leads hundreds of developers spread across tens of teams, so I’m constantly exposed to developer issues. Over the course of my career, I’ve learned a few things about how – and how not – to cultivate a productive culture for developers. This will be an ongoing discussion, for sure. But to get things started, here are a few things to think about if you’re trying to reduce the “Innovation Tax” your currently paying: Give your developers business context Don’t insult the intelligence or maturity of your developers. They can – and must – understand the business rationale for their work. In fact, painting the strategic target for developers will result in a better work product as they align their key decisions in the architecture and design experience of your software. Once they understand the business context, they’ll find better ways of achieving it bottoms-up than any tops-down leader, even a CTO like me, possibly can. Respect tech debt — and pay down the principal In my experience, the single biggest source of low morale among developers is the combination of too much tech debt and management’s dismissal of it. Taking on some debt to get a release out is fine - if you do it knowingly and pay down the principal later. But leaders who don’t pay attention to mounting debt demonstrate in a very visceral way to developers that they’ve become Gantt-chart leaders, and lost touch with their ethos of engineering. Developers don’t do well with cognitive dissonance, so when you tell them to build the next great thing on top of a dumpster fire, you lose credibility, they lose patience, and your company loses money as the pace of innovation slows to a crawl. Understand what your developers are really doing I could talk about this one for days, but the bottom line is that if leaders don’t understand how developers spend their time, they have no business leading the teams. It’s easy to just focus on new features, but you must acknowledge and address the fact that adjacent work like maintaining databases or a legacy staging environment is pure drudgery that provides no innovation value, costs a fortune in developer time, and saps morale. Listen to the developers when they say that they need to revamp an adjacent or dependent system to understand why it’s important. Remove OKRs and vanity metrics Top-down innovation is an oxymoron. You have to trust that developers want nothing more than to see their work come to life. The more management tells them how to do their job – through objectives and key results or any other key performance indicators – the more they limit the scope of innovation. Paint the target, then get out of the way. Align your goals This goes back to providing business context. Leaders and developers need to believe they are working together toward the same goal. An oppositional relationship takes developers out of flow, and you can lose a whole day of productivity from a single negative interaction. Again, I’m not advocating coddling; developers have their part to play in the complex recipe that builds a successful company, just like everybody else. But for that to work, you must align business, technical, and organizational goals, and build honest and transparent relationships with your devs. Like I said, I could riff on this topic for many more days (or posts). And keep in mind, mismanaging developers is just one form of innovation tax. I’ll be exploring other hidden levies in this space over the coming months. But hopefully this starter list of dos and don’ts gets the conversation going. Please feel free to add to it (or subtract from it) on Twitter at @MarkLovesTech .
혁신세: 생산성이 낮고 사기가 떨어진 개발자가 유발하는 비용
저는 개발자들의 응석을 다 받아줘야 한다고 생각하지 않습니다. 개발자든, 최고 경영 책임자든 누군가에게 특권을 부여하는 문화에 동조하지도 않습니다. 우리 모두는 현실을 살아가는 성인이고 전문가입니다. 직책에 관계없이 서로를 동등한 성인으로 대해야 합니다. 컴퓨터 프로그래머부터 금융 분석가, 영업 담당자에 이르는 모두가 회사에 고유한 가치를 제공하고 있습니다. 저 같은 경영자는 모든 팀원들이 보유하고 있는 중요한 역량들을 이해하고, 인정하고, 후원하려고 노력해야 합니다. 개발자부터 시작해보겠습니다. 제가 가장 아끼는 구성원 중 하나죠. 디지털 시대로 접어들면서 귀에 못이 박히게 듣고 있는 말이 있습니다. “모든 회사가 소프트웨어 회사로 변모하고 있다"는 말입니다. 디지털 분야, 특히 애플리케이션 개발에서의 혁신이 새로운 비즈니스 창출 및 경쟁 우위에 중요한 역할을 한다는 것을 강조하기 위한 표현이라 하겠습니다. 새로운 애플리케이션을 신속하게 배포하고 다양한 혁신 기술을 함께 제공할 수 있는 능력이 비즈니스 성공의 직접적인 동인으로 자리 잡았습니다. 애플리케이션이 새로운 경제의 통화라면 개발 팀은 시장 조성자입니다. 하지만 제 경험에 따르면 디지털 경제에서 속도와 혁신을 부단히 그리고 전략적으로 강조하고 있음에도 불구하고, 개발 팀은 오해를 받고 제대로 관리되지 않아 방치되기 일쑤입니다. 이러한 현상은 기업의 규모와는 관계가 없습니다. 뭔가 앞뒤가 맞지 않습니다. 게다가 엄청난 비용이 발생하고 있습니다. 저는 이것을 혁신하는 회사가 대가로 치르는 세금이라고 생각합니다. 개발자가 수행하는 업무의 속성을 이해하지 못하거나 개발자에게 안전하고 생산적인 환경을 제공하지 못하는 기업은 이 세금을 치르게 됩니다. 그리고 이 문제를 제대로 해결하지 못하면 장기적으로 시장에서 살아남을 수 없습니다. 요즘에는 프로덕션 코드를 작성하고 있지 않지만, 그럼에도 불구하고 저는 여전히 개발자입니다. MongoDB는 수십 개 팀에 소속된 수백 명의 개발자들을 이끌고 있기 때문에 저는 항상 개발자 문제에 부딪히게 됩니다. 개발자를 위해 생산적인 문화를 조성하려면 해야 할 일과 하지 말아야 할 일이 몇 가지 있습니다. 이 업계에서 경력을 쌓으면서 터득한 것이죠. 물론 앞으로도 계속해서 논의가 필요한 문제겠지만, 일단 소개해보겠습니다. 현재 지불하고 있는 "혁신세"를 줄이고 싶다면 꼭 생각해 봐야 할 몇 가지가 있습니다. 개발자에게 비즈니스 맥락을 제공하라 개발자의 지능이나 성숙도를 우습게 보지 마십시오. 개발자는 자신이 일하는 이유와 근거를 이해할 수 있고 또 이해해야 합니다. 실제로 개발자가 전략적 목표를 그려놓으면 소프트웨어의 아키텍처 및 설계 경험과 조율해 중요한 의사 결정을 내리게 되면서 더 나은 결과물이 나옵니다. 비즈니스 맥락을 이해하는 개발자는 최고 책임자, 심지어 저 같은 CTO 보다 훨씬 현명하게 상향식 관점에서 성과를 내는 방법을 찾을 수 있습니다. '기술 부채'를 인정하고 원금을 갚아라 제 경험으로 볼 때, 개발자들의 사기를 떨어뜨리는 가장 큰 원인은 과도한 기술 부채와 이를 해결하려는 경영진의 의지 부족입니다. 탈출구를 마련하기 위해 약간의 부채를 지는 것도 나쁘지는 않습니다. 부채라는 것을 알고 나중에 원금을 갚기만 한다면 말입니다. 그러나 개발자들은 리더가 기술 부채 증가에 관심을 기울이지 않고 있다는 사실을 본능적으로 눈치 챕니다. 기술 부채 증가에 관심을 기울이지 않는 리더들은 프로젝트 일정에만 집착하고 엔지니어링 정신을 잃어가기 때문입니다. 그러면 개발자들은 인지 부조화에 빠지게 됩니다. 상황은 재앙에 가까운데도 개발자에게 차기 혁신을 내놓으라고 요구하는 리더는 신뢰를 잃게 되고, 개발자들은 인내심을 잃게 됩니다. 회사는 혁신 속도가 지체되면서 비용 손실을 입게 됩니다. 개발자가 실제로 어떤 일을 하는지 파악하라 여기에 대해서는 며칠이고 얘기할 수 있을 정도로 할 말이 많지만, 결론은 이렇습니다. 개발자가 업무 시간을 어떻게 쓰고 있는지 이해하지 못하는 리더는 팀을 이끌 자격이 없습니다. 새로운 기능에 초점을 맞추기는 쉽습니다. 하지만 리더라면 개발에 관련된 부수적인 작업(데이터베이스나 레거시 스테이징 환경의 유지관리 등)이 지루하고 고된 단순 반복작업이라는 점을 인정하고 문제를 해결하기 위해 애써야 합니다. 그러지 않으면 혁신 가치도 제공하지 않으면서 개발자의 시간만 많이 잡아먹게 되어 개발자의 사기만 떨어지고 말 것입니다. 개발과 관련된 부수적인 시스템을 쇄신해야 한다는 개발자의 목소리에 귀를 기울여 그 중요성을 이해해야 합니다. OKR과 허영 지표를 제거하라 하향식 혁신이라는 말에는 모순이 있습니다. 개발자가 원하는 것은 오직 자신이 개발한 기술을 실현하는 것이라는 사실을 믿어야 합니다. 목표, 주요 성과 또는 기타 KPI를 통해 개발자를 압박할수록 개발자는 스스로 혁신의 범위를 제한하게 됩니다. 목표를 설정했다면, 개입하지 말고 빠져줘야 합니다. 목표를 조정하라 이는 비즈니스 맥락을 제공하라는 말과 같습니다. 리더와 개발자는 같은 목표를 달성하기 위해 협력하고 있다는 사실을 믿어야 합니다. 대립 관계가 되면 개발자의 업무에 차질이 생기고, 리더는 단 한 번의 마찰로 인해 하루라는 귀중한 시간을 낭비하게 됩니다. 다시 말하지만, 응석을 받아주라는 말이 아닙니다. 개발자는 다른 모든 직원들과 마찬가지로 회사의 성공을 위해 맡은 바 역할을 수행해야 합니다. 제 말은 비즈니스 목표와 기술 목표, 조직 목표를 조율해서 개발자들과 솔직하면서도 투명한 관계를 구축해야 한다는 것입니다. 앞서 말했듯이 이 주제에 대해서는 며칠이고 말할 수 있을 정도로 할 말이 많습니다. 개발자에 대한 관리 미흡은 혁신세의 한 형태라는 것을 기억하시기 바랍니다. 앞으로 몇 달에 걸쳐 여기에 숨겨진 다른 세금은 없는지 살펴볼 것입니다. 지금까지 초보 리더들이 해야 할 일과 하지 말아야 할 일에 대해 알아봤습니다. 이에 대한 논의가 앞으로도 계속되었으면 합니다. 덧붙이고 싶거나 빼야 한다고 생각하는 내용이 있으시면 언제든 자유롭게 Twitter( @MarkLovesTech )에 올려주시기 바랍니다.
The Evolution of Data
It used to be so simple. Not that long ago, the universe of corporate data was a fraction of its current size. We fit most of it into rigid little rows and columns. And we didn’t ask all that much of it: some transaction processing, a few charts and graphs, a little business intelligence. I’m exaggerating, of course. We’ve been pushing the boundaries of data processing since at least 1964, when SABRE, the world’s first airline passenger system, launched on two IBM mainframes and 1,500 terminals, processing an average of one transaction per second. But it’s not hyperbole to say that today’s data would barely recognize its early ancestors. First of all, there is more of it – 59 zettabytes and counting. Second, the very definition of data has changed dramatically, expanding well beyond payroll records and stock prices to include diverse data types like weblogs, fraud scores, maps, and fingerprints. But perhaps the biggest change has been the role data plays in the enterprise. Data has always been used to inform business strategy. But today, data often is the business strategy. Consider this: 20 years ago, there was no such thing as a Chief Data Officer. Today? Almost two-thirds of Fortune 1000 companies have one. Why? Because we are asking more of our data than ever before. And rightly so. In the digital economy, every company competes on the basis of insight-driven innovation. More and more, those innovations take the form of software built around clever algorithms. And the raw material to both craft and execute those algorithms is data. All of which means the ability to efficiently and effectively manage data -- at speed -- is a strategic imperative for any company, in any industry. And yet, even as the volume, variety, and strategic importance of data has rapidly evolved over the last two decades, many enterprises haven’t changed how they manage it. Of course, I’m talking about the continued use of legacy relational databases, which are too rigid and don’t scale well enough to handle the demands of modern application development. Solving this problem was the entire reason for the “NoSQL” movement in the late 2000s, and MongoDB’s invention of the document-oriented database in the first place. But I’m talking about something bigger; a longer-term trend that demands a fresh look at the way we work with data. Our customers are telling us that the fundamental requirements of their various data sets aren’t just changing, they’re converging. This is a surprise, and a reversal of the trend of siloization and specialized tools for the last 50 years. Let’s take a step back: For decades, enterprises have maintained systems of record and systems of engagement. Systems of record are foundational, mission-critical, sources of truth that are accessed primarily by internal programs and users. Systems of engagement are the digital interfaces with which customers and employees interact. And recently we have seen the addition of systems of insight, which combine data from various sources to inform decision making across the enterprise. For a long time, each system lived on different computers, had different data management requirements, and were funded by different departments. But that is changing. With the hard and fast divisions between back office and front office dissolving, we now need all of our data systems to do everything. They need to be both fast and accurate. They need to be both accessible and secure. They need to handle both transactions and analytics. In particular, with the rise of model training and inference, a different kind of analytics is arriving; one where it is programs that are asking the systems of insight questions and reacting to them in real time, rather than humans asking questions and then writing programs to implement them. This is a fundamental shift; so fundamental that you could liken it to the change from the IBM 7090s that powered SABRE to those that (will?) power SKYNET. This “convergence” of data requirements is both a challenge and an opportunity. Just like document databases enabled us to rethink how data was accessed and stored, convergence is forcing us to rethink the systems we use to manage data across the enterprise yet again. Companies across the industry, from Snowflake to Databricks to MongoDB, and every cloud provider, are working to provide the systems that let companies get more value from their data, using microservices-based networks or programs that drive informed, real-time decision making. Interestingly, this comes at a time when most companies are undergoing radical digital transformation projects in order to become innovation-powered, software-driven, and cloud-based. In other words, even though everyone is already quite busy, there has never been a better time to think beyond the database, and architect an actual “data platform” that can process, store, secure, and analyze data in real-time, across all the relevant data sets - either without copying the data or making such copying invisible. Over the coming months, I’ll be sharing more about what these data platforms look like and how they support the creation of modern applications. I’ll also try to peer into the foggy future with you, looking at the constraints and freedoms of the modern enterprise, and predicting what products will be built to address that matrix. But for now, I’m interested in hearing what you’re seeing under the hood of your data(base) estate. Are you experiencing a similar convergence of data requirements? If so, is it factored into your digital transformation strategy? And if not, what changes are you seeing in the way you use data across the enterprise? Please reach out to me here or on Twitter, at @MarkLovesTech - because I really do. Thanks for reading - may the data odds be in your favor until we chat again. Mark
과거에는 매우 단순했습니다. 그 때는 기업 데이터라는 분야가 현재 규모의 일부에 지나지 않았습니다. 지금과 같은 규모가 된 것은 얼마 되지 않은 일입니다. 우리는 대부분의 데이터를 정해진 작은 열과 행에 집어 넣습니다. 예전에는 몇몇 트랜잭션 처리, 몇 가지 차트와 그래프, 약간의 비즈니스 인텔리전스로도 아무런 문제가 되지 않았습니다. 물론 다소 과장한 감이 없지 않아 있습니다. 데이터 처리의 한계를 뛰어 넘기 시작한 것은 1964년입니다. 그 해에 세계 최초의 항공사 승객 시스템인 SABRE가 2개의 IBM 메인프레임을 토대로 1,500개의 터미널에서 출범하면서 1초당 평균 1건의 트랜잭션을 처리할 수 있게 되었습니다. 그러나 오늘날 데이터는 초창기 데이터와 완전히 달라졌다고 해도 과언이 아닙니다. 첫째, 크기가 59 제타바이트에 달할 정도로 커졌습니다. 둘째, 데이터에 대한 정의가 완전히 바뀌었습니다. 급여 기록 및 주가를 뛰어 넘어 웹 로그, 위험 평점, 맵, 지문 등 다양한 유형의 데이터가 포함되었습니다. 하지만 아마도 가장 큰 변화는 기업에서 데이터가 맡고 있는 역할일 것입니다. 데이터는 항상 비즈니스 전략을 알리는 데 사용되어 왔습니다. 하지만 오늘날에는 데이터가 곧 비즈니스 전략인 경우가 종종 있습니다. 이렇게 생각해보십시오. 20년 전만 해도 CDO(Chief Data Officer)라는 직책 자체가 없었습니다. 오늘날은 어떻습니까? 포춘지 선정 1000대 기업의 2/3 정도가 CDO를 두고 있습니다. 그 이유는 무엇일까요? 그 어느 때보다 데이터에 대해 요구하는 것이 많아졌기 때문입니다. 당연한 얘기입니다. 디지털 경제에서는 모든 기업이 인사이트 기반의 혁신을 토대로 경쟁합니다. 이러한 혁신 덕분에 점차 더 영리해진 알고리즘을 중심으로 소프트웨어가 개발되고 있습니다. 이러한 알고리즘을 개발하고 실행하는 원 재료가 되는 것이 바로 데이터입니다. 데이터를 효율적이고 효과적이며 신속하게 관리할 수 있는 능력이 모든 산업 분야, 모든 기업의 전략 과제가 되었습니다. 지난 20년 동안 데이터의 양과 다양성, 전략적 중요성에 있어 급격한 변화가 있었지만 많은 기업들이 데이터 관리 방법을 바꾸지 않았습니다. 물론, 기존의 관계형 데이터베이스는 유연성이 부족하고 확장이 불편하기 때문에 오늘날 애플리케이션 개발 요구를 처리하기에는 적합하지 않습니다. 그럼에도 불구하고 계속 사용되고 있습니다. 이러한 문제를 해결하려는 노력의 일환으로 2000년대 말 "NoSQL" 운동이 전개되었습니다. 그리고 MongoDB가 최초로 문서 지향 데이터베이스를 발명하게 되었습니다. 그러나 제가 말씀드리고 있는 것은 더 광범위한 것입니다. 즉, 데이터 활용 방법에 대한 새로운 시각을 요구하는 더 장기적인 추세에 관한 것입니다. 다양한 데이터 세트의 기본적인 요구사항이 그냥 변화하는 것이 아니라 수렴되고 있다고 고객들은 말합니다. 이는 놀라운 변화입니다. 지난 50년 동안 줄곧 사일로화되고 전문화된 도구라는 추세가 뒤집히고 있습니다. 한 발 물러서 생각해봅시다. 수십 년 동안 기업들은 레코드 시스템과 인게이지먼트 시스템을 유지해왔습니다. 레코드 시스템은 기본적이고 미션 크리티컬한 진실 공급원으로, 내부 프로그램과 사용자가 주로 액세스합니다. 인게이지먼트 시스템은 고객과 직원이 상호 작용할 때 사용하는 디지털 인터페이스입니다. 최근에는 다양한 소스에서 나온 데이터를 통합하여 전사적 차원에서 의사 결정에 필요한 정보를 제공하는 인사이트 시스템이 추가되었습니다. 오랜 세월 각 시스템은 상주하는 컴퓨터가 서로 달랐고, 데이터 관리 요구사항도 서로 달랐으며, 자금을 대는 부서도 서로 달랐습니다. 하지만 이러한 추세가 변화하고 있습니다. 엄격하게 분리되었던 백 오피스와 프론트 오피스 간의 경계가 허물어지면서 모든 데이터 시스템이 하나가 되어 비즈니스를 수행하는 시대가 되었습니다. 신속하면서도 정확해야 하고, 접근이 쉬우면서도 안전해야 합니다. 또한 트랜잭션과 분석을 모두 처리할 수 있어야 합니다. 특히, 모델 훈련 및 추론이 새롭게 떠오르면서 다양한 유형의 분석이 등장하고 있습니다. 사람이 질문을 하고 모델을 구현할 프로그램을 작성하는 것이 아니라, 프로그램이 시스템에 인사이트 관련 질문을 하고 실시간으로 이에 대응하는 방식으로 바뀌고 있습니다. 이는 그야말로 근본적인 변화입니다. 마치 SABRE의 기반이 IBM 7090에서 SKYNET으로 바뀌는 것과 같습니다. 이러한 데이터 요구사항의 “수렴"은 도전 과제이자 기회입니다. 문서 데이터베이스 덕분에 데이터의 액세스 및 저장 방법을 재고할 수 있었던 것처럼, 이러한 수렴 현상 덕분에 데이터를 전사적으로 관리하는 데 사용하는 시스템을 다시 한 번 재고해야 하는 상황이 되었습니다. Snowflake부터 Databricks, MongoDB에 이르는 업계의 모든 기업들과 모든 클라우드 제공업체들은 정보에 입각한 실시간 의사 결정을 뒷받침하는 마이크로서비스 기반의 네트워크 또는 프로그램을 사용해 데이터로부터 더 많은 가치를 창출할 수 있는 시스템을 제공하기 위해 노력하고 있습니다. 재미있는 것은 혁신을 추구하는 소프트웨어 중심의 클라우드 기반 기업이 되기 위해 대부분 기업들이 획기적인 디지털 전환 프로젝트를 진행하고 있는 시점에 이러한 추세가 등장했다는 사실입니다. 다시 말해 모두가 이미 발 빠르게 움직이고 있습니다만, 지금이야말로 데이터베이스를 재고해 보기에 가장 적합한 때입니다. 데이터를 복사하거나 이러한 복사 작업을 눈에 보이지 않게 수행하지 않고 모든 관련 데이터 세트를 토대로 데이터를 실시간으로 처리, 저장, 보호 및 분석할 수 있는 실질적인 "데이터 플랫폼"을 설계할 수 있는 절호의 기회입니다. 저는 앞으로 몇 달에 걸쳐 이러한 데이터 플랫폼이 어떻게 설계되어 있고, 어떻게 최신 애플리케이션 개발을 지원하는지에 대해 자세히 알려드릴 계획입니다. 또한 오늘날 기업들이 할 수 있는 것과 할 수 없는 것을 살펴보고 현실의 문제를 해결하기 위해 어떤 제품들이 개발될 것인지 예측해 보면서 안개가 낀 듯 막연하기만 한 미래를 여러분과 함께 더듬어 가볼 생각입니다. 하지만 지금은 데이터베이스라는 기업 자산에서 어떤 일이 벌어지고 있는지 들어보고 싶습니다. 여러분도 이와 비슷한 데이터 요구사항의 수렴을 경험하고 계십니까? 그렇다면 디지털 전환 전략에 이러한 트렌드를 반영하고 계십니까? 아니라면 전사적 차원에서 데이터를 사용하는 방법에 있어 어떤 변화가 나타나고 있습니까? 제게 연락하고 싶으신 분들은 이 블로그나 Twitter( @MarkLovesTech )를 이용해 주시기 바랍니다. 읽어주셔서 감사합니다. 다시 만날 때까지 데이터가 여러분의 편이 되어주기를 바랍니다. Mark