As you may have read previously on the blog, the MongoDB team is adopting the Go language for a variety of projects. At last month’s New York MongoDB User Group, Sam Helman presented on how MongoDB is using Go for new and existing cloud tools. In Sam’s talk, you’ll learn how MongoDB is using Go for the backup capabilities in MongoDB Management Service and a new continuous integration tool.
Why go with Go? Between the lightweight syntax, the first-class concurrency and the well documented, idiomatic libraries such as mgo, Go is a great choice for writing anything from small scripts to large distributed applications.
Thanks to g33ktalk for recording Sam’s talk.
Open Source Software in Business
Open Source software (OSS) has made significant strides in enterprises. From the New York Stock Exchange to the smallest mom-and-pop store, the Linux operating system is powering today's business servers. Its growing popularity is due in no small part to the fact that thousands of IT professionals around the world recognize that producing and distributing this type of open standard software has tremendous benefits over traditional commercial products. Unlike off-the-shelf software, open source gives users access to the underlying source code of the application that programmers write and how they instruct a computer to perform certain tasks. Commercial software is considered proprietary to the company that produces it. “It works well, saves money, and is more secure. SMB or large enterprise, there's no down-side, says Steven Vaughn-Nichols, a long-time Linux and open source journalist. “OSS tends to be more secure -- its code is open so anyone can find, report and fix the bugs. You have no idea what may or may not be hiding in proprietary code.” OSS also tends to be cheaper than its proprietary brothers, he adds. “Better security and cheaper to boot? What's not to like?” Previously, security and licensing acted as traditional barriers to adoption. Now, OSS is driving change from the bottom up, according to the seventh annual “Future of Open Source Survey,” of more than 800 respondents, sponsored by North Bridge Venture Partners and Black Duck Software. Survey results indicate OSS is experiencing a growing influence in organizations. This is due to a cultural shift supported by executives' openness to work with active and strong communities to influence projects and spur innovation, the study found. Additionally, “open source has reached a depth and maturity where quality, access to code and costs are no longer barriers to adoption. “This trend is reinforced by thousands of developers working to reduce defects in code, improve its security and innovate with new features and enhancements that get closer to what users want - because those users can have a hand in making it so,” the study found. The biggest factor in OSS adoption, respondents said, is quality, a ranking that increased from third place in 2012. Freedom from vendor lock-in dropped to second place this year, from first place in 2012. Lower costs, big data, and systems integration are the top three business problems open source is solving. In terms of sector growth, the study found that government, as well as health care and media/entertainment are moving toward adoption of open source. Technical capabilities and features were cited by 45 percent of respondents as reasons for why they choose OSS over proprietary solutions. Only 12 percent chose commercial vendor support as being an important factor, according to the study. Linux has taken over some verticals, notes Vaughn-Nichols. “Most stock markets’ core servers, for example, now run Linux. It really has been moving everywhere in other businesses.” As for the types of applications, he says it started with edge servers, file and Web servers, and over the years Linux is now being used for a large variety of apps, including CRM. Sears, for example, used open source software to build a private cloud platform. According to published reports, the company opted for OSS because it helped reduce costs and boost flexibility. And Sears and Chevron are using Hadoop, an open source database to analyze large amounts of data. Because the code is reusable, OSS is also being used by MasterCard to build prototypes of mobile apps. This gives developers the ability to create apps quickly, the company said. While the energy industry typically hasn’t shown an interest in OSS and has generally only used Linux application servers, Chevron is using Hadoop to find oil in areas including the Gulf of Mexico. As for smaller businesses, Vaughn-Nichols says many are probably not aware that their Network-Attached Storage (NAS) device and Internet router “are almost certainly running Linux.” The OS is everywhere, he adds. “It's in your pocket with your Android smartphone and it's what powers your Google Web searches. The only place it really isn't is the conventional desktop.” Better quality was the number one factor for open source adoption in business cited by respondents in the Future of Open Source survey. Other factors in order of importance were: freedom from vendor lock-in; flexibility/access to libraries of software, extensions and add-ons; elasticity/ability to scale at little cost or penalty; superior security; pace of innovation; lower costs; and access to source code. Increasingly, open source software is being chosen over proprietary alternatives. Industry observers say it’s a viable choice for businesses because it provides: Control – Unlike commercial software, OSS lets you make decisions about how to run your business. Flexibility – OSS is licensed in a way that lets you modify it yourself or hire a third party to tailor the software to meet the needs of your business. Reliability – OSS typically has fewer bugs and is more reliable than software developed using a standard commercial development process. Cost – There are little to no upfront costs with OSS. Users only pay for the support they need and most importantly, when they need it. Longevity – If a commercial software company goes out of business, you lose all of your support, bug fixes, security patches and possibility of future upgrades. Contrast that to a mission-critical software application a business is using: all it has to do is find a consulting firm, programmer or another third-party provider. “Enterprises see [open source software] as leading innovation, delivering higher quality and driving growth rather than being just a free or low-cost alternative,’’ says Michael Skok, general partner at North Bridge Venture Partners. “Going forward, as broader adoption creates a virtuous cycle of innovation and investment, we can expect more disruption from open source, new business models and many more exciting new projects and companies." Vaughn-Nichols also sees OSS becoming more ubiquitous in business. “It's becoming invisible. People tend to think of software as what they see in front of them. For most people that means Windows or Macs. Behind all of them everything -- and I mean everything -- has been switching over to OSS.”
How DataSwitch And MongoDB Atlas Can Help Modernize Your Legacy Workloads
Data modernization is here to stay, and DataSwitch and MongoDB are leading the way forward. Research strongly indicates that the future of the Database Management System (DBMS) market is in the cloud, and the ideal way to shift from an outdated, legacy DBMS to a modern, cloud-friendly data warehouse is through data modernization. There are a few key factors driving this shift. Increasingly, companies need to store and manage unstructured data in a cloud-enabled system, as opposed to a legacy DBMS which is only designed for structured data. Moreover, the amount of data generated by a business is increasing at a rate of 55% to 65% every year and the majority of it is unstructured. A modernized database that can improve data quality and availability provides tremendous benefits in performance, scalability, and cost optimization. It also provides a foundation for improving business value through informed decision-making. Additionally, cloud-enabled databases support greater agility so you can upgrade current applications and build new ones faster to meet customer demand. Gartner predicts that by 2022, 75% of all databases will be on the cloud – either by direct deployment or through data migration and modernization. But research shows that over 40% of migration projects fail. This is due to challenges such as: Inadequate knowledge of legacy applications and their data design Complexity of code and design from different legacy applications Lack of automation tools for transforming from legacy data processing to cloud-friendly data and processes It is essential to harness a strategic approach and choose the right partner for your data modernization journey. We’re here to help you do just that. Why MongoDB? MongoDB is built for modern application developers and for the cloud era. As a general purpose, document-based, distributed database, it facilitates high productivity and can handle huge volumes of data. The document database stores data in JSON-like documents and is built on a scale-out architecture that is optimal for any kind of developer who builds scalable applications through agile methodologies. Ultimately, MongoDB fosters business agility, scalability and innovation. Key MongoDB advantages include: Rich JSON Documents Powerful query language Multi-cloud data distribution Security of sensitive data Quick storage and retrieval of data Capacity for huge volumes of data and traffic Design supports greater developer productivity Extremely reliable for mission-critical workloads Architected for optimal performance and efficiency Key advantages of MongoDB Atlas , MongoDB’s hosted database as a service, include: Multi-cloud data distribution Secure for sensitive data Designed for developer productivity Reliable for mission critical workloads Built for optimal performance Managed for operational efficiency To be clear, JSON documents are the most productive way to work with data as they support nested objects and arrays as values. They also support schemas that are flexible and dynamic. MongoDB’s powerful query language enables sorting and filtering of any field, regardless of how nested it is in a document. Moreover, it provides support for aggregations as well as modern use cases including graph search, geo-based search and text search. Queries are in JSON and are easy to compose. MongoDB provides support for joins in queries. MongoDB supports two types of relationships with the ability to reference and embed. It has all the power of a relational database and much, much more. Companies of all sizes can use MongoDB as it successfully operates on a large and mature platform ecosystem. Developers enjoy a great user experience with the ability to provision MongoDB Atlas clusters and commence coding instantly. A global community of developers and consultants makes it easy to get the help you need, if and when you need it. In addition, MongoDB supports all major languages and provides enterprise-grade support. Why DataSwitch as a partner for MongoDB? Automated schema re-design, data migration & code conversion DataSwitch is a trusted partner for cost-effective, accelerated solutions for digital data transformation, migration and modernization through a modern database platform. Our no-code and low-code solutions along with cloud data expertise and unique, automated schema generation accelerates time to market. We provide end-to-end data, schema and process migration with automated replatforming and refactoring, thereby delivering: 50% faster time to market 60% reduction in total cost of delivery Assured quality with built-in best practices, guidelines and accuracy Data modernization: How “DataSwitch Migrate” helps you migrate from RDBMS to MongoDB DataSwitch Migrate (“DS Migrate”) is a no-code and low-code toolkit that leverages advanced automation to provide intuitive, predictive and self-serviceable schema redesign from a traditional RDBMS model to MongoDB’s Document Model with built-in best practices. Based on data volume, performance, and criticality, DS Migrate automatically recommends the appropriate ETTL (Extract, Transfer, Transform & Load) data migration process. DataSwitch delivers data engineering solutions and transformations in half the timeframe of the existing typical data modernization solutions. Consider these key areas: Schema redesign – construct a new framework for data management. DS Migrate provides automated data migration and transformation based on your redesigned schema, as well as no-touch code conversion from legacy data scripts to MongoDB Atlas APIs. Users can simply drag and drop the schema for redesign and the platform converts it to a document-based JSON structure by applying MongoDB modeling best practices. The platform then automatically migrates data to the new, re-designed JSON structure. It also converts the legacy database script for MongoDB. This automated, user-friendly data migration is faster than anything you’ve ever seen. Here’s a look at how the schema designer works. Refactoring – change the data structure to match the new schema. DS Migrate handles this through auto code generation for migrating the data. This is far beyond a mere lift and shift. DataSwitch takes care of refactoring and replatforming (moving from the legacy platform to MongoDB) automatically. It is a game-changing unique capability to perform all these tasks within a single platform. Security – mask and tokenize data while moving the data from on-premise to the cloud. As the data is moving to a potentially public cloud, you must keep it secure. DataSwitch’s tool has the capability to configure and apply security measures automatically while migrating the data. Data Quality – ensure that data is clean, complete, trustworthy, consistent. DataSwitch allows you to configure your own quality rules and automatically apply them during data migration. In summary: first, the DataSwitch tool automatically extracts the data from an existing database, like Oracle. It then exports the data and stores it locally before zipping and transferring it to the cloud. Next, DataSwitch transforms the data by altering the data structure to match the re-designed schema, and applying data security measures during the transform step. Lastly, DS Migrate loads the data and processes it into MongoDB in its entirety. Process Conversion Process conversion, where scripts and process logic are migrated from legacy DBMS to a modern DBMS, is made easier thanks to a high degree of automation. Minimal coding and manual intervention are required and the journey is accelerated. It involves: DML – Data Manipulation Language CRUD – typical application functionality (Create, Read, Update & Delete) Converting to the equivalent of MongoDB Atlas API Degree of automation DataSwitch provides during Migration Schema Migration Activities DS Automation Capabilities Application Data Usage Analysis 70% 3NF to NoSQL Schema Recommendation 60% Schema Re-Design Self Services 50% Predictive Data Mapping 60% Process Migration Activities DS Automation Capabilities CRUD based SQL conversion (Oracle, MySQL, SQLServer, Teradata, DB2) to MongoDB API 70% Data Migration Activities DS Automation Capabilities Migration Script Creation 90% Historical Data Migration 90% 2 Catch Load 90% DataSwitch Legacy Modernization as a Service (LMaas): Our consulting expertise combined with the DS Migrate tool allows us to harness the power of the cloud for data transformation of RDBMS legacy data systems to MongoDB. Our solution delivers legacy transformation in half the time frame through pay-per-usage. Key strengths include: ● Data Architecture Consulting ● Data Modernization Assessment and Migration Strategy ● Specialized Modernization Services DS Migrate Architecture Diagram Contact us to learn more.