Here yesterday, you'll know this track is focused on how organizations choose to adopt MongoDB, why they started that journey, what it looked like. We had seven great speakers yesterday. We had six industries represented. And we heard a series of tales and journeys and struggles about what it looked like. But in the end, all these companies had two things in common. One, a vision, and two, the willingness to act upon it.
Some really great things came out from the UK government, shared with us their decision to move away from having outsourced providers with off the shelf packages to bring that in house, and what that looked liked to develop with MongoDB up to and including awards on their gov.UK site. We heard Citibank talk about the quantum leaps that they made in global data usage across the globe. We heard the Gap talk about building a supplier management system in 2.5 months. Amazing stuff. Today promises to be just as good. Really excited about hearing these people with a vision and a willingness to act.
This morning we've got three great talks. Talks about what it is to cure cancer-- imagine that. Falling in love, for all of you who are looking for the dream partner of your life, is coming at the end of the session this morning. And now we're going to start with the Veterans Affairs Department, a great story. But first I'm going to ask, one, could you please put silent on your mobile phones. Two, at the end of the track, if you could please fill out the survey forms. They're at the bottom of the MongoDB world app. If you don't know what that is, I'd be happy to tell you later. Each and every time you fill in that app, that survey question, you get an opportunity to win an Xbox. So please do that.
And now, I have the pleasure of having Mr. Joe Paiva here. He's the chief technology strategist at the VA. He Brings more than 20 years of experience to his job in the IT field. A third of that is in the private sector, a third of that was actually working within the US government, and a third of that is within the military. He's brought in MongoDB to that. And I'm really excited to hear what it looks like for the VA to use MongoDB. We'll do questions at the end, so thank you for joining us. Mr. Paiva, thank you for being here today.
Thanks. So first, thank you for making me-- [APPLAUSE] --thanks for making me about 15 years younger than I really am. So I like that best of all. So let me just-- a couple points for today that I just-- I'm a government guy, couple things I have to say up front. Nothing I say represents the position of the US government Administration Department of Veterans Affairs. I'm just speaking for me, and nothing I say is meant to provide any type of endorsement or recommendation for any product, service, contractor, company, for profit or otherwise. Right, so just all that up front just so we're good. The other thing is I only have about four or five slides, so I'm hoping there's lots of questions. And I love getting questions right in the middle of my slides. Interrupt me, raise your hand. I very much prefer a back and forth to a one-way dialogue.
So with that, we'll kind of dive into it. The first slide here-- I can't read that. But it doesn't matter. The point of the first slide is to give you a little background on the VA. The VA does about, we have about $3.4 billion a year IT spend. About 8,000 people on our IT department. We have about 24 million customers, about 380,000 very, very dedicated, loyal, hardworking employees who are really-- if you ever get a chance to get out there and see some of these people and facilities, they are the most hardworking, dedicated, wonderful people you could ever meet. And the 24 million primary customers are veterans, but we also service their families and other beneficiaries.
And we do that in about six lines of business-- and just making sure. OK. The right slide's up there, right? We have about six lines of business that cover everything. Everyone knows about our health care line of business. But we also do insurance, we do education, we do home loans. So it's weird. For those from different countries, the US has this unique industry called death services. And we're actually one of the biggest players in the market in the death services business, which is actually a huge, multi-billion dollar business in the US. Only in the United States can you actually turn dying into a profit-making business.
What most people really don't know about the VA is that we are one of the biggest open source NoSQL shops in the world, bar none. Commercial, government, otherwise. Almost 20 years, ago the Department of Veterans Affairs built something called Vista. So essentially, it's our electronic health record system. But if you're familiar with commercial electronic health record systems, it's much more than that. It's actually, in addition to health record system, it's basically every clinical application you would use in an acute care facility or an outpatient facility, so it's this huge suite of clinical health care applications. And by virtue of being government, there are actually people who did FOIA-- freedom of information-- request, got that application.
And now there are multiple, open source distributions of that application being used all around the world. There are some big companies, I mean big stock market trading companies, who literally did a FOIA request to get that software, took that software, repackaged it as commercial software, and are out there selling it today making hundreds of millions of dollars of revenue each year.
So this is why we used health care software. And it's all built on the granddaddy of NoSQL databases, something called MUMPS, which is-- MUMPS is to Mongo like jazz is to rock and roll. You just got to go far enough back in the DNA climate. Couple burning platforms for us-- from a technology perspective, just to finish the thought on MUMPS. The burning platform isn't that the MUMPS database is so bad. It's just that people that code MUMPS might watch Fox News. They're all going to be dead soon. So we have this problem, right? We need actually a database that's going to be sustainable 10 years from now. That's kind of the technology burning platform.
From a business perspective, we have four kind of things I always talk about. This growing and evolving cyber threat-- I'll pause here to say I would argue that some version of these four things should be on the top of every CIO, CTO, CTS's mind right now. I live and die, because of my military affiliation, I live and die in the cyber warfare world all the time. And I can tell you that cyber warfare is just like regular warfare, right?
There was once upon a time, in the United States, we were the only people that had a big airplane or a machine gun or an atom bomb. And now everybody has them. Cyber weapons are just like those weapons. They proliferate. You're never the only person to have them. The difference is they proliferate much faster at a much lower cost. So things that we see as threats today, that are only threats for one or two natio-state kind of organizations that have the capability to do these bad things. Six months from now, sixth graders will have them, right? Kiddie hactivists, Russian mobsters, whoever. Everyone will have them. And so there's this continuously evolving threat.
The second part of this is-- I want to make sure to do it in the order that's on here-- is the transformation of the health care industry. So all industries transform, but the health care industry is undergoing one of the biggest transformations you can possibly imagine right now. Historically, health care has been based on the delivery of clinical services from a fixed geographical point. And I come from the-- most of my experience is in the hotel restaurant business from an IT perspective.
And the original electronic health record systems sold to hospitals were just like property management systems sold to hotels. They were all about billing. It was all about how do you make sure you bill for every ancillary service that someone gets while they're in the hospital. And then the stuff for outpatient stuff was how do you quicken the pace so you can so your urologist can see you in three minutes versus six minutes or six minutes versus 12 minutes. That was kind of the genesis of software in health care. Over time, the risk guys got involved. And they said we need to look at drug-drug interaction, how do we reduce mistakes.
But what's happening now is totally different. What's happening now is a complete transformation. Instead of being about the delivery of clinical services, the health care industry is about the delivery of health management. And instead of being focused on delivering those services from a geographical point, they're delivering those services wherever the patient may be on a 24 by 7 basis. My son is a paramedic in DC. He's got essentially an iPhone case that he hands to someone.
They hold it, and their EKG is electronically transmitted to the ER in real time. I mean, that kind of stuff-- we talk about the internet of things and how much more data you can collect. Because sensors got so cheap because of the Wii and telephones. People are making slippers now for old people so they don't get lost. Yes sir?
So do you have IOT stuff on your road map? Are you guys gonna pilot [INAUDIBLE]
When you say IOT, tell me what you mean. Just so I know.
The concept of wearables, for example, with an economy that--
Oh, absolutely. Absolutely. That absolutely-- there's no-- the internet. Oh, you meant internet of things. Yes, yes. The internet of things. Boy, you stumped the DC guy with an acronym. That's pretty good. The internet of things is definitely huge for us. Because we call it patient-generated data, machine-generated data, but essentially what happens is, in the old days, just take the EKG for instance. The old days being all of about a year ago. And still today in most rural hospitals you get an EKG, someone plugs you in, they put on things, you get an EKG. And then it's a piece of paper that goes in your record, right?
Someone has to look for the EKG piece of paper. Now all of a sudden you're plugged-in, you have an EKG that's continuously being transmitted all the time, anytime you want, right? With this thing called community-- I don't want to get too much into this, but community health workers can do a lot of stuff on site. I was just about to talk about wearable devices for people who may be early-stage dementia. So if they somehow get lost or disappear or end up horizontal at a time when you expect them to be vertical, that stuff will fire off alarms.
So there's this, just complete difference in the way we are going to be delivering health care. And whether you believe all health care is going to be delivered that way or 10%, as the IT guy it doesn't matter. Because if I'm going to deliver even 1% of health management services in that model, then 100% of my information systems have to support it. So I won't go into all this because we don't have a lot of time. But the rest of this is kind of stuff that you guys are all used to, right? You want to get more bottom line out of less top line. Or you want to get more bottom line out of top line that's not growing as fast as you want the bottom line to grow. So that's kind of-- that's the universal thing. So one of the-- before I had this job, was I was the acting director of infrastructure and then the deputy director for infrastructure in Net Ops at the Department of Defense. And one of the cool things about having a $9 billion IT budget is you get a lot of smart people to come talk to you.
And we did. We had people from Microsoft and HP and Intel. We had the 12 people in the world that actually write iOS level-- and I don't mean Apple iOS. I mean real iOS on the chipset operating systems-- come and work with us on security infrastructure and how to design this stuff. And this is a brute oversimplification. But essentially, here are the things that you have to know about what we laid out for the future of the enterprise is we said a couple things.
One is, because of the cyber threat we can no longer assume-- no one in this room. If anyone in this room believes their internal network is secure, you're incompetent. Period. End of story. And I don't care where you work. So NSA learned that the hard way. There's no such thing as defense against the insider threat. There's no such thing as a completely secure network.
And so what we said in our going forward model for the DoD-- and it was adopted by the VA-- was, we're going to treat the network, no matter where you get on it, no matter who you are, we're going to assume the network is compromised. Not because we think it is, and I'm not saying that the VA's network is compromised or the DoD's network is compromised. I'm just saying that no network is impervious or invulnerable to compromise. So this actually worked out well for us. When we came up with this design three years ago, four years ago, I guess now, it was kind of before BYOD was a big deal, right?
But it actually makes bring your own device and work anywhere a lot easier. And the reason I was actually able to sell this wasn't because anyone supported my wonky ideas about security. It was because they all recognized it was a way to get to this point where any person, any authorized person on any device, could get to the information they need to do their job. And so that's what this was really all about. It was when you look at this, what, hopefully, you see is, it doesn't matter whether you're an internal or external network, you go through the same security infrastructure.
The other thing you, hopefully, see on this is that apps are completely separated from data. And this is where this talk comes in. We realized early on that we could never afford to build all the apps people want. And we would never, within the IT world, have really the expertise to build all the apps people wanted. Plus, we wanted to avail ourselves of this big open source community of which we were a part. And so in order to get there-- what this doesn't do a great job of envisioning and the next slide is more technical. But it's still not too technical. But we wanted a single data store. Not physical data store, but a logical data store, this one data store, where all of our data would go. And we recognized that it's going to be a lot of different types of data, a lot of different types of data stores. But this one, logically federated data store managed by one organization that had a layer of data services on it.
And my goal with this was-- I used to say to people we paid millions and millions and millions of dollars to have all these people lock away our data in this tidy, neat, little, third normal form relational databases. And then we pay these other people hundreds of millions of dollars to find a way to get it back out. And it just didn't make any sense. We would have people that would go to these scrums, right? And then we would do these scrums, these oscillates, and people would come up with these HTML5 and JavaScript wire frame like apps. And the customers would be all excited. They'd be thrilled. Oh my gosh! We just spent a week with these guys and we fell backwards off the desk into people's arms and presto, bingo. We have an app.
And then weeks would turn to months. And months would go by. And the customers like, did that app that we-- what happened? Where is-- what's the deal? And we'd be like, well, that app was just the front end. We need to actually figure out how to connect it to the databases. And it's got to be connected to 10 databases. And we have come up with all the schemas, we have to reconfigure all the control devices. And we have to go through all these IA approvals.
And we have to go through all these configuration change management boards. And we've got to map this data to that data, and blah, blah, blah. And by the time you're done, that great app that had everyone all pumped up is just sitting around for three, six, nine months and you can say, yes, we're agile. We are a great agile shop. And we deliver great incremental capability every 30, 60, 90 days. Incremental capability does not impact your bottom line.
When you need change, you need change. You want all the change, right? If I'm sparring with my son who's getting too big to spar-- both of them. I liked it better when they were smaller and I could hit them harder. Just getting in a punch here and there, those incremental punches doesn't do it, right? He's beating the crap out of me. I need to hit him hard right? And so agile is cool. But when you need a knockout punch, you need a knockout punch. Not a jab, right? And so sometimes, you have to really be able to deliver. And so what we wanted was a way to be able to go from this-- we did this wire frame to act much, much faster.
And so it was in that backdrop where Will and Chris [? Saik ?] and Will [? Laforce ?] came to my office about two years ago. We're sitting in this little room with a white board, my favorite room. It's just a little room full of white boards and glass. And people sit around and think thoughts. And we kind of-- I don't know who came up with it first. I know a lot of other people came to the same conclusion, so it's not a unique thought. But we came up with this idea of a schema agnostic enterprise CRUD service.
And the concept was really simple. We would have an enterprise service that we would give you the code snippet for, and you know, just an ajax call, and you would just save whatever it is. We didn't care what was in your form. You'd just save it. If it was a spreadsheet you just saved it. If it was PDF, we'd just save it. We had five variations of this thing that we ended up coming up with. We came up with this standard kind of thought about it where we have a document. Up front we'd have the document demographics, if you will, which was some metadata or some key value pairs that described the person it was about and the key information about that veteran.
And then the second part was some key value pairs that were standardized that talked about what kind of data this was, essentially. So we kind of made those two things mandatory. So no matter what kind of document you were doing, you had to have these two parts-- this demographic section and this document. We called it document provenance. Just made it up, it sounded good. But it was just stuff to kind of say this is the kind of document it is.
And then the payload section of the document where it's whatever you want. And the idea was that we would launch this service, this schema agnostic service, and then people could just go out and code whatever they want. They could be 90-day wonders, the clinicians that code stuff in HTML5 and JavaScript. And as long as they know how to do an ajax web service call, and they grab this snippet and they call, they do this restful call, and then we have-- I'll come to that in a minute. There's some other things we've got to do, but essentially, the app works. It'll save whatever you want.
I don't know how many other people in the room have done this concept of a schema agnostic web service call. You've done it? So I'll tell you the problems I ran into. And then please jump in and tell me if you ran into the same problems. The problems I ran into-- not problems. I would say the challenge is we had to get the second part of that. The technology doesn't solve the business end.
If you have everyone doing whatever they want, and everyone just starts uploading documents, even though you have these three standard sections and they can upload whatever they want, if you don't have someone govern that middle section, the provenance section, and decide, these are going to be the allowable document types and we're gonna agree to what key value pairs we use. So if I say eyes and the allowable things for eyes are brown, blue, green, hazel, whatever. You can't say eyes and then use B R N, B L U and bloodshot or whatever. You kind of have to come to some agreement. Not that the technology can't handle the disparity. But because it just gets ugly from a contextual perspective building applications, right?
But we figured that out fast. The long and short of it is we had the first version of this service up and running in like weeks. It was that fast. It was literally that fast. We stole some stuff off GitHub-- stole, borrowed, whatever, copied. We used some stuff off the jQuery.com libraries. We used some of that stuff. But we had this thing up and running in a matter of weeks. And it works today. Literally today.
While there's still some things to iron out, this is on GitHub. If you go to the VA's GitHub, this thing is right there. You can take it and use it. If you install this schema agnostic CRUD service you can have, you can build whatever you want. You can build an HTML5 or a JavaScript app. And as long as you know how to do an ajax web service call, it will take it. And it will store whatever you want. CRUD-- create, read, update, delete. So we never delete, but we do create, read, update. Create, update, easy. Read, we had to think about for a while.
And so if you look at this slide, you'll see what our goals-- I'll come back. I've kind of already talked about most of that. But the circle at the bottom is an important part of the slide. For the three people in the room old enough to remember object-oriented programming, that's what this diagram is modeled after, right? And we decided of take that paradigm and use it in our NoSQL implementation. And based on our enterprise services implementation, we decided that what we would do is, we'd have this layer of enterprise services. And some of them would only be worked on at the core center. They would be universal, and everyone would use them.
So like, for instance, all the different security services relating to authentication, access control. Authentication, authorization, auditing, all but-- there's a whole myriad of access control related services that we have to do. We put those in the center and say, this one group does those. You can't have anyone else do it. And it actually had to come to a point where I disallowed projects from building their own security.
When they came in for the system design-- at any given time we have 300 or 400 active projects. When they came in, we'd tell them, no. You take that out. We're taking that money back. You can't use it on that. You cannot do your own security infrastructure. You have to use the enterprise security infrastructure. You can't do your own identity management. You have to use the enterprise identity management.
We eventually-- and this is where the blue circle comes in. Getting your customers to want to fund-- You gotta be kidding me. Really? Getting your customers to be willing to fund enterprise services is like field of dreams kind of stuff. They want new capabilities, new applications, right? And so from that perspective-- thanks, man. Where was I? Oh. You can't get people to fund just enterprise CRUD services like field of dreams, they will come, right? So what we ended up doing is creating the blue circle that said, OK. You have a portfolio for health or a portfolio for death services. You guys create these enterprise CRUD services for your portfolio. And anyway that's what this diagram is meant to show.
I, apparently, have spoken way too long. And have run out of time. We have five minutes. Before I talk anymore, does someone have questions? Please, please, a question. So how do you enforce that? At what level do you start enforcing that this is how it's going to be done and this is how we're going to do it? Because eventually, when you start going from your centerpiece all the way out, you have the ability for people to go rogue. And how do you how do you basically stop them from doing that?
So there were two things we did. One was kind of-- I gotta be quick. One thing we did is, we have this concept of enterprise design patterns I implemented where I invited everyone via-- we have Link but the same idea as WebApps-- we crowd-sourced the design patterns. And we said, everyone join us in developing the design patterns, even contractors, outsiders, other government agencies, academia. Help us develop the design patterns.
So I actually got all the people who write the apps to believe in the design pattern upfront. The other thing we did is we put gates in place where I get veto authority over-- my staff exercises for me-- veto authority over system design. So we exert, then, a line item veto during the design process. And some, like it's negotiated. I mean there's still people that go rogue. And sometimes we need to give them that flexibility. But it's a process. Yes, sir? Someone back there earlier? Whoever's next.
So what are the things, like AppKit-- how has that kind of influenced where you guys are going with iOS devices now? Talking about analytics on health care. And are you guys newly inspired by the internet of things moving with devices and whatnot?
So I'm not sure I fully understand the question, but for time's sake I'll try to say what I can. We're totally into the internet of things in terms of we know we're going to have lots more medical devices. My general policy, and what I've implemented, is that you can only use like Xcode or device-specific applications for things that there's absolutely no other way to do. And there's a convincing business case, because otherwise, my position is everything is Bootstrap, Twitter, pure HTML, and JavaScript. And we only use device-specific coding when absolutely necessary.
What about your data warehouse, a data [INAUDIBLE] concept, analytics? I didn't see that in the diagram? It's--
We have multiple data warehouses, as you might imagine, right? And we have not started to-- taking things in bites, I'm letting the existing data warehouses continue to exist as they exist today. And then we're kind of addressing that slowly. For me the fire storm was the apps we're building now. So I had to focus on where there was already a project in place for enhancement for a new app and I did try to go back and invest in something that was currently operational.
Ok, we have time for two more questions. And then we can do them in the back of the room at the end.
Yeah. Go ahead. I'll be happy to answer questions in the back after.
So Joe, what is-- so from an architectural perspective, like have you guys published a set of guidelines as to exactly which type of database technology you use for what type of application? And what are your guardrails, in terms of what you're telling your application development teams?
Great question. I have some very rough guidance I've put out. And it's not solid yet. And that is so far what I've said is if you use a NoSQL use our enterprise NoSQL, I've prohibited individual teams from buying databases. So at the enterprise level, where we're trying to cull down from 17 different titles to something less than 17 different titles. And so by centralizing all our database purchases, we're hoping that'll take care of itself. But I am interested in anyone's good guidelines for how to decide document database versus key store versus SQL. It's not really clean in my mind yet. But--
Isn't that Will's job?
MongoDB.
MongoDB, MongoDB. Legitimately right we're going to be a SQL and NoSQL shop for a long time. What we're trying to do on the NoSQL side is never get to the point where we have 17 NoSQL databases like we do SQL databases, right? So that's why we have this enterprise CRUD service built on top of Mongo and the reason quite frankly that we went with Mongo is because it's open source. And it has this huge community and ecosystem that was for just being candid that was the biggest differentiator in my mind because I have to look at long term cost of ownership and sustainability and we were just looking at the community and the open source piece of it. And our overall commitment, remember there's probably a billion people in the world get health care based on our electronic health record system. So not having that open source would be a problem in places like Africa and all that.
Joe. Wes [? Selestin ?] I'm from Commonwealth Gay Alliance. So my question is regarding the service layers and the data layer. Do you have any specific architecture you're using as far as the ETL or enterprise service bus to basically orchestrate all these different services?
We are. We actually have a couple. The one that we're going forward with right now-- and this is again not an endorsement. I don't endorse any particular product. We just happen to own a lot of IBM, and so we're using IBM WebSphere and Worklight in that layer for our big new applications. And it's working. It's working well for us.
But we also have other products. Because we have Talend and some other products, as well. But for our major new stuff, we just happen to own this IBM stuff, so it's working well. So there's return on net asset value versus return on investment. Sometimes the government doesn't understand that. So I'm trying to teach the government about return on net asset value. Use what you own before you buy new stuff.
OK. Thank you so much for your time today, sharing with us one of the largest providers of NoSQL technology in the world. Even if you don't endorse anyone. In general, they're great and has taken you down into weeks of development as opposed to months and years. Thank you so much for joining us. Let me remind you to please fill in your survey application for questions. In 10 more minutes we'll hear a great story from Sanofi and how they're looking at driving change in cancer research. See you soon.